Disabling the label together with the color button should make it easier to tell whether a particular color button is disabled or enabled, because without disabling the label a disabled color button looks very similar to an enabled color button.
Before:
After: