What about as a descriptor but not in a derogatory way? I'm a musician and people often refer to others as tone deaf if they can't actually tell what they are singing (for example).
I don't think there is another phrase that's used in that situation.
As an insult, yes, also applied to other such as blind as a bat, calling someone crazy, cripple, spaz, triggered etc.