Is it time to start calling out bad cyclists?

I tried the arm out, palm down, pat-the-dog Highway Code thing and my sense is it angers drivers, as if some crazy cyclist is telling them what to do.

I think there's a common misconception that this means "I think you are going too fast, slow down" rather than "FYI, I'm slowing". Years ago (before I drove or cycled) when I first saw a cyclist do this and asked a driver what it meant, that's what they thought.