Actually in most of the cases I've said that ('& where did it get you in the end?') they've conceded that it was indeed pointless, in at least a couple of cases they looked utterly crestfallen. I am blessed with height and what can look like a very angry face, that possibly helps.
I bet they really appreciate that.