Without doubt I've come to the same conclusion. There are aspects of the HiFi game that don't make any sense at all. Listen to an album like Harry Belafonte at Carnegie Hall and you have to wonder what we've forgotten about recording techniques in the past 60 years.
I'm convinced that the original recording and any digital remastering has more impact on the perceived sound rather than the bit rate.