How much is down to the bike?

Indeed. But one thing I've wondered is whether it matters whether weight is lost on the combination of bike and rider or the bike itself. Putting aside matters such as that a person with a fair bit of fat of them will tend to have some corresponding extra muscle, is shaving a couple of pounds from a bike any better than just losing a bit of one's own weight?
I've always assumed that a pot-bellied cyclist who spends £80 on a titanium bottle cage versus a steel one or whatever would get the same benefit by shifting a teeny bit of his own weight. Is this principle correct, or is weight on the bicycle more of an issue than weight on the rider?

Saving 50 grams on a bottle cage would equate to the same as saving 50 grams on your jersey. Saving it on rims/tyres/pedals? will have an effect that feels greater.