-
I wear very little, but still carry the kit and then put it on when I need it, like descents or stops.
My missus thinks I'm a mental but I'm normally running hot when riding so I can deal with most descents without even bothering to put on the layers - I use the downhills to dump heat. Depends on terrain and conditions, obviously but climbing in sub-zero I could be in summer kit for all the heat I generate.
What if your body has an amazingly efficient cooling system and you sweat loads at the hint of exercise or warmth or chilli?