It's not that hard. If you've done a workout indoors and leave or you only have a short commute.
ie. I leave the gym sweating like crazy so the first 20min riding after that I don't even notice it's cold. Give it an hour though and fingers start to snap off...
Followed a guy still wearing shorts and tshirt this morning. Not sure if awesome or mental! My ankles are still frozen.