Which components actually make a difference?

tires would be the biggest, then rims, then hubs imo

tires are massive, tyre size and also profile (relating to rims) can affect cornering, contact spot and rolling resistance.

hubs also, i've held good quality hubs and cheap hubs and spun them just holding the axles and pushing the hub body and there is a big differenece in how long they spin, novatec's being the best value if anyone's asking, they spin for almost as long as the expensive ones but are a fraction of the price

I noticed that with a hope hub, but do you actually notice the difference when riding?