I use a flannel first and get off all the loose water ringing it out a few times in the process. Leaves less for my towel to do and less water evaporating away from the towel later - I live in an old stone wall terrace house so reducing damp air is a serious consideration.
I towel off after a shower, the water is clean, therefor towels are perpetually clean.
(n.b. I don't have a bath in my home)