I think the secret is actually your 'short, fast rides' rather than 'actual rides'.
I guess it depends on a lot of things, including road conditions and your definition of an 'actual ride' but I've done over a hundred miles on 32c tyres and a good 70 on 23c with Fizik tape with no bother. Just about to put it on my new road bike too with the intention of doing some 100 mile plus rides, again on 23c tyres. Do wear padded gloves though on longer rides.
I guess it depends on a lot of things, including road conditions and your definition of an 'actual ride' but I've done over a hundred miles on 32c tyres and a good 70 on 23c with Fizik tape with no bother. Just about to put it on my new road bike too with the intention of doing some 100 mile plus rides, again on 23c tyres. Do wear padded gloves though on longer rides.