Limitations to training on singlespeed?

If you're training for time trials, you should probably be close to time trial gearing. It's surprising what hills you can get up on 88" fixed when the option is walking ;-) If I'm doing high intensity intervals, I'll be on my race gear or 1 tooth more at the back to get some spin into the efforts.
The problem is the SS is essentially my only bike for the majority of the time now. It has to be used for getting the miles in the legs, as much as for any specific training. I suppose I could gear up for 1 or 2 specific time trial training rides during the week, then use the 70GI for the 60 mile sunday ride.