Why don't you play polo?

I'd have a go, but its just going for the first time issues, that and the fear of hurting myself or others through ineptitude.

I think that you shouldnt worry about injuring yourself or others at the beginning. Beginners usually start off slow and unsteady - you're hardly moving fast enough to do yourself or others any damage.