Gap at bottom of headset?

I don't want to damage anything either and always think it will be fine so continue to fit crown races with a hammer and screwdriver.

Never is fine though and always get a few marks on the crown race but so far got away with it and the bearings always spin perfectly!

Why not use a bit of wood, then you won't get any damage, touch wood.