Track or SingleSpeed Cranksets

Forgive my ignorance since I'm only fixing my very first single speed (freewheel not fixed) bike and I do not know many things in detail. I presume that something is different but I don't know what.
I'm 6 1" tall and my inseam is 35". I use 175mm crank arms on my mtb and my road bike. I'm trying to find a track crank set with the same length, but it is very rear. The majority of the track crank arms are 165-170 mm and even much taller riders than me use them.
Should I use a shorter length or stick with 175 mm? What do I loose/earn?

Thank's in advance for any useful answer.

People go for shorter cranks on a fixed, for a number of reasons. They are easier to spin at high cadence, reduce the amount of toe overlap with the front wheel, and reduce the chance of pedal strike while cornering. Having said that, I've heard of many a tall person using 175s, especially at your height. I was'nt aware they were rare.

http://www.businesscycles.com/trcomp.htm#crankset