Cottered BB/crank problem

Reviving an old thread, but with a slightly different problem.

I'm restoring an old bike, which came with a cottered Williams track crankset a previous owner won in a race. Irritating as cotter pins are, I'd like to use the crankset, but when I've put it all together the cranks aren't level. One is at 6 o'clock; the other is closer to 1 o'clock than 12 o'clock. It looks to me that the hole for the cotter pin on one of the cranks (the drive side) isn't quite level, and that that is throwing the alignment out. Before I

5) Throw the fucking things away and use cotterless cranks.
is this going to make a hell of a lot of difference, and is there anything I can do - filing? - that can rectify the situation?

Any advice much appreciated. Cheers.