-
The gist is this; sort out your saddle position first, this can make a big difference to how the rest of the bike feel.
If it's too high, you may feel your hamstring muscle getting pulled slightly, too low and you'll feel your quadriceps muscle stressed at the slightest effort.
Once you sort out the saddle position, you then can move onto the stem length, too long and you'll find you're straining your neck trying to keep your head up, too short and you may find it feel cramped.
This is a pretty simplified way of finding a good fit, but it help narrow down whether your bike is the correct size for you or not, if you wish you can invest in a bike fit to fine-tune it more.
5mm is a massive jump, it's usually best to do it in small amount, half a millimetre every now and then is a good way of fine tuning it.
miro's right, push it back and get a stem 10mm shorter.