Uhh with a 80mm stem and feeling too long, then you are probably onto 54 with 80mm-100mm stem.
If you plan on doing cyclocross races, then your lower back would enjoy a super short setup. On asphalt bikes I race 56 frames and 130mm stem, but in cx racing I'm doing 54 toptube with 70mm stem.
I'm sure you can ride anything from 54-58 and be happy, but since you are used to 56, then go ahead on that size.