It sounds a lot like the feeling I had with my low trail Pelago, when I took the rack of.
I'd recommend you try a heavier rack to (like the Pelago Commuter or maybe the Rasket).
That would provide a stabilizing weight while not relying on the load on the rack to settle the steering feel.
It might be worth a punt before you source a new fork - you can always sell the rack if it doesn't work out.
It sounds a lot like the feeling I had with my low trail Pelago, when I took the rack of.
I'd recommend you try a heavier rack to (like the Pelago Commuter or maybe the Rasket).
That would provide a stabilizing weight while not relying on the load on the rack to settle the steering feel.
It might be worth a punt before you source a new fork - you can always sell the rack if it doesn't work out.