I suspect that as @M_V suggests whacking it with the hammer means you've pushed it/the wedge into the butted section at the bottom of the steerer, as shown in the diagram below:
So I'm also wondering if the new stem was longer than the old one?
If you have hammered the stem into the butted section you have to come at it from below, that's the only way that will work. I know because I did exactly the same thing with a quill adapter, and I am stupid. I managed to get mine out by going in from below with a long quill stem bolt and whacking it repeatedly with a hammer.
I suspect that as @M_V suggests whacking it with the hammer means you've pushed it/the wedge into the butted section at the bottom of the steerer, as shown in the diagram below:
So I'm also wondering if the new stem was longer than the old one?
If you have hammered the stem into the butted section you have to come at it from below, that's the only way that will work. I know because I did exactly the same thing with a quill adapter, and I am stupid. I managed to get mine out by going in from below with a long quill stem bolt and whacking it repeatedly with a hammer.