I'm a small too and have the jacket in a small. I'd say if you expect it to be normal brevet or "club" fit then maybe size up. I think it's true to size to my other small stuff and a bit looser than some of my PT stuff (small race cape, gore tex jacket)
It's probably my favourite Rapha piece. Pretty warm for it's weight and breathable enough