assuming the hub bearings are very smooth, which they should be for a wheel to spin freely, then there's very little resistance to any lateral movement. so, especially with weight on it, the trailer would move left and right all the time, probably knocking you off balance, no?
The left right movement around the middle plane is what keeps you upright on a bike.
I think that's how singletrack trailers work too but have never used one, so could be wrong.
I don't know, only found the pic, no description. Why would it be twitchy? The wheel just follows the bike, no?