Brompton owners

... Is it something psychological when people see a brompton that they assume it'll be slow and they on a full bike are therefore faster and indeed superior or maybe because it's smaller they assume it's easier/more acceptable to push past?

I think drivers do perceive you to be more passable. I do find I take the primary position more on the Brompton.