Most dangerous road users

I don't believe people who are permanent residents here should be allowed to drive here without passing the UK driving test.

Which is pretty much what the law says. If you don't have a license issues in the EU then you've got 12 months to take a test;

http://www.direct.gov.uk/en/Motoring/DriverLicensing/DrivingInGbOnAForeignLicence/DG_4022561