Being a Lord does make these kind of statements a little awkward (although he was Dave until recently).
That is the view of a large number of traditional labour voters though. If you're in a £50k 2-up 2-down you're going to view someone living in a £1m house in Islington as pretty well off.
best punchline ever.