Using a brake on a non-machined rim

So as long as I don't mind the black marks around my front rim (it's red everywhere else) I can just ignore the fact that it's happening, as the black and the red will both disappear eventually leaving me some shiny silver?

Just leave it. Don't file it!