-
Use a decent exterior filler and finish it with a sponge and then don't sand or use a higher grit. Wiping/hoovering/rinsing the surface can help too. There are a lot of factors with exterior paintwork if you want it to last 10 years or more.
One case where you can get problems is using Toupret Fine Filler on top of the primer or 1st coat. If it's interior walls you can use something like Gardz. Dulux do a very thin primer for their weathershield system which is good for any porous surfaces.
Had a bit of a lightbulb moment. You can get some early failure of acrylics on exterior surfaces if you have a lot of fine filler dust on the surface. Happens occasionally if you go a bit overboard with the filling and sanding.