I do wonder, if without a rinse (or a vacuum) you're just pushing the crud around the record with a cloth, rather than removing it. Anyhoo, there's no right or wrong, as the long as the record sounds fine, it's all good.
That must be why the recommended technique on rotating cleaners like the disco antistat is to rotate only in one direction many times, a couple of dozen or something.
I do wonder, if without a rinse (or a vacuum) you're just pushing the crud around the record with a cloth, rather than removing it. Anyhoo, there's no right or wrong, as the long as the record sounds fine, it's all good.