You could make the carradice more of a 'scoop' which is basically what the PR is. It all depends how off road you're going. I think panniers would be fine for south downs, but anything more mad I'd be tempted to go for a dry bag strapped along top or maybe heavily strap down your panniers to stop them rattling and bouncing. The other disadvantage is that panniers may it harder to go down narrower trails.
Panniers have gotten me over South Downs a couple of times, but I want to go back on the krampus and take some of the more fun sections a bit faster :) There aren't many narrow bits unless you head off into the woods in the country parks.. might just work out a frame bag.
You could make the carradice more of a 'scoop' which is basically what the PR is. It all depends how off road you're going. I think panniers would be fine for south downs, but anything more mad I'd be tempted to go for a dry bag strapped along top or maybe heavily strap down your panniers to stop them rattling and bouncing. The other disadvantage is that panniers may it harder to go down narrower trails.