The bit on the right is doable, but a bit shite this year (too many horses when the ground was soft), I usually come from the road, either north or south and turn into the farm, or vice versa if coming from the other side.
But I believe you should go through the church graveyard.
https://www.google.com/maps/@51.6690931,0.172236,371m/data=!3m1!1e3
So peeps are coming from the right and going straight up their driveway instead of turning right and then doing a left turn into the church path?