Any question answered...

Hi all - I've recently become the occupier of some land purchased by a family member - I oversee and manage all aspects of it. Prior to this the previous owner, through particular circumstances, ended up allowing the occupier of the next field whose horses had broken down his fence and used that (now 'my' land for ease of understanding) land pretty much as their own and because the previous owner basically didn't use the land she just needed up allowing (or not stopping him) from basically doing what he wanted with it, without charge or sticking to his promises of maintaining, fixing etc etc. Anyway, I finally managed to catch him in person and he was clearly disgruntled about there being new owners albeit wasn't oppositional. We are seeking (planning going in) to use the land as a secure dog paddock and told him of our plans and that we are not responsible for securing his horses and we need to get work done to clean up the land - not least remove the rubbish, scrap material and rotting vehicle that's currently on it. He says he can't afford to fence it securely (about 130 meters) so is looking for a charity to take the horses and then relayed sob story after sob story before also saying he was mortgage free and wished he'd known the previous owner was selling it as he'd have found the money to buy it. It is his responsibility to fence his land aside from his responsibility to ensure his animals don't pass onto or damage others' land (which they are).
Anyway, the access to both pieces of land is part of my title - he simply has the right to pass over by foot or vehicle - to enter and egress from his land and he insists that a front gate (that sits across my land) is shut and securely locked at all times - interestingly the front gate also has a plaque with the name of his house on it marked X on the plan. Now, call me neurotic and cynical but is it possible he is (well certainly was) seeking to claim adverse possession of what's now my land? I suppose my presence and the fact I'm applying for planning have lots of records and pictures of myself on the land and records of meetings of contractors etc for improvement works evidence that it's mine and i'm using it. It's just unnerving that he is refusing his responsibility of securing his animals and boundary and is trying to give me conditions and asking me to ensure the gate is locked etc. He is also from a well known traveller community and in my experience, many within that community have a 'greater than typical' understanding of issues relating to land possession etc. The previous owner was completely oblivious to any potential risk and as he has now owned the land for 9 years (seemingly just a year short of a potential adverse possession claim). Also he has put two baths for water for the horses (which he pumps from the steam on his side) on my land, literally just the other side of the boundary when there seems no logical reason whatsoever for him to station them there. Anyway - apparently he is going to be clearing his shit, paving stones, tyres, wire, timber and horse box etc off my land at the weekend and I'll be there to help him as he said he had a bad back. It is absolutely our inclination to ensure we manage our land as we wish, never encroach on his and ensure his right to access/exit his own land is in place (although this is subject to him paying 50% of the maintenance upon demand which obviously won't happen). Maybe I'm reading too much into it but he certainly seems deflated when I turned up with a friendly smile and handshake. Any advice greatly appreciated. Here's a simplified plan for context my land in red and his in blue.