From the older streetview images it is clear this was a separate plot
The view from the other side of the water.
Can you provide a current view straight on of the plot please?
My pictures you have provided are all featuring what in the past was the adjacent property. If it remains a separate property next to the plot then that is not a reason to remove. It is up to players to consider their own safety and abide by local trespass rules.