How to Constrain Panning

When you have a plot, the pan feature is very nice. However, it’s possible to pan very far away from what has actually been plotted. Is there a way to constrain plotting so that a user interacting with the plot hits a wall that they can’t pan beyond?

I’ve also asked in stack overflow here.