To be honest, the fact that he does the opposite on the other side, and he does this enough that you say for definite what's happening, I would be inclined to think pain, or at least stuck/stiff somewhere.
If it's behavioural, then a schooling stick and leg, with rewards for any improvements. It's best to work with a good trainer for it.
I'd definitely try to rule out any physical reasons first.
I'm pretty sure it is behavorial, since he does the opposite on the other side and it's a particular corner he does this. Sometimes he will even slow down and try to turn around. It happens near the gate so he just wants to get out even though we barely did anything! But thank you for answering.