if you do these square warm ups put cones and shape them as as a big square but make sure their for away from the wall it teaches your pony to balance. Hope this helps
^ I agree. Also if you ride the transitions on a circle by the centerline (if it were a dressage arena) and allow the horse to gather his balence in the transitions
Usually when horses tend to keep close to the side or the arena when cantering or even trotting for that matter they are struggling to balance themselves and try use the wall as support, especially if the rider's balance isn't good either. Some exercises to fix that is lunging over cavalettis, teaching the horse self carriage, riding out a lot in bigger areas where there isn't somewhere he can lean for support. After a few weeks go back into the arena and see if he has improved. For your own balance, pilates and yoga is fabulous.
Ride about 6 feet off the rail at all times
Try circle work or like @lautaylor
said with the 3/4 line. Good luck
Ride on the 3/4 line and leg yield him to outside track then back so he's not always on it then the habit becomes less of a habit.. good luck
I would try to use a little outside leg to push him out off the wall but don't go to the extreme where he is off the rail and in the middle