I would do zig zag leg yielding on the centerline. Your horse is not properly through your legs when it doesn't leave the outside track really easily. Your horse feels "safe " on the outside track and always leads towards it which could be the reason why it is easier to do leg yielding in the direction to the outside track.
When you do the zig zag leg yield on the centerline you confuse him and make him listen to your leg aids instead of just decide for himself that he wants to go to the outside track.
Try always 5-10 meters in one direction and then change direction. Even if the leg yield isn't perfect the moment you change direction he truly follows your leg aids instead of going on his own. Whith this exercise you make him sensible to your leg aids which is the purpose of leg yielding.
Let me know if it works :)
It worked on many of my horses and also on the young ones that didn't have a lot of balance.
Ask for only one or two steps at a time. My horse is learning too, and I've found it help to do two steps of leg yield, then straight a few then two steps, then straight. As he becomes more supple and willing to move forwards and sideways, I'll ask for three steps etc.
be sure your horse isn't cheating his way out of the leg yield going to the wall. My guy loves to cheat his way through lateral exercises and tends to drift with his shoulders and I think I'm getting a correct leg yield, but it's actually not.