Young horses tend to be quite sensitive, As they are still figuring out where their own legs are. try when your ride looking over the outside ear it will keep you balanced even when the horse isn't, encouraging your horse to flex his neck to the outside will stop him from falling in and being unbalanced. And of coarse lots of riding so he gets a chance to be more relaxed and figure out what your asking and how to balance himself.
Circles , surpintines, and pole work
Ride a lot of bareback and trot poles so when you get in the saddle you are super ready.
Circles & trotting poles helped my mare a lot back in her day