I wouldn't rely on riding through the uneven and the lunges for too long. You don't want to develop any bad habits from compensating for the gait. Sounds like the horse needs some finishing and balance work. You could work on lunging in side reigns, practicing balance exercises with a trainer that ask the horse to use his/ her legs and back appropriately. My mare was the same way as a 4 year old and we did about a year of balance work, strengthening, and patience doing circles over and over. We heavily rewarding good, solid, balanced pace circles. I hope that helps even a tiny bit, best of luck!