I would recommend that you do work on a lunge as that allows for someone else to be in control (pick a quiet horse to do this on). Let your stirrups and reins go and just work on the feel of the horse and your position. Then slowly pick up the reins and stirrups.
It also helps to get a coach who you trust and who knows what you are going through. Start with basic flat work and gradually go up :) It helps to have a quiet horse who you can trust and that doesn't spook easily.
I would also recommend that while you are becoming more confident you wear a body protector (This really boosted my confidence when riding younger horses).
One final thing, work in small amounts. Don't expect yourself to do a 2 hour ride perfectly. Strive for improvement in little amounts and most importantly BE PROUD when you do something good.