Spending more time with your horse in general tends to strengthen the bond between horse and rider. You want your horse to respect you, to have their trust. Nowadays, we see all of these riders that can bring 4-5 horses out ( such as Double Dan Horsemanship ) and have them all work to his will. You need to assert yourself as the herd leader, and earn your horse's respect from there. Even things such as lunging sessions, or grooming sessions would help bring your horse to trust you.
If you have a roundpen try lunging without a lunge line and keep him working until you see signs of him giving into you. Like licking his lips, moving his ears into focus on you, etc. Once you feel comfortable and you feel that he's ready stop lunging turn around and take a couple steps away. Don't turn around to look at your horse for a couple minutes let him come to you.
Spend lots of time with him! Work on trail or throw some scary obatacles in the arena! It will help build his confidence and trust.
Horses have 2 main goals in life. To reproduce and to survive. You part with him is about survival. He needs to know that you are the rider he can count on. He needs to know that you are not part of the danger. To give him carrots or telling him he's a good boy doesn't work for him. You need to impose your space on him. You must make is feet move and not you move away to gain respect. After it's all about the quality of riding you can offer. Horses respect quality o riding. BTW difficult horses are normally the best ones. Be patient.
spend more time with your horse, like grooming more and be gentle