I do a lot of groundwork to get my horse paying attention to me. If he is anxious he wants to move his feet (flight response). Which I let him do, but I direct where his feet go and make it hard work (especially make sure to keep him out of my space). Eventually he figures out if he focuses on me I release the pressure. When he listens really well I give him a minute to rest and soak and relax. When he gets anxious I also desensitize him and release at the smallest sign of relaxation (so if he is moving his feet and stands still I stop whatever I am desensitizing him with and rub his neck, when he consistently stands still I wait for him to not twitch then I rub, once he is not twitching I wait for him to blink then I rub, and eventually he starts to lower his head and blink and sigh at the scary thing I am waving around). There are 2 important principles: 1. Only apply enough pressure that they are slightly anxious, they still need to be able to think to learn and 2. Do not release pressure until they are making at least a tiny step towards relaxation, if you release pressure when they get scared and try to run away they will learn running around makes the scary thing stop and will continue to run from the scary thing
First you have to set the pecking order with this horse to make them respect you and become submissive by using your body language. This way your horse will pay attention on what you're doing more than on his fears and anxiousness. Desensitize him to a lot of different things, make him learn that you'll never put him in a bad situation, in order to make him trust you and dealing with his fears, using the thinking side of his brain instead of the reactive side. He will get used to spooky objects, noises... And that will make him be more confident and less stressed. Always keep your horse focused on you, let him time to analyse, but don't let him avoid what you're asking.