Yes, horses are very sensitive to noise, their ears have developed for survival. The best way to drive your animal is to introduce it in sound environments, to get accustomed. If you leave it in the stall and take it only in competition season your pet will not have how to adapt or accustom to the noise.
I once had a mustang, she spooked very easily. I hundred pie plates (the disposable aluminum ones) from trees in her dry lot. I would feed her under them. Between feed times wind would blow them. She became very calm, so calm my then 2 year old could walk her around by lead rope!
I think ear plugs may be counterproductive, as they may make your horse feel even more wary of his surroundings... I guess you could try and see... but I would instead suggest doing things that gradually increase his confidence (hacks, days out, whenever he spooks at something, take it slowly and give him time to get close, gradually he'll become less sensitive and more inquisitive)
It sounds like your horse may be unsteady by the noise of what he can hear but not see; get him comfortable with the noises by showing him the producer of the sound and show him it's harmless. As effective as ear plugs are, I recommend giving your horse more confidence with the noises that frightened him.
Try horse ear plugs. Personal, my horse hates them. But I know plenty of people who have success with it. Also feed can help. I changed my horse onto Teiple Crown Complete and it has help keep him more calm, because he use to not pay attention to me. He is much better now with paying attention.
Ear Poms are great for that. Fly bonnets as well. My pony likes to get distracted a lot so we use those, but we also use neck stretchers which is weird i know, but it makes him focus on what we’re doing instead of his surroundings.