Definitely agree with Rhianna Jayne it's all abt being steady with your seat and relaxing all your muscles! I have a spooky horse who flips out at slightest movement or sound!
I would walk for a while till he gets relaxed and when you start to trot let him do what he wants but at the trot. I'm not sure if this will work for you and your horse but it worked for mine.
Circles always seem to relax my horse :)
Ride with your seat, you relax your muscles and he will do the same, it's all about keeping steady, :) tuck your bum underneath you so your back is straight, the key is a good posture. Try to ride really positively, push him on instead of pulling back it takes a while to get used to :) good luck! X