When the horse is working better from the hind end this will stop. Under paced and weak horses tend to do it. Make sure he's going forward and engaged.
I think the less equipment the better. Also try to do exercises that strengthen his top line so he becomes stronger and can keep his head still, but you also need to try to keep your handle as still as possible. It’s probably a hard concept for him to grasp but he’s attempting to stabilize himself with you on him it’s harder if you can’t keep your hands still!
it's a natural motion of the head & neck
I would try a running martin gale! It really gathers them so that might work, my horse has one and it works amazing for anything!!
Do not worry about the head yet if he is green. The head will relax when he relaxes and learns to do his new job. Not the answer you want hear I am sure, there are things to do that could prevent the habit but those are only band-aids and crutches that if you remove them you have the same problem. Time is what it takes and LOTS of patience