Hi, this is a personal opinion but I would start ftom the ground. If he isn't confident, having a rider on his back second guessing his every move will only make him more nervous.
I would start with ground work, lunging & long reining. I would start with a pole on the floor, then two & then three. I would then raise the poles one at a time. Once he is comfortable & not questioning what you are asking him to do, them raise them to tiny cross poles that he's able to walk over. I would increase from walk to trot to canter as he gets more confident (adjust poles correctly when changing gait).
Once he is happy tackling a larger cross I would start all over with the poles with a rider on his back, lots of praise & don't question what he is going to do. Just travel as though he is going to go over it with no doubt. As soon as the rider doubts, the horse will pull out. How can he trust you, if you don't trust him?
Just to add on to the comment on small jumps. The jumps should be small enough for your horse to walk over. So if you need to have your horse take it at a walk. Then build up. Don't rush yourself, and if your horse rufuses first try to push him to walk over it and if not successful circle and try again
Start low like poles or small cross rails. Use your leg more to guide the horse to the jump and to make sure that he doesn't sneak out. My horse is the same, its not her confidence but she likes doing drive-bys and tries to sneak away from the jumps.
Again start small with poles and cross rails and make sure that you AND the horse is comfortable with the jump before moving on.
Try using cones or guide rails beside the jump to help prevent the horse sneaking away.
I know it was long but hope it helped! :)
I would say start will low jumps or even poles and use guide rails to guide you horse into the jump also talking to your horse is a amazing to boost confidence and make sure you have a good rein contact and that you are confident as well as horses do pick up on emotions!
Hope this helps