Try a bitless bridle or just a side pull, you don't have to ride every horse with a bit, with proper training you can ride without one. If he's uncomfortable with a bit you should try other methods before trying to make him take one. Some horses just aren't meant for a bit.
You must need some time to know this new horse and the way he operates. To help him relax and be in on the bit, you can do some exercises such as little and bit circles, bending, flexing, shoulder-in, leg-yielding... But careful, during all those exercises, check your horse engages the hindquarters!