As everybody said, the most important is to warm up the back and legs!
I also do. It's important that the horse warms up his back, neck and limbs before carrying our weight!
I try to do it often. It depends of my horse's state. When I see he is tired or seems to have stiffness, I stretch all his legs, his neck and do some poll yielding to help him relax and strech!
Always! I stretch the forelimbs and walk the horse for 5 minutes so the saddle can get the best position on the back and won't hurt him when I will ride.
Before riding, I lunge my horse 5 to 10 minutes and then I saddle him, so he can warm up first. And I stretch his forelimbs and neck before I ride.