Make sure that you know what you're after, such as a project, show jumper or a schoolmaster. Make sure your experience matches your own needs as well as your horse's needs. It'd be a bad decision to want to go out and compete with a forward going, experienced horse and you went for an inexperienced, heavy kick along plod.
Like Laura says, ask as many questions as you need! Especially ask for vetting history, any known vices, physical issues and the horse's education.
Of course, it isn't exactly essential to ride the horse you're after, but it definitely helps to see if the horse can adapt to your style of riding and that he understands what you're asking of him.
If you do end up buying chosen horse, the worst thing you can do is hop right on him in a new environment unless he's a perfect, bombproof, trusty steed. Allow him to settle in his new home and gain new friendships with other horses. Rushing a new horse will result in him having anxiety.
Groundwork is essential to earn their respect and trust, so I'd definitely focus on this for a few days or weeks before even getting on him.