Make sure you don't jump at the first horse you see, and have a professional check the horse out. You will want to buy a horse that excels at what you love to do. Your coach or a vet is a good idea to bring with you. Ride the horse, and try to do everything during the trial. Such as handling, brushing and tacking up. If you don't like the way the horse behaves during leading, then consider a different horse. Just as an example.
The most important thing for me is to ride the horse more than once.
You'll also keep paying for lessons on top of this since you haven't been riding for very long
Make sure you can afford everything for the horse. Buying the horse is actually the cheapest part. Buying the tack, bruses, halters, wormers, food, treats all become very very expensive.
Have to remember you will then have vet bills to pay for, hood trimmings, board.
After that you have to consider if something were to happen to the horse, hurts their tendons collics, you'll be able to afford it.
Once you know you can afford it, you have to find the right horse for you. I would suggest buying a beginners horses, they'll be the safest for you.
Very good breeds to start with are quarterhorses and paints. They're very cool headed and very forgiving. I bought an ottb for my first and in the beginning I had a lot of trouble. Learn I love my horse endlessly and wouldn't give her away for anything, but I will always suggest paints and quarterhorses as people's first horses