The horse is the cheapest thing you'll buy, if you were to buy a horse you'll need to make sure that you'll have enough money when the horse looses a shoe, rips a rug, gets sick etc sometimes if your not sure, lease a horse for a few months then once your confident that you can afford it then buy a horse :)
Lease! That $2,000-$6,000 will only be the starting price! What about the vet? Farrier? Board? Trailering? Tack? Grooming supplies? Blankets?... etc. Start leasing and if you feel that you'll be able to afford your own horse and you're ready for that time and commitment then buy a horse! If you really want your own horse you could try a lease-to own option if you find the right horse! Good luck!!