Maybe your horse had a bad experience one day with another person?
I would suggest to be very careful with your moves, don't approch the horse to quickly, talk to him a lot to make him feel relaxed and confident.
Spend a lot of time with him so he really knows you and knows that your are a friend.
Try to give him a slice of carrot, or apple or whatever treats he prefers but don't insist if he doesn't want you to touch him.
The most important is to take your time and be patient and one day he will give you his trust!