You really feel a connection with them and you feel like you are getting closer and feel comfortable with the way you ride them and you feel like you two are both growing at whatever your trying to accomplish
In much the same way you can tell if a person is right for you. You may not click straight away, but this is one of the reasons a trial is important if you're looking for your long term horse. Of course, there are certain things you can rule out straight away, such as certain heights (if you're 6ft, a 12hh pony isn't right), certain temperaments (if you're a novice, something highly sprung mightn't suit), certain breeds (a cob is most likely not going to be a grand prix jumper), the discipline of the horse (some are just suited for dressage, others love jumping, some are just happy plodding around the place), etc,.
However, you could have the best rider and the best horse, but they still might not click with each other and neither of them will get their full potential out. To know whether or not you get on with a horse just takes time, exactly like people.