With patience and consistency. Basically you have trained your horse to be what we call ‘barn sour’ and it will take a lot of work on your part to re-school your horse’s thoughts about returning to the barn.
If you insist, you didn’t train your horse to be totally unruly whenever you turn back toward the barn but consider this from the horse’s perspective: upon returning to the barn, work is finished, the cinch is loosened, the saddle removed, feed is put in the box, a bath or a good brushing is administered, after which you are turned out with your buddies to rest and relax – left alone to do what horses do.
From now on, don’t ride back to the barn to do all these activities. Stop before you get to the barn, dismount, loosen the saddle and walk – leading your horse to the barn. Better yet – ride past the barn, stop, dismount, scratch your horse’s neck under the mane, talk and enjoy a few quiet moments, then turn and walk to the barn.