My trainer switched barns, a few years ago. I followed her to the new barn after a couple weeks. I think it is best to stay with a trainer you’ve known for a while, because they know how you ride and everything. However, I think you should do whatever is most comfortable.
If you really love your trainer then definitely move with her. You won’t be starting all over again bc your trainer will be there and she knows your level of riding and even if you may leave a school horse at your barn you really love, you will probably find one at your new barn that you will also really love.