I think the constant back and forth is really affecting the kids. Why not let them first settle in at the new place?
Once they are set in their routine, you can slowly start taking them over for the weekends. Even then, start with a day trip and come back home at night. You can extend it to a night stay once they are comfortable.
Read more