Sounds like an episode of everybody loves Raymond!
Can you get the locks changed and give them an excuse that you lost the house keys and you are scared an intruder might come by and rob the house. Then just make sure you never give them the new house keys :)