Have any other parents ever been concerned that their child might love their yaya more than them? I feel like my kid spends more time with her than me, and it sounds ridiculous but it's something that worries me!

If your child spends more time with their yaya than you, then I guess it's a valid concern. Just make sure that your child understands why you're not spending enough time with him or her, and make up for it when you get the chance.