It's normal especially for full-time working mums, since baby has more contact time with nanny, someone who helped you take care of your baby when you are not around. Don't fret, just spend whatever time you can when you are back home, be in your baby's presence. After a while, he will understand that when mummy is home, he gets to see you, hear you talking to him, feel your touch and smell :) It may take a while for him to understand, but be there with him and play with him as much as possible when home. If possible, do a video call with him during your lunch time weekdays, get nanny to let him see your face, hear your voice.
that's what my husband experiencing since he wasn't with us for 9 months after he was born. and I am full-time at home mom. and he cries when I gave him to his father, to his grandparents.