Is it ok for babies to stay full day at IFC during first month? Do they adapt well?

My 3 months plus baby has just started IFC. The IFC has advised us that it would be better to pick my baby up early like 430pm during the first month as baby may still be adapting. I can do so for 2 weeks, after which I will need to go back to work and as my working hours is till 6 plus, I won’t be able to fetch my baby at 4 plus. Is it really bad for my baby if I leave my baby at the IFC the whole day till 6 plus 7? I feel so guilty that I am putting her at ifc and now, won’t be able to fetch her home earlier as suggested as me and my hubby both need to work. Any experiences? Thank you. #worryingmom #1stimemom