Cannot Sleep during Pregnancy

Will it harm the fetal if we can only sleep around 3 hours a day. I just cant help to fall a sleep these days. I'm so scared if i might hurt the baby. I am at 4 weeks 5 day.