Husband hits me in his sleep in my 1st trimester??

Anyone experienced your husband hitting you/elbowing you in his sleep? It's been quite frequent on my lower back/just my back. He was asleep though when i reacted & said ouch, he just woke up and said sorry, change his position & went back to sleep. I understand he was asleep but it was so freaking painful it lasted for a few minutes that tears came out because of pain. I'm getting annoyed and afraid it might affect my baby? I'm still in my first trimester and it's my first pregnancy!