Can you have sex in early of pregnancy? Is it safe?

It is said that you should avoid having sex in the first trimester and the latter part of the third trimester.