Talk to him and settle things between the both of you because if you let it pass through it may cause damage in your relationship that can be bad for your child.
Mas maganda kausapin mo sya ng maayos para malaman nya yung side mo. Everything will be fine naman.