Nope it will not affect your baby! I was sick every month during my second and third trimester and I asked my gynae about it. My gynae advised me to drink more water, sleep + rest more, eat vitamin C daily to boost back your immune system. Remember to inform your GP doctor that you’re pregnant so they will prescribe pregnancy safe medication.
It should be quite safe as long as you do not develop a high fever. Like you, I was down with a cold (mild) at about Week 21. Doc gave me pregnancy safe meds, including some antibiotics, for a week. Most importantly, have sufficient rest. I rem the GP asked if I had irregular discharge. If you do, consult your gynae. Rest well!