I believe you are experiencing burnout. I experienced that before. We did not get any helper because it was covid surging time. So I had to be full time mom and I was also recovering from chronic pregnancy complication. It was tough as I also had ensure the house is "livable". Sure, the husband helps alot but sometimes you just need a helper once a while (if it can be helped) otherwise, you will reach your limit and enter the burnout mode and before you know it, you starts yelling at your baby. You can also get burnout when you do everything on your own and not sharing how difficult it is... I read somewhere, what matter is what you do after such incident and also it matter alot, if you check what triggers this burnout mode. Do not be scare to seek help. Counseling sometimes help, and of course a house helper and maid can help alot. If you have supportive loving parents, that will be much more better choice.
I’m not sure about will it impact his mental development but yes it happens… same situation, but what I did is to talk firmly to my baby and let the baby know the current situation, if it doesn’t work and before I lost control…. I will put the baby in a safe spot and get out of the room and scream in the toilet for that 3-5 sec and dashed back. U might feel better…