it's actually better to let baby get used to loud surroundings. then baby can just fall asleep anywhere. so no worries about that. germs wise if you're worried just remind ppl to wash hands before touching the baby. I come from a big family as well and honestly, as long as they're close family member and I know they're not sick, I don't mind. it's YOU that your husband must be worried about. Post partum, it can get overwhelming to have ppl around at all times. meaning u can't be yourself no matter how much they "understand". so either go into ur room and ignore the visitors or if it gets too tiring, talk to your husband.
Usually for gatherings, I will just stay in the room, lock the door if I have to then say I’m breastfeeding and fell asleep 🤓. For the noise and sounds, baby will get used to it. My mil like to talk loudly, on tv like cinema and I have 2 dogs who barks at every single human who walks past but baby got used to it and wasn’t bothered by the noise.