Are there any disadvantages of not to letting baby have a pacifier? Like affect their brain development?

There's no negative affect if the baby doesn't have a pacifier. I never introduce my baby to a pacifier. The baby will adjust itself to whatever it's available. Most of babies will put things in their mouths, so we will see that babies like to have pacifiers or thumps in their mouths. If a pacifier can help them to sleep well or it can stop them from crying, then why not. But if you ask whether it's necessary, from my point of view i would say No!