My baby is coming to 4 mths but we realise his feet are cold at night (if without socks on) even though he doesn't sleep in air-conditioned room. Is this common among babies or does it mean that his body is weak?

It's common. At 5 months my LO has cold and clammy feet too. Babies do have immature vervous systems and it can take a while before they are able to control their temperature fully. Many babies balance out a bit when they get more mobile, around 9 months they seem to regulate temperature better.