I would like to know if i want to bring baby for jaundice check up, can I just walk into the polyclinic?

Yes :) I assume this your first jaundice check up? Because follow ups usually would have an appointment already set. As per what Evelyn said, there is a priority queue for newborns. Waiting time is really fast, max 30minutes.