Your health and baby is most important and if you are on HL, your boss should not expect you to get back to work in office. Try to talk to your boss again and if all things fail, ask if you can work from home. If not, please make an appointment with your HR and talk to them.