My baby is used to drinking warm breast milk. Is it okay to heat formula in a bottle warmer?

You would usually make milk at the time baby needs it. So just mix hot water with room temperature water. No need for bottle warmer.