It might your baby prefer breast feeding over bottle feed. To overcome this you will have to alternate feeding with FM in bottle, BM in bottle, direct latch. In this way, baby will get use to it. Because of the bonding mummy and baby have they just wants to be close to us due to our body smell, etc. So try to do the alternate method, keep trying till baby use to it. You can also try different milk bottle to ensure baby comfortable with the bottle teats.