It could be the way your baby latch, continue to moisturise ur nipple and just pump to bottle feed ur LO with BF. You can't stop pump otherwise will cause engorge. But you can tempo stop direct latch while ur nipple heal.
Try silverette nursing cups or breast milk collection shells to reduce friction when you wear bra?