Personally for me whenever that happens, its because baby is sleepy. I used to think that it was due to the poor latching. Recently i realised that it is because he wants to sleep and was trying to use me as a pacifier. Im not sure if its the same case for you. You may want to try to observe and see if its the same for you.
Hope it helps.
Read more