baby starts kicking only in middle of the night.

How many weeks are you? If you're active during the day and busy working, you might not notice baby kicking. Plus your walking motion may rock baby to sleep. At night because you're lying still it's easier to feel baby's motion. During the day if you can try lying down or staying still for a while and focusing on baby maybe you can feel the kicking. Drinking a glass of cold water may help too