Do we need to be concerned about amount of baby movements at 20W?

I feel movements only sometimes in the morn or night.. not really other times so was wondering is this only important at 3rd tri?#firsttimemom