it might sound absolutely ridiculous, but an aggressive child is actually a sign of a healthy development!
the outwardly signs that you are mentioning are signs that he is channelizing his internal anger and not holding on to them, which is a positive sign. the fact that you connect his aggression with some instances where you too did the same makes sense, kids almost always replicate the parents, especially those things we don't want them to replicate. for now, i would suggest you don't lose your cool and instead of worrying what others feel, focus on him, show him your love and attention and tell him repeatedly that you love him. also, whenever he says i will hit you, get down to his height level, look at him, hug him and say its not nice to say so, you are a good boy, no hitting. these links might help
https://www.zerotothree.org/resources/12-helping-young-children-channel-their-aggression
https://empoweringparents.com/article/hitting-biting-and-kicking-how-to-stop-aggressive-behavior-in-young-children/
Read more