My ex boyfriend wants me back!!!! should i take him back or not?

ilast year i met my ex i guess he had just got out of prison we talked for a while but what i didnt like was that he smokes weed and drinks and likes too hang out with his friends alot.i didnt really care at that time because i just wanted to hang out but then i started too fall for him and then i got pregnant so we moved in together and he would have his friends over every single day they would stay outside but that would bother me soo much that one day i got my stuff and left then he cried that he wanted me back so i came back after a month we started to fight again because he always wanted us to go with his friends and i didnt wana go so he would wana snatch my keys away and wouldnt give them back becuse he was scared i would leave then he started to shove me and push me when i wanted too leave and i was pregnant he pulled my hair once so i ended up leavin and all he would do is party and drink everyday again so he kept beggin me again so i gave in and moved bak in with him then i had my son and we started fightin again over stupid things he said he didnt care if i left so i leftt and now im sad thinking wat to do he texs me sayin he wants me back its bein a week we broke up he has anger problems im jus scared he not gon ever change for me or my son i dont even trust him with my son because im scared he wont give him back to me until i take him back what should i do i wan us to be a happy family i need advice plus i have 3 from a previous marriage and he loves them sooo much my birthday is saturday and he wants too spend it with me wat should i do