ADHD and ADD Step son with Father that wont help

Me and my husband have been married a year and a half.  I have 2 children and he has two.. Strangely our youngest are the about same age and so are our oldest.  I knew his children had very bad emotional problems with bipolar, ADHD and ADD when we got married but also I knew his ex wife had custody and I only had to deal with them every other weekend.  One question I asked before we got married is "is there any chance that she would want us to take custody of the children".  The reason I asked is because I knew I could handle the kids on weekend but i knew I couldn't handle them full time, plus I had there weekends matched with the weekends that my children went to there dads.  This is because none of the kids get along.  That sounds cold but I know my limits and I also knew it wasn't fair to ask my children to have to deal with it full time.  Now the problem happened in Jan when his ex wife brought the kids to our house and left them with all there belongings one weekend and said she was done and that they were outs.  Since then his oldest has cussed me pushed me and threatened my kids. She has since turned 18 and we had to kick her out.  Now his 14 yr old son is doing the same things and I'm truly scared of him and his father does not seem willing to do anything about it except take him to Dr and have his meds adjusted and amped up which has done absolutely no good.  My kids have gotten to the point they won't stay home because they cant deal with him at all and they feel threatened.  We have had to call the police on him this past weekend because he was weilding a butcher knife at me and my daughter.  That didn't do any good at all, the police just talked to him and left him with us.. My question after this rant (sorry for the rant) is how bad of a person would I be if I told my husband that the son had to go...  I don't want my marriage to end but I have to keep me and my children safe and I actually feel that I he will harm one of us in the near future.  I also went to the Dr myself and he told me and my husband that I was going into depression and I have never had problems with this before but I actually feel that I can't handle the arguments and not knowing from one minute to the next what kind of mood my husbands son is going to be in.  I have gotten to the point where I don't even want to go home after work anymore.