What to do if he wants to leave?

So my husband has ADHD (I do not) and has problems with his anger.  About a month ago, he blew up so bad in front of our kids (22 month old twins) that I ended up leaving with them for the night and taking them to my parents' house.  I told him explicitly that these blow ups were absolute deal breakers.  So he has now informed me that when he gets so mad that he feels like he is going to blow up that he is just going to leave.  So I thought he meant just for an hour or two to collect his thoughts.  No...he means overnight.  Like he has an overnight bag packed that he can grab and go if he feels like he needs to.  So I have an issue with this.  I feel like since we have children, it's not okay for him to just up and decide that he is going to leave overnight.  My ability to depend on him really takes a hit when I don't know if he is going to just up and decide to leave.  As you can imagine with two kids who are almost two, it is very very stressful being the only person taking care of them and I depend on him to help tag team with me on the weekends.  So I expressed how scary this makes me feel and asked if the general policy could be that he takes an hour or two, we reassess, and then go from there.  He absolutely exploded about that idea.  I tried to step it back and say that it could be something we talk to a counselor about because we obviously have different opinions on it, and he said that if we were in a session and I brought it up, he would walk out.  He says it is his right to leave and go to a hotel whenever he feels like he needs to.  I just don't really know what to do here.  I obviously can't really bring it up with him again because it took us back into a "black hole" again and it's just not ever going to be a productive conversation.  Am I way off base here?  I'm just trying to figure out what to do with this.  I'm feeling quite shaken up right now.