Is Love supposed to be enough?

My husband (who has a diagnosis of severe ADHD) had parents who were abusive/alcohilic/neglectful, but the one thing he valued was that they loved each other no matter what and stayed together till the end. So despite their dysfunctional, hurtful, abusive relations with each other, he believes love should be enough and no matter what happens you should stay with them until the end. I believe that if the marriage gets to the point where it's destroying one's well-being and happiness in life, and all attempts to reconcile those things with your partner have failed, it's time ot think about leaving, and that sometimes, love is just not enough.

Because his ADHD issues are severe enough to have taken quite a toll on my well-being (to the point of suicidal thoughts), I have for a few years inched closer and closer to divorce as he repeatedly showed no changes despite long talks and heart to hearts, and real intention on his part to change. At this point, my leaving would be a matter of survival, despite how much I love him. To him, it looks like I will never truly love him and accept him, and is so ANGRY that I would even consider leaving him, and that I should accept him and stay no matter what. He verbatim has told me my feelings on this were wrong, and I was bad for thinking it.

To me that hurts because I believe it's indicative of him truly not understanding how much his ADHD has cost me and our family. It has ran our entire lives into the ground. If you truly love someone, why would you want to keep them in such unhappiness? I would never expect him to stay with me if he was this unhappy. In fact, I have my own disorder, Misophonia, which  can be a huge stressor for others, and from Day 1 I told him if it ever got to be unbearable, I would understand if he needed to leave because of it and I'd think no less of him for it. I would never want to make someone so unhappy. 

So I guess I'm asking, is love really supposed to be enough, no matter what? Am I wrong in feeling I can't survive in this marriage and therefore need to escape it? I do accept his ADHD and that he will always have it, but I can't accept him not managing his behaviors to a livable point, and I though he agreed and understood that. But am I supposed to stay despite severe dysfunction, despite the cost to the kids and me? Where is the line?