Blamming the ADHD

My boyfriend and I have been together for a little over a year now. He has inattentive ADHD and has a history of verbally abusive relationships. I myself have OCD and have a history of physically abusive relationships. We are a very loving couple and can typically very understanding of each other. Unfortunately, when situations get stressed, that sometimes is not the case. Very often in arguments, I get the same response: “I’m sorry but that’s just how I think/am. I’m ADHD, okay, I’m sorry.” It becomes very difficult to communicate the issue because every time he senses some possible fault in himself, he attributes it automatically to his ADHD. To me it feels like he’s guilting me, tells me that he hates himself and wishes he didn’t have ADHD. I don’t ever make any attacks at him but I do try to give him advice on how we can communicate better. He in turn believes that I’m “chewing him out.” He always insist I say something I didn’t and whenever I try to challenge him on it he becomes self pitying and starts saying how he just thinks differently and misinterprets. He constantly asks questions that to me feel very accusatory, almost as if to question my actions, and then when I feel like these questions are unnecessary, he says that he’s just asking questions to appease his ADHD and that I shouldn’t have a problem with it. I feel afraid to say how I may feel offended because he doesn’t understand that there is such a thing unintended offense. Now I promised to never start a fight because he feels like “no one understands me” even though I’m patient and I’m trying to have him understand how other people may think. I feel like I’m walking on glass when I’m with him now, too afraid to say anything for fear he’ll become depressed for his feelings or rejection of my own feelings (ie., they have no substance because “I would never hurt your feelings, I just have a disorder”). He speaks harshly when he feels attacked and it makes me so upset because this is someone I love and want to be with for a long time but I feel like leaving because he says he’ll change but doesn’t appear to because he thinks that his ADHD should be an excuse for everything we argue about. Someone please help me. I don’t know how much more I can bear.