Is it all my fault?

My boyfriend and I have been together for a little over a year now. He has inattentive ADHD and has a history of verbally abusive relationships. I myself have OCD and have a history of physically abusive relationships. This makes things very difficult since I am not currently on any form of anxiolytics and he is on Ritalin which also, as you all may know, has a side-effect of anxiety. Go figure. This makes communication very hard because when he gets confused or flustered or is trying to refocus himself, he becomes angry. His outlet for his frustrations are channeled towards anger as opposed to anything else. He often tends to you repeat himself and reinforce things in way that almost feels guilt tripping. I feel trapped by his incessant need to ask questions and clarify every little thing. Sometimes we argue about things like the correct usage of a word or a tick I have and he argues usage to the bitter end, which aggravates my OCD with my need for things to be just-so, sending me into a panic attack. And of course, panic attacks are something he cannot comprehend as well. There was an instance in which he didn't understand why I wanted him to tell me what time we were planning to go on an outing the next day and questioned me as to why it was so important to know the exact time at that moment, to which I entered an anxiety crisis and he continued bombard my mind (which was already closing in on me) with questions like "what is going on", "why are you crying", "why won't you look at me/answer me", and statements like "I don't understand why you're freaking out right now" and "what is the purpose of knowing the exact time." Had it not been for my roommate who used deep breathing exercises with me, I might have began thrashing. His ADHD drives him mad with questions that cause him to stutter with the quickness of his mind and make him angry and frustrated. He is not violent to anyone except himself. He hits himself and slams his body into hard items in frustration. He wants to understand everything so desperately but he can't and I cannot understand him completely because his mind races so quickly and I am slow and methodical. He's brain is very "guns blazing" and "Hi-Ho Silver!" I'm scared for his safety and our arguments cause me too much anxiety. I feel like I shouldn't share my opinions, thoughts, and feelings because he cannot cognitively understand without stressing me out because most of my processes are unexplainable, irrational, and can only be attributed to my OCD. Sometimes I feel like if I was just normal, I could help him. I feel like a terrible person because I feel like the way I am makes everything worse. I used to think that it was him that was trying to understand but it has become clear to me that he pushes himself to the limit to understand even though the way he processes things is often convoluted and scattered. I feel like I'm the problem because it's not that he's the hard one to understand...it's me. And I'm making everything worse. Are there any other spouses/partners of someone with ADHD and have OCD themselves or vice versas? Can anyone help me? I want to be with this man for a long time, but as though the way I am is getting in the way. Is there anyone with ADHD who is with someone with OCD and can tell me what my boyfriend may be feeling? Please help :(