Any advice on how to deal with difficult subjects with ADHD partner?

Everyone was so helpful here last time I asked a question so I thought I would post again. What is your experience with either the way you, or your partner who has ADHD deals with difficult subjects, or handles arguments? 

I have noticed that when we start arguing, my partner completely shuts down. I don't often see it coming though I am beginning to notice the warning signs. It doesn't seem to matter what the argument is about or who starts it, once it becomes highly emotive (if I get upset, or raise my voice, for example, or if it is something he hasn't considered before), he just goes silent. I ask him if he is planning on talking, or if he has a response, and he usually says "I don't know." So we sit in silence while I try and give him some time, however if the thing we are discussing is important to me, I tend to push for a reply. Especially if the argument started because of something he said.

it tends to escalate from there, with me increasingly getting upset and asking for a response, and him getting frustrated and unable to talk. It usually cumulates to the point where I feel he literally doesn't care and I ask if he wants me to leave/if there's any point in continuing etc. or he will walk out of the room and start doing something physical, like cleaning, saying he can't deal with it.

As soon as I have calmed down, I try and talk to him calmly and he usually repeats that he can't deal with it. It tends to be at this point that I realise his ADHD related issues have kicked in. I tend to reach out to him physically, which stops him, and we usually embrace. He then gets upset and says how useless he is, and apologises a lot for how he can't effectively deal with the situation once it reaches that point. We then make up but tend not to revisit the source of the argument until a later date.

Im just wondering if anyone else experiences this or had any advice? I am slowly learning to recognise his signs and sometimes I can derail it. But it's incredibly frustrating if the argument is caused by something he has said that upsets me (sometimes he doesn't even know why he has said something) because it feels as if I am having to set aside my upset to make allowances for his inability to deal with the situation once I become emotional. How do I handle that? I know I need to, because this isn't his fault, and I want to be as understanding as possible, but at the same time, both him and I believe he needs to try and learn how to deal with his side of things too. Help!