Withdrawing and I’m meant to just be ok with that?

My ADHD partner and I live separately. That's good in some ways as reduces conflict over housework. What maddens me is that when he's decided we've had an argument (it's usually a small bicker that somehow cuts him like a knife) he will withdraw for days on end saying 'let's take a break.' He will message but not pick up the phone. Weekend plans made previously become uncertain. It drive me crazy but when he's in withdrawal I have no option to explain how I feel. Is this really just ADHD or is it just mean?