Non ADHD partner seeking advice re undiagnosed ADHD partner

Hi, I'm really hoping someone can shed some light on my current situation with an undiagnosed ADHD partner. We're only dating and have been seeing each other for just under a year. I don't have ADHD and had no experience of it before. It took me 6 months of utter confusion to work out that the man I've been seeing has ADHD. I felt like I was going mad. I haven't told him I think he has ADHD as he doesn't seem to ever want to talk about anything problematic and as he is also dyslexic I have a feeling that he has had years of being criticised and has learnt to ignore or bury anything which might be perceived as negative about himself. We're both in our late forties and he has never been married. Basically, I have to do all the work and be the leader in the relationship. He has been without a phone for about 3 months and so he asks me to just pop round when I can. That would be fine, however on one occasion I popped round to see him and after half an hour he asked me if I had something to do that day because he had to do something for a friend,something he could have done any time and for someone he sees more than me. I don't go round there more than a couple of times a week so I don't feel like we see each other much anyway. So, feeling hurt I didn't go to see him for over a week and he made no attempt to see me in that time. When I finally saw him I told him I didn't want to go round to see him only to find that he didn't want me there but he assured me I could see him any time, or to be more precise he said I could pop round for half an hour whenever I wanted. So, still feeling hurt I did exactly that and each time I went round I promptly left as soon as I could because I don't want to feel like I'm not wanted there. About the second or third time I did it he asked me if he'd see me sometime later that week so I said I could come Saturday night. That was last night. I had been there a couple of hours when we had sex (that doesn't happen frequently enough for me either) and after a bit of a cuddle he got up to go downstairs and I shortly followed. He seems to sleep downstairs on the sofa a lot and when I went down he beckoned me to lie down with him but I needed the loo and by the time I'd got back he was sat up looking moody as though he wanted me to go and he'd put away the cover and a pillow that he'd been using on the sofa. I felt awkward then, feeling like he didn't want me there and I then put on my jacket but only because I was cold, but he was very quick to ask me if I wanted to go. I ended up going, dreading that I would otherwise be out staying my welcome.

i don't want this to keep happening as I'm feeling more and more insecure by the day. Does anyone have any idea if this is ADHD behaviour or just disinterest and he's stringing me along. How do i talk to him about it without him just making out everything is ok and dismissing my feelings. I care about him a lot and otherwise love being with him, but I'm now feeling afraid of going to see him in case he wants me to go again.