Taking a break...

I am a long time reader of this forum as it helped me understand the challenges I had in my on & off relationship with an ADHD man in the last 3 years.

We met & started dating immediately 3 years ago. I was very much in his hyperfocus as he constantly wanted to be around me, was devoted, loving & upfront with his feelings. Even I was too - I believed he was the perfect match for me because we had so much in common plus we had the same life goals. 
Our relationship only lasted 5 months because he told me he suddenly stopped having romantic feelings for me. I was stunned & heart broken because I was so sure he was it. I cut him out. I didn't want to see him again because I didn't believe that "feelings just stopped" was a real reason. I lasted about a week before contacting him again to suggest trying to be friends. Afterall, we had so much in common & got along really well, making a cut because we didn't work as a couple seemed like a waste.

We have been friends for 2 & half years. After being friends for the first month after our break-up, we started sleeping together again & had been doing so ever since - on & off. What was worse is that we both started seeing other people several times in this 2 & half years but weren't exactly faithful to these other people because we kept coming back to each other. It has continued to be impossible for us to spend time alone together & not be intimate in some shape or form. Of course there was jealousy from both sides as well.

Half way through last year, our career paths went in different directions (same industry but I ended up having more success). This caused me to be in a new circle of people & have less time available to be with him. He became distant. I missed him & told him so. Our physical aspect of our 'friendship' continued but he didn't tell me anything anymore. It felt less & less like a friendship. It was during this time that I realised that I was still in love with him. 

About a week before I planned to tell him my feelings for him, I found out he was sleeping with someone else as well as me - which was not unusual for us but it affected me more this time. I decided I couldn't go on like this anymore so 3 weeks ago I told him that I can't be friends with him anymore because I still love him & it's getting too hard for me to deal with emotionally. I thought he would be upset by me taking away my friendship but he just said "Ok, I understand". He didn't seem upset or didn't want to try to object to see if he could make things better. I was shocked by this reaction & upset too. He has never been one to show his vulnerability so I don't know whether it was a self preservation reaction or what. 
Even so, currently it has been 3 weeks which is the longest time we've gone without any form of communication and interaction & it's so difficult for me because I do miss him & am wondering if I should tell him that I miss him or ride it out for a bit longer to see if he comes back to me first?