Improve my boundaries - pouty behavior

This is my first post.  Hubs 51 (untreated ADHD but does acknowledge it, mostly jokingly) and I (50) have been married for 11 years, my second marriage (w/ kids), his first.  I've read a lot on this blog and it's helped me to understand so much about how our brains are wired differently and that we don't each act the way we do to drive the other one bonkers.  

When I have an opportunity to do something w/ girlfriends he gets pouty when he can't  join in.  He has played as a sub in my ladies golf league a few times and I've realized that isn't fair to the others, they signed up for a ladies group and I was in the wrong for including him occasionally.  Every week he asks if everyone is going to play, do you need a sub, calls me at the end of the day.  At times I lied and there were only three of us b/c I didn't want to flat out say no.   I know I owe my friends an apology, that's easy.  What I need to do is let him know that I won't be inviting him to sub any more, it's a chick thing, not personal, they didn't sign up to play with me and hubs.  (Side note:  they like him and if invited on a golf date w/ us they would say yes)  

If a girlfriend invites me to do something (go to a concert, girls day on the boat) he gets pouty that he isn't included in the festivities and drops many hints that he'd love to go.  Sometimes things are "the more the merrier" and that's all good, but I am not going to invite Hubs to tag along on something that I was invited to do as an individual.   Just yesterday he had the fun event - an invite to go play golf with his friends.  Awesome - have a great time honey!   He took a half day of vacation, I was the one at work doing the unfun, adulting biz.  I'm thrilled he had a beautiful day do be with friends doing something he loves.

He does a lot of things socially/has hobbies and friends so it's not as if he never gets out.  We also do a lot as a couple, so it's not as if we were lacking in that department.

We've been in marriage counseling in the past (twice) due to work imbalance issues.  I was not ok with doing the heavy lifting anymore with him either underemployed or not working at all.  Kids are grown (so no need for child care) and we're not in a position for him to act retired or semi retired.    It was messy and hard but I feel like we've come through the other side in a good place.  Emotionally he's a lot better off working, which is awesome, great to see his self pride increase.  I know that was a huge boundary flex on my part and I'm proud of it.   

It feels silly to write this but I'm tired of going off do to something fun a have this manipulative, pouty attitude coming at me.  While a thicker skin would help me a lot (in more ways than just this, but that's another issue), I need to speak up about this: I'm tired of hearing it.  Any advice?

Thanks for listening.