ADD and sexless marriage

Further to PoisonIvys thread about an ADD spouse who's only interest is sex. This is my marriage. He has no other interest in me. No friendship. No companionship. I have quit giving him what he wants. I am tired. But I do often wonder if I am damaging the relationship too by not doing my part with the sex thing. I just don' know...any thoughts?