Advice on getting away for a bit

I am reading one of Melissa Orlov's books; I am brand new to this forum and would greatly appreciate any advice you might have for me. My husband (w/ ADHD) and I have been together for a year and a half, but we grew up together, so I've known him most of my life. I am completely in love with him and always have been. He is brilliant, loving, and my best friend. I am a teacher and the sole breadwinner.

My husband was laid off a few years ago and had a lot of money saved at the time but has since lost it in the stock market. He does teach yoga and does other part-time jobs occasionally for money--but for the most part, I pay for everything, including his cell phone and car insurance. Money is our worst issue because every time I bring it up, he gets furious, and we end up in an awful fight. He is a very combative person, and I am conflict avoidant.

I decided to take this summer off and not teach summer school, so our problems have really escalated because we are together ALL THE TIME. Lately he has been manic (he has Bipolar II as well), and he talks at me constantly. I say "at me" rather than "to me" because these are monologues. He never pauses for me to respond, and he is unable to read my body language. If I get up to go to the bathroom, he follows me, still talking. I am a writer, and I have designated writing time with my office door closed, but he will just open it and start reading something to me, for example, with no regard for what I am doing. I never get any peace, and I feel like I am suffocating.

I would like to go out of town this weekend with my brother and his family to relax by the lake for a couple of days. I haven't had a vacation in a long time. I have to teach seven classes in the regular semesters (I teach at a community college) in order to pay all of our bills, so I am exhausted. My husband feels abandoned when I leave town without him, but he is dog-sitting this weekend, so he would not be able to come with me anyway. The truth is I just really need a break from him. I went to the beach last summer with my family, and it really rejuvenated me and our relationship (but there was a lot of drama due to his feelings of being abandoned).

What do you think the best way would be for me to go on this weekend road trip without hurting his feelings? How should I broach the subject?