ADHD spouse needs consequences

I need advice for how to handle a financial issue that seems to continue in our marriage.  My husband did not have his taxes deducted from his unemployment checks 3 years ago and the IRS is now wanting to collect.  I tried filling out non-obligated spouse forms hoping that I would not have to be responsible for his debt.  I found out that you pretty much have to be divorced to not be held responsible.  My husband does not see paying the IRS as a high priority and has not followed through with the payment arrangement he set up with the IRS.  I also had him sign an agreement with me saying he would make the payments once a month until it was paid off.  It's been several months now and he has only made a few small payments.  I received a notice in the mail in my name saying that it has gone to a collection agency and they will put a lein on our property if we don't pay.  I have a huge problem with this since the lein is in my name and I am the one paying the mortgage.  When I approached him about the letter, he said it doesn't matter if they put a lein on our house because we aren't moving any time soon and they can only collect on the lein if we sell.  So I am going to be forced to pay the balance myself or suffer the consequence of having my credit ruined and having a lein on the property.  It's just not fair that I am the one suffering the consequence for his debt and lack of follow through.  This is not the first time this has occured in our marriage.  The same thing happened five years ago.  He owed a much larger amount and since we had a joint bank account, the IRS was able to go and just take whatever was in our account.  Luckily the account did not have the entire amount available to withdraw.  So, they took what they could and put a lein on our condo.  When we went to sell the condo, we had to have the remaining balance go to the IRS instead of putting that money into our new home.

I am sooo frustrated and thinking of calling the IRS and telling them to go take it out of his account.  The problem is this won't take care of the entire balance since his unemployment checks are so small.  I could ask him to hand over his next 3 or 4 unemployment checks to take care of the entire balance, but he will come up with some excuse why he can't do it.  I need serious advice.  He needs to be accountable for his own financial debts.  The problem is as long as we are married, the IRS doesn't care whose debt it is.