You did what is right. Now if you want to do what is wrong, you can withhold the payment of what you think is wrong to make a signal. Disclaimer. This is a legal gery zone and not advisable generally, but with your account needing to be closed it's a fun way to waste time. It's something I do professionally but I would not do it in private.
If your bill is $100 and the fee is $10 then you pay $90.
They'll send you reminders. Eventually they'll threaten with debt collection. Check your laws. If you have complaints and objecting the invoiced amount (you do) it will be illegal for them to pursue the amount until the dispute is resolved. It won't be. With amount like this they'll likely keep sending reminders until their accountant steps in and tells them to write off old debt after 5 years.
Keep in mind that you only withheld $10. They'll claim to be owed hundreds of reminder fees and interest. None of that is legal to collect by debt collection. Again check your local laws.
In the end, it's not worth the cost of debt collection. It's not worth their effort to pursue it. It will disappear eventually.
Anyway. I do not recommend this, but I still do it every day as part of my job. We take shit from no one. They oblige as long as I keep cool, and keep sending them "I already disputed this/what are you even talking about/could you please specify the amount/please provide documentation/I need a full overview" mails.
It's usually not worth the stress privately but it's possible.
Eventually if enough people complain about the fee, they'll have it fuck off.