If you do find their site works on a other browser (and I mean, the browser, not due to an addon, etc.) then complain to the company / bank / etc.
If they take your complaint seriously then I’d persevere with them… if they don’t then they’re not customer focused and your business (money) goes elsewhere.