The question isn’t “are they safer than the average human driver?”
How is that not the question? That absolutely is the question. Just because someone is accountable for your death doesn’t mean you aren’t already dead, it doesn’t bring you back to life. If a human driver is actively dangerous and get taken off the road or put in jail, there are very likely already plenty more taking that human drivers place. Plus genuine accidents, even horrific ones, do happen with human drivers. If the death rate for self-driving vehicles is really that much lower, you are risking your life that much more by trusting in human drivers.