Except… CalDAV exists. It’s built on top of WebDAV and not email. IMAP has the ability to share objects, and some calendar apps use that to share calendars the same way you’d share emails, but there’s an international standard for doing calendars without email.
I personally use a self-hosted NextCloud for calendars and contacts. It integrates into pretty much every app out there, or I can use the web interface or NextCloud apps. Email is only needed for the admin account to send email notifications of someone gets locked out etc.