An event isn’t an activity. And yes, if an event I loved turned into a meeting place for those people, I’d find a new event.
If Elon Musk and I both enjoyed cycling, for instance, fine. That doesn’t put me in the same space as him. Which is fundamentally different than going to a weeklong party he’s attending. Have your fun, just don’t pretend it’s revolutionary or creating a better world.