You’re glossing over a lot of complexity that the GrapheneOS team is doing for you. The reason that they only support Pixels is because Google is relatively open. Their FAQ says the following:
In most cases, substantial work beyond that will be needed to bring the support up to the same standards. For most devices, the hardware and firmware will prevent providing a reasonably secure device, regardless of the work put into device support. … Broader device support can only happen after the community (companies, organizations and individuals) steps up to make substantial, ongoing contributions to making the existing device support sustainable.
You can’t expect Android users to be able to en masse move to a fork if Google decides to close the tap.
I have LineageOS (CyanogenMod) running on one of my spare phones and it is easier now than it used to be ten years ago (speaking from experience), but you still need to have a phone that is supported, and the OEM needs to allow unlocking. I had to wait a couple of weeks to be able to unlock. So it’s definitely not trivial.