TPM 2 based disk encryption. This is basically what bitlocker does, but it isn’t great. It uses an encryption key stored on your TPM chip, that shouldn’t ever be accessible to be exported. This means the disk should only be decryptable in the machine it’s in. That in conjunction with secure boot can give you some guarantees that the only way to access data is through the the computer itself (no pulling the disk first). The issue is there are many potential vulnerabilities that could subvert this, logoFAIL being the most recent.
You could setup a proper KVM. The two gotos are PiKVM and TinyPilot. Jeff Geerling did a good video on these. It’ll cost a few 100 bucks but can definitely be worth it. You might consider a motherboard with a builtin KVM in your next build too.
Setup NBDE (Network Bound Disk Encryption). This is pretty new, but what I’m planning to move to. Redhat has an implementation with Tang & Clevis (server and clients). You might be able to eventually use Clevis with other alternative backend too.