The thing is… If someone has access to your system enough to replace your bootloader, they could probably just slip a USB keylogger between your keyboard and computer. Or set up a small hidden camera. Or plug all your devices into a raspberry pi to spoof the login screen.
It strikes me as odd that people assume that an attacker with a few hours physical access is going to bother going down the “change the bootloader” route when there are other, easier routes available.
Ironically, the only practical use case I can see for Secure Boot is when you have a dual boot setup where you don’t trust one of the OSes. Which I’m betting wasn’t Microsoft’s intention at all.