A removable battery wouldn’t do anything about it. They would just include a little unremovable CMOS battery that would be used to power the Bluetooth while the device is off or the main battery is removed.
The entire point of it is so that you can use Find My when the phone is off. If it’s lost, or stolen and they turn the phone off, it can still be found in Find My.