It does protect against physical attacks. PCRs are used to tie keys to specific hardware and software configurations and versions, boot paths, kernel command line arguments, etc and will lock out if changed. One of the reasons Ubuntu waited so long for official support was to set up the infrastructure for unified kernels and signing, the kernel and initrd are unified and signed and verified before it will unlock to protect against sophisticated attacks that most people will never encounter. For most people worried about theft, having it lock out when the boot order is changed would be enough. And when running, brute forcing the login process is slow and can be made even more painful with lockouts.
The TPM functions very differently than putting keys on a permanently attached usb drive.