There is potting compound with high thermal conductivity for things that produce a lot of heat. A YubiKey hardly uses any power, so heat should not be an issue.
The main downsides of potting are that it makes repair practically impossible and it can add a lot of weight if there is a large volume to be filled.