Using containers on Linux has basically no performance loss compared to running on the host. They share a kernel and nothing needs to be virtualized (unlike containers on macOS and Windows), so anything you run in a container is basically the same performance as running it on the host.
I still agree though: using Nix is better than using Distrobox for many other reasons.