what are your system specs? at a low effort you should be getting a lot more FPS, what cli command are you using? but I guess it would be best for you to export to /tmp given enough ram and then go from there
EDIT: for context, when encoding libjxl I would do -distance 0 -effort 2 for lossless output