On Chrome, I only ever recall seeing the dialog when I install an extension, or if an extension is updated to use additional permissions.
Firefox MV3 is different, in that the all_urls permission cannot be granted on install. If an extension requests all_urls, it installs with the permission disabled. The user has to manually enable it for one site or all.
IPvFoo is mostly useless without all_urls, which is why I made it show that button until the permission is granted.