IDK what OS you are on but on Linux most file managers have support for remote filesystems. SFTP (SSH-FTP, not to be confused with FTPS which is FTP-secure) is ubiquitous and if you use scp then you already have SSH set up.
If you need Windows support it is more of a pain. You may need to set up Samba or WebDAV and permissions can suck. But you can also download a third-party file browser that supports remote protocols.
So basically SFTP, and I fairly regularly just use a graphical file manager when I am doing one-off operations.