You should isolate it. Use Docker or Podman or any container solution to put the server in a container for isolation from the main system. Any external directory outside the container the server won’t write to should be in read only mode.
If the server is only for your friends. Stop exposing your network to the public and instead use VPNs.