Not really through Caddy but for my setup I have it so the ssh port for Forgejo is only accessible through tailscale. So for push/pulling updated my ssh config file to something like
Host git.mysite.com HostName tailscaleMachineName User git Port 1234
Then doing git pull [email protected]:user/project.git works just fine as long as I am connected to tailscale
Otherwise you could open the port for Forgejo’s ssh so that you can access it without any vpn