Need help routing Wireguard container traffic through Gluetun container [Solved]
The solution has been found, see the “Solution” section for the full write up and config files.
Initial Question
What I’m looking to do is to route WAN traffic from my personal wireguard server through a gluetun container. So that I can connect a client my personal wireguard server and have my traffic still go through the gluetun VPN as follows:
client <–> wireguard container <–> gluetun container <–> WAN
I’ve managed to set both the wireguard and gluetun container up in a docker-compose file and made sure they both work independently (I can connect a client the the wireguard container and the gluetun container is successfully connecting to my paid VPN for WAN access). However, I cannot get route traffic from the wireguard container through the gluetun container.
Since I’ve managed to set both up independently I don’t believe that there is an issue with the docker-compose file I used for setup. What I believe to be the issue is either the routing rules in my wireguard container, or the firewall rules on the gluetun container.
I tried following this linuxserver.io guide to get the following wg0.conf
template for my wireguard container:
<span style="color:#323232;">[Interface]
</span><span style="color:#323232;">Address = ${INTERFACE}.1
</span><span style="color:#323232;">ListenPort = 51820
</span><span style="color:#323232;">PrivateKey = $(cat /config/server/privatekey-server)
</span><span style="color:#323232;">PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE
</span><span style="color:#323232;"># Adds fwmark 51820 to any packet traveling through interface wg0
</span><span style="color:#323232;">PostUp = wg set wg0 fwmark 51820
</span><span style="color:#323232;"># If a packet is not marked with fwmark 51820 (not coming through the wg connection) it will be routed to the table "51820".
</span><span style="color:#323232;"># PostUp = ip -4 rule add not fwmark 51820 table 51820
</span><span style="color:#323232;"># Creates a table ("51820") which routes all traffic through the gluetun container
</span><span style="color:#323232;">PostUp = ip -4 route add 0.0.0.0/0 via 172.22.0.100
</span><span style="color:#323232;"># If the traffic is destined for the subnet 192.168.1.0/24 (internal) send it through the default gateway.
</span><span style="color:#323232;">PostUp = ip -4 route add 192.168.1.0/24 via 172.22.0.1
</span><span style="color:#323232;">PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth+ -j MASQUERADE
</span>
Along with the default firewall rules of the gluetun container
<span style="color:#323232;">Chain INPUT (policy DROP 13 packets, 1062 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">15170 1115K ACCEPT 0 -- lo * 0.0.0.0/0 0.0.0.0/0
</span><span style="color:#323232;">14403 12M ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 1 60 ACCEPT 0 -- eth0 * 0.0.0.0/0 172.22.0.0/24
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain FORWARD (policy DROP 4880 packets, 396K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain OUTPUT (policy DROP 360 packets, 25560 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">15170 1115K ACCEPT 0 -- * lo 0.0.0.0/0 0.0.0.0/0
</span><span style="color:#323232;">12716 1320K ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 0 0 ACCEPT 0 -- * eth0 172.22.0.100 172.22.0.0/24
</span><span style="color:#323232;"> 1 176 ACCEPT 17 -- * eth0 0.0.0.0/0 68.235.48.107 udp dpt:1637
</span><span style="color:#323232;"> 1349 81068 ACCEPT 0 -- * tun0 0.0.0.0/0 0.0.0.0/0
</span>
When I run the wireguard container with this configuration I can successfully connect my client however I cannot connect to any website, or ping any IP.
During my debugging process I ran tcpdump
on the docker network both containers are in which showed me that my client is successfully sending packets to the wireguard container, but that no packets were sent from my wireguard container to the gluetun container. The closest I got to this was the following line:
<span style="color:#323232;">17:27:38.871259 IP 10.13.13.1.domain > 10.13.13.2.41280: 42269 ServFail- 0/0/0 (28)
</span>
Which I believe is telling me that the wireguard server is trying, and failing, to send packets back to the client.
I also checked the firewall rules of the gluetun container and got the following results:
<span style="color:#323232;">Chain INPUT (policy DROP 13 packets, 1062 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">18732 1376K ACCEPT 0 -- lo * 0.0.0.0/0 0.0.0.0/0
</span><span style="color:#323232;">16056 12M ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 1 60 ACCEPT 0 -- eth0 * 0.0.0.0/0 172.22.0.0/24
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain FORWARD (policy DROP 5386 packets, 458K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain OUTPUT (policy DROP 360 packets, 25560 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">18732 1376K ACCEPT 0 -- * lo 0.0.0.0/0 0.0.0.0/0
</span><span style="color:#323232;">14929 1527K ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 0 0 ACCEPT 0 -- * eth0 172.22.0.100 172.22.0.0/24
</span><span style="color:#323232;"> 1 176 ACCEPT 17 -- * eth0 0.0.0.0/0 68.235.48.107 udp dpt:1637
</span><span style="color:#323232;"> 1660 99728 ACCEPT 0 -- * tun0 0.0.0.0/0 0.0.0.0/0
</span>
Which shows that the firewall for the gluetun container is dropping all FORWARD traffic which (as I understand it) is the sort of traffic I’m trying to set up. What is odd is that I don’t see any of those packets in the tcpdump of the docker network.
Has anyone successfully set this up or have any indication on what I should try next? At this point any ideas would be helpful, whether that be more debugging steps or recommendations for routing/firewall rules.
While there have been similar posts on this topic (Here and Here) the responses on both did not really help me.
Solution
Docker Compose Setup
My final working setup uses the following docker-compose
file:
<span style="color:#323232;">networks:
</span><span style="color:#323232;"> default:
</span><span style="color:#323232;"> ipam:
</span><span style="color:#323232;"> config:
</span><span style="color:#323232;"> - subnet: 172.22.0.0/24
</span><span style="color:#323232;">services:
</span><span style="color:#323232;"> gluetun_vpn:
</span><span style="color:#323232;"> image: qmcgaw/gluetun:latest
</span><span style="color:#323232;"> container_name: gluetun_vpn
</span><span style="color:#323232;"> cap_add:
</span><span style="color:#323232;"> - NET_ADMIN # Required
</span><span style="color:#323232;"> environment:
</span><span style="color:#323232;"> - VPN_TYPE=wireguard # I tested this with a wireguard setup
</span><span style="color:#323232;"> # Setup Gluetun depending on your provider.
</span><span style="color:#323232;"> volumes:
</span><span style="color:#323232;"> - {docker config path}/gluetun_vpn/conf:/gluetun
</span><span style="color:#323232;"> - {docker config path}/gluetun_vpn/firewall:/iptables
</span><span style="color:#323232;"> sysctls:
</span><span style="color:#323232;"> # Disables ipv6
</span><span style="color:#323232;"> - net.ipv6.conf.all.disable_ipv6=1
</span><span style="color:#323232;"> restart: unless-stopped
</span><span style="color:#323232;"> networks:
</span><span style="color:#323232;"> default:
</span><span style="color:#323232;"> ipv4_address: 172.22.0.100
</span><span style="color:#323232;"> wireguard_server:
</span><span style="color:#323232;"> image: lscr.io/linuxserver/wireguard:latest
</span><span style="color:#323232;"> container_name: wg_server
</span><span style="color:#323232;"> cap_add:
</span><span style="color:#323232;"> - NET_ADMIN
</span><span style="color:#323232;"> environment:
</span><span style="color:#323232;"> - TZ=America/Detroit
</span><span style="color:#323232;"> - PEERS=1
</span><span style="color:#323232;"> - SERVERPORT=3697 # Optional
</span><span style="color:#323232;"> - PEERDNS=172.22.0.100 # Set this as the Docker network IP of the gluetun container to use your vpn's dns resolver
</span><span style="color:#323232;"> ports:
</span><span style="color:#323232;"> - 3697:51820/udp # Optional
</span><span style="color:#323232;"> volumes:
</span><span style="color:#323232;"> - {docker config path}/wg_server/conf:/config
</span><span style="color:#323232;"> sysctls:
</span><span style="color:#323232;"> - net.ipv4.conf.all.src_valid_mark=1
</span><span style="color:#323232;"> networks:
</span><span style="color:#323232;"> default:
</span><span style="color:#323232;"> ipv4_address: 172.22.0.2
</span><span style="color:#323232;"> restart: unless-stopped
</span>
Once you get both docker containers working you still need to edit some configuration files.
Wireguard Server Setup
After the wireguard container setup you need to edit {docker config path}/wg_server/conf/templates/server.conf
to the following:
<span style="color:#323232;">[Interface]
</span><span style="color:#323232;">Address = ${INTERFACE}.1
</span><span style="color:#323232;">ListenPort = 51820
</span><span style="color:#323232;">PrivateKey = $(cat /config/server/privatekey-server)
</span><span style="color:#323232;">
</span><span style="color:#323232;"># Default from the wg container
</span><span style="color:#323232;">PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE
</span><span style="color:#323232;">
</span><span style="color:#323232;">## Add this section
</span><span style="color:#323232;"># Adds fwmark 51820 to any packet traveling through interface wg0
</span><span style="color:#323232;">PostUp = wg set wg0 fwmark 51820
</span><span style="color:#323232;"># If a packet is not marked with fwmark 51820 (not coming through the wg connection) it will be routed to the table "51820".
</span><span style="color:#323232;">PostUp = ip -4 rule add not fwmark 51820 table 51820
</span><span style="color:#323232;">PostUp = ip -4 rule add table main suppress_prefixlength 0
</span><span style="color:#323232;"># Creates a table ("51820") which routes all traffic through the vpn container
</span><span style="color:#323232;">PostUp = ip -4 route add 0.0.0.0/0 via 172.22.0.100 table 51820
</span><span style="color:#323232;"># If the traffic is destined for the subnet 192.168.1.0/24 (internal) send it through the default gateway.
</span><span style="color:#323232;">PostUp = ip -4 route add 192.168.1.0/24 via 172.22.0.1
</span><span style="color:#323232;">
</span><span style="color:#323232;"># Default from the wg container
</span><span style="color:#323232;">PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth+ -j MASQUERADE
</span>
The above config is a slightly modified setup from this linuxserver.io tutorial
Gluetun Setup
If you’ve setup your gluetun container properly the only thing you have to do is create {docker config path}/gluetun_vpn/firewall/post-rules.txt
containing the following:
<span style="color:#323232;">iptables -t nat -A POSTROUTING -o tun+ -j MASQUERADE
</span><span style="color:#323232;">iptables -t filter -A FORWARD -d 172.22.0.2 -j ACCEPT
</span><span style="color:#323232;">iptables -t filter -A FORWARD -s 172.22.0.2 -j ACCEPT
</span>
These commands should be automatically run once you restart the gluetun container. You can test the setup by running iptables-legacy -vL -t filter
from within the gluetun container. Your output should look like:
<span style="color:#323232;">Chain INPUT (policy DROP 7 packets, 444 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">27512 2021K ACCEPT all -- lo any anywhere anywhere
</span><span style="color:#323232;">43257 24M ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 291 28191 ACCEPT all -- eth0 any anywhere 172.22.0.0/24
</span><span style="color:#323232;">
</span><span style="color:#323232;"># These are the important rules
</span><span style="color:#323232;">Chain FORWARD (policy DROP 12276 packets, 2476K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">17202 8839K ACCEPT all -- any any anywhere 172.22.0.2
</span><span style="color:#323232;">26704 5270K ACCEPT all -- any any 172.22.0.2 anywhere
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain OUTPUT (policy DROP 42 packets, 2982 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">27512 2021K ACCEPT all -- any lo anywhere anywhere
</span><span style="color:#323232;">53625 9796K ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED
</span><span style="color:#323232;"> 0 0 ACCEPT all -- any eth0 c6d5846467f3 172.22.0.0/24
</span><span style="color:#323232;"> 1 176 ACCEPT udp -- any eth0 anywhere 64.42.179.50 udp dpt:1637
</span><span style="color:#323232;"> 2463 148K ACCEPT all -- any tun0 anywhere anywhere
</span>
And iptables-legacy -vL -t nat
which should look like:
<span style="color:#323232;">Chain PREROUTING (policy ACCEPT 18779 packets, 2957K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain INPUT (policy ACCEPT 291 packets, 28191 bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">
</span><span style="color:#323232;">Chain OUTPUT (policy ACCEPT 7212 packets, 460K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">
</span><span style="color:#323232;"># This is the important rule
</span><span style="color:#323232;">Chain POSTROUTING (policy ACCEPT 4718 packets, 310K bytes)
</span><span style="color:#323232;"> pkts bytes target prot opt in out source destination
</span><span style="color:#323232;">13677 916K MASQUERADE all -- any tun+ anywhere anywhere
</span>
The commands in post-rules.txt
are a more precises version of @CumBroth solution in the comments.