Making all network traffic for a Linux user use a specific network interface

I’ve recently been testing out a VPN service, and normally while running the VPN, all internet traffic goes over the VPN interface. This isn’t really ideal, as I only want traffic from specific applications to use the VPN. IPTables doesn’t seem to have the option to filter specific processes, but it can filter based on a specific user account.

IPTables itself doesn’t really deal with routing packets to interfaces, so we can’t use it to directly route packets. We can however mark packets from the user so they can be routed by the ip routing table. I’ve created a script to flush and apply firewall rules, which does what we need ( obviously set the variables at the beginning of the script to match your details ):

#! /bin/bash

export INTERFACE="tun0"
export VPNUSER="vpnuser"
export LANIP="192.168.1.0/24"
export NETIF="br0"

iptables -F -t nat
iptables -F -t mangle
iptables -F -t filter

# mark packets from $VPNUSER
iptables -t mangle -A OUTPUT ! --dest $LANIP  -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT --dest $LANIP -p udp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT --dest $LANIP -p tcp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT ! --src $LANIP -j MARK --set-mark 0x1

# allow responses
iptables -A INPUT -i $INTERFACE -m conntrack --ctstate ESTABLISHED -j ACCEPT

# allow bittorrent
iptables -A INPUT -i $INTERFACE -p tcp --dport 59560 -j ACCEPT
iptables -A INPUT -i $INTERFACE -p tcp --dport 6443 -j ACCEPT

iptables -A INPUT -i $INTERFACE -p udp --dport 8881 -j ACCEPT
iptables -A INPUT -i $INTERFACE -p udp --dport 7881 -j ACCEPT

# block everything incoming on $INTERFACE
iptables -A INPUT -i $INTERFACE -j REJECT

# send DNS to google for $VPNUSER
iptables -t nat -A OUTPUT --dest $LANIP -p udp --dport 53  -m owner --uid-owner $VPNUSER  -j DNAT --to-destination 8.8.4.4
iptables -t nat -A OUTPUT --dest $LANIP -p tcp --dport 53  -m owner --uid-owner $VPNUSER  -j DNAT --to-destination 8.8.8.8

# let $VPNUSER access lo and $INTERFACE
iptables -A OUTPUT -o lo -m owner --uid-owner $VPNUSER -j ACCEPT
iptables -A OUTPUT -o $INTERFACE -m owner --uid-owner $VPNUSER -j ACCEPT

# all packets on $INTERFACE needs to be masqueraded
iptables -t nat -A POSTROUTING -o $INTERFACE -j MASQUERADE

# reject connections from predator ip going over $NETIF
iptables -A OUTPUT ! --src $LANIP -o $NETIF -j REJECT

Now all packets from the user will be marked for the VPN. We also need to add a routing table, by adding the table name to the rt_tables file. In Gentoo this file is in /etc/iproute2/rt_tables , other distros will have it in different places, on my machine the file looks like this, the last line being the one added:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
200     vpnuser

Next we need a script to configure the routing rules for the marked packets:

#! /bin/bash
VPNIF="tun0"
VPNUSER="vpnuser"
GATEWAYIP=`ifconfig $VPNIF | egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' | egrep -v '255|(127\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})' | tail -n1`
if [[ `ip rule list | grep -c 0x1` == 0 ]]; then
ip rule add from all fwmark 0x1 lookup $VPNUSER
fi
ip route replace default via $GATEWAYIP table $VPNUSER 
ip route append default via 127.0.0.1 dev lo table $VPNUSER
ip route flush cache

If you are using OpenVPN, you will need to ensure this line is in your config file, to prevent all traffic from sending over the VPN by default:

route-nopull

You may also need to add these lines into /etc/sysctl.d/9999-vpn.conf to ensure the kernel lets the traffic get routed correctly ( this disables reverse path filtering ):

net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.br0.rp_filter = 0

Then run:

sysctl -p

To apply the new sysctl rules. You may also need to restart your VPN if you are already connected.

Now run the two scripts ( the second script needs to run when the network interface starts – this is in /etc/conf.d/net on Gentoo, or the ‘up’ command in OpenVPN’s config file ) , and the specific user should only be able to access traffic on the VPN, and other users on the system should access the network as normal.

58 thoughts on “Making all network traffic for a Linux user use a specific network interface”

  1. Hi,

    In your script, you are trying to set the mark for all packets which don’t match the Tunnel IP. So won’t this match all packets generated by other users ?
    iptables -t mangle -A OUTPUT ! –src $NETIP -j MARK –set-mark 0x1

    I am trying to do VPN split tunneling, and trying to follow your script. I am facing problem trying to match the last ACK packet, which doesn’t have the UID.

  2. Hi,

    $NETIP is the LAN ip address of the network card, so if the other users send packets from an ip address which is the VPN’s IP it is sent via the vpn.

    I’ve updated the article with the latest version of my script, hope this helps.

    Simon

  3. Hi Simon,

    Thanks for this article – I’m now using your techniques to select a network interface by UID.

    I’m curious about the masquerading line. Is there a reason why you don’t restrict this to the target UID, rather then masquerading everything on that interface? Presumably there is an overhead associated with this.

    Cheers,

    Rob

  4. Hi Rob,

    Glad my article has helped.

    It’s been a while since I wrote the script, but I think it was just put in as an extra safety measure to ensure everything on the interface comes from the correct IP. I don’t know of any reason why it couldn’t match the UID when it does the postrouting rule, but I haven’t tested it. Performance wise, I haven’t noticed any issues masquerading everything, but it might be a worthwhile optimisation to put in.

    Thanks

    Simon

  5. I am trying to do the same thing. I don not know what values to enter for the NETIF and VPNIF in the scripts.
    I am trying to get one application running as user Transmission to use a VPN tunnel (tun0) and all other applications running as user User to use eth) without using the tunnel.
    Any help would be appreciated.

  6. Hi Steve,

    NETIF should be the name of the primary network interface on your machine ( usually eth0, br0, wlan0 or something similar ).

    VPNIF should be the interface for the vpn tunnel ( sounds like tun0 in your case ).

    Hope this helps

    Simon

  7. Hi, thank you for this. πŸ™‚

    I put the (slightly modified) scripts in my OpenVPN config, so it runs when that starts (I added the second script on the last line on the first script, since ovpn don’t allow multiple scripts).
    When I ping something from my vpnuser when VPN is connected, I get responce. When I disconnect VPN I get 100% packet loss, but all other users work.

    I’m wondering if the other users are supposed to get a non-VPN internet? Or should they just not be restricted when the VPN is down. Because as long as the VPN is up they also get routed through the VPN.

    If that is the case, could you please take a look on my commented files below and see if you spot anything. Thank you!

    Here are my scripts and OpenVPN conf file:
    Script 1: http://pastebin.com/VCxM4N2B
    Script 2: http://pastebin.com/hgt9SLwG
    OpenVPN Conf: http://pastebin.com/qqw7w1S9

  8. Hello Simon!

    Thank you very much for your script. I’m also trying to route a user running transmission only via the tun0 interface, but it’s not working properly.

    I’m on Ubuntu, and first I run:
    sudo service openvpn start
    to create the tun0 interface. Now traffic from all users are routed through tun0.

    Then I run your two scripts:
    sudo ./script1.sh
    sudo ./script1.sh
    But they have no effect. All users are still using tun0.

    What do you think could be wrong? I’ve added the vpnuser to rt_tables. The initial parameters of the first script is defined as follows:

    export INTERFACE="tun0"
    export VPNUSER="vpnuser"
    export LANIP="192.168.19.0/24"
    export NETIF="eth0"

    And the second:

    VPNIF="tun0"
    VPNUSER="vpnuser"

  9. Hi Jon,

    From your openvpn config it looks like you need to add ‘route-nopull’ to your openvpn config, that should stop the non-vpn users getting routed via the VPN.

    You may also need to add these lines into /etc/sysctl.d/9999-vpn.conf to ensure the kernel lets the traffic get routed correctly:

    net.ipv4.conf.all.rp_filter = 0
    net.ipv4.conf.default.rp_filter = 0
    net.ipv4.conf.eth0.rp_filter = 0
    

    And then run:

    sysctl -p

    I’ve also just updated the scripts with a few small changes, so you may want to try the latest script and see if that makes any difference.

    Hope this helps

    Simon

  10. Hi miceagol,

    As with my message to Jon above, you may need to add ‘route-nopull’ to your openvpn config – by default openvpn will pull the remote routes, which usually force all traffic over the vpn.

    Hope this helps

    Simon

  11. I have been looking for a solution for this for a long time, this is the best one I have come across yet – it just worked, so thanks!! I have forwarded rtorrent port through and everything is working well, the only issue is that rutorrent (the rtorrent web interface) says the port is closed, when it is blatantly open and working. I assume that this is because rutorrent is working under the web -server user which is a different user than the rtorrent process. Not really sure how to fix this as dont want to forward my entire web-server through VPN.

  12. In relation to the above, I resolved this issue simply by adding a route. The rutorrent check_port tool uses canyouseeme.org (which has an ip address of 107-20.89.142). I therefore issued at CLI, ip route -p add 107.20.89.142 tun0 and now interface can connect to site, so port is shown as open. Thanks again for the solution provided on this page.

  13. Thank you for this post. I finally had some success in selective routing on my gentoo machine. I have a few follow up questions though:
    # block everything incoming on $INTERFACE
    iptables -A INPUT -i $INTERFACE -j REJECT
    Why are we blocking every input on tun0 (except for those accepted of course) ? Will there ever be traffic sent to that interface that one did not ask for using port forwarding. As i understand openvpn connections are not supposed to initiate connections?

    Regards

  14. Simon, thanks so much for these scripts. I have them working nicely for SABnzbd. It’s great because, if the VPN connection drops, SABnzbd just stops. Nice! I have a question, though. How can I modify the scripts to handle more than one user? For example, what if I want to cover the sabnzbd and transmission users? Is it as simple as duplicating all lines that contain $VPNUSER, changing them to $VPNUSER1 and $VPNUSER2, and doing the same for the routing tables?

    Also, is it necessary to only allow certain ports? I guess my question is similar to maynardj’s. If ALL traffic for the given users if forced through the VPN, then does it matter which ports are used? Just curious.

  15. Hi maynardj,

    The reason for rejecting all incoming packets except those specifically allowed is just a safety thing to prevent accidentally exposing of ports – this guide was originally written while I was using PPTP, and the VPN provider I was using at the time forwarded all ports by default, exposing locally running services over the VPN unless they are firewalled off.

    Thanks

    Simon

  16. Hi dildano,

    Great, glad it helps.

    Yes, duplicating the rules should work ( you will probably just need to duplicate the iptables rules though, the route command shouldnt need to change, as long as you just want to send the data over one connection ), although a better solution might be to use the ‘gid-owner’ iptables parameter instead of ‘uid-owner’ to match a group, and add the vpn users to a single group.

    As with my response to maynardj, the reason for only allowing specific ports is a safety thing to ensure only traffic you want to specifically allow to connect is allowed.

    Hope this helps

    Simon

  17. Thank you very much for this post Simon! It was quite helpful.

    I just wanted to share what solution I came up with based on what you provided plus other research I did. I came up with somewhat of a hybrid based on what you provided. If the program you want to route through vpn supports binding to a specific IP address there is a way to accomplish the same thing (without having to deal with marking packets). This is beneficial as you may have noticed that using the method in the original post, due to the time at which the marking is applied/processed, any iptables OUTPUT rules will still think the packet is going out over the eth0 interface instead of tun0 (at least in my case, perhaps other distros are different). So if you tried to do something like -A OUTPUT -o eth0 -m owner –uid-owner vpnuser -j DROP in iptables with the marking system (method in original post) iptables will actually drop the packets even though they are going out tun0, as it thinks they are going out eth0 still (at the time iptables OUTPUT rules are applied). See example at very end of post if you want more info on this (by logging dropped packets).

    You still use most of the stuff from the original post. EX: You will probably want most of the same iptables rules (other than not needing any of the mangle rules to mark packets). You will also still need to populate the vpnuser routing table using whatever method you want, in my case i use the “route-up” option in the openvpn config file to launch a .sh file plus “route-noexec” so it doesn’t fill the default routing table with the openvpn route. The handy thing about route-up is that it will pass a bunch of environment variables with pretty much all the info/ip addresses you need to create a proper routing table etc. All the variables are explained in the openvpn documentation, but if your are lazy like me, just pipe “env” to your system logging utility (or just redirect it into a file using > ) and then you can see all the variables, and more importantly their values in your system log file. Sample env.sh file to launch with “route-up” that will write all variables to system log (assuming your distro uses logger). Don’t forget to chmod any .sh files you create for this to be executable, otherwise it obviously wont work. Honestly its probably cleaner to just dump it into a file, but I’m lazy and was already digging around in the system log for other info so this is what I did. Feel free to use > instead.
    #!/bin/sh
    env | logger

    Just call env.sh using the route-up command in the openvpn config file and it should write everything to the system log for your perusal. Then you can use the info from that to create proper up.sh/down.sh files that will be called with openvpn’s “route-up” and “down” commands.

    As I’m using this to run rtorrent through my vpn tunnel (tun0) i have rtorrent running as it’s own user for the sake of this argument i’ll call it “vpnuser” as you have.

    I created a VLAN, ex: eth0.4 (or eth0.{whatever # you want}) if you don’t have any VLANs, then eth0.1 is fine.
    I then assigned myself a static ip on that vlan. The following files may be stored in a different location depending on your distribution, so be sure to check where the proper place for these is, I am running CentOS, so this is where they happen to be for me. If you’re not sure just try “ls /etc/sysconfig/network-scripts/” and see if there are already other files located there like “ifcfg-eth0” and others. If there aren’t, you probably need to find the appropriate directory for your distro.

    sample files to make VLAN eth0.4 with a static ip of 10.0.10.2
    Note: in this case i used a netmask of 255.255.255.240, you may want to use a different netmask depending on your situation, but I’m pretty much just using 1 ip on this vlan so that netmask works fine in this situation
    “/etc/sysconfig/network-scripts/ifcfg-eth0.4”
    DEVICE=eth0.4
    BOOTPROTO=static
    ONBOOT=yes
    TYPE=Ethernet
    IPV6INIT=no
    IPADDR=10.0.10.2
    NETMASK=255.255.255.240
    NETWORK=10.0.10.0
    VLAN=yes

    automatically add a rule for eth0.4 VLAN to use “vpnuser” table for routing. Instead of having to call “ip rule add from 10.0.10.2 table vpnuser” from some script all the time, this will simply apply whenever the eth0.4 VLAN is active.
    “/etc/sysconfig/network-scripts/rule-eth0.4”
    from 10.0.10.2 table vpnuser

    Then in my rtorrent config I set it up to always bind to 10.0.10.2 (ip of VLAN eth0.4) so I can be sure it will using the proper “vpnuser” routing table based on what we did in the 2 files above. obviously every application will have a different way to tell it what ip to bind to so i wont include a sample config for this.

    You may find the following command useful for testing. wget –bind-address=10.0.10.2 -qO- http://ipecho.net/plain
    That uses wget bound to the 10.0.10.2 ip so you can test the routing from that ip, and the rest of the command, assuming it can actually reach the internet, will display the public ip address used to get there (just echoes the data from the ipecho.net/plain website which is simply the ip that accessed that site). If you are seeing your VPN’s public ip address then you’re good to go. If you still see you regular non-vpn public ip address then your “vpnuser” routing table isn’t set up right.

    You still should run whatever programs you want going through the vpn as the “vpnuser” user, and then use an iptables rule to drop all packets from that user going out over eth0. You will have to make sure the programs running as vpnuser are bound to the eth0.4 VLAN or all the packets will simply be dropped as doing it this way doesn’t make all packets from vpnuser go through tun0, just the ones coming from the 10.0.10.2 ip, but this way you can ensure any programs you were running as that user never go out over eth0.

    That should cover most of what you need to know, but I’ll happily clarify anything. No promises how often ill check back here though, i was hoping this post could mainly just be used as a reference.

    side bar on logging specific dropped packets with iptables: instead of dropping packets by forwarding them to the DROP chain, you can forward packets to some other chain you made in iptables for logging instead (that logs the packets and then drops them). You can accomplish this by creating for example a chain named “LOGGING” and then adding the following rules (the limit function below was used to stop it from completely filling the log up if there are tons of packets being dropped, feel free to adjust as necessary):
    -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Rule-Dropped: " --log-tcp-options
    -A LOGGING -j DROP

    Then, using the previous example above just change it to -A OUTPUT -o eth0 -m owner –uid-owner vpnuser -j LOGGING. now all packets from user “vpnuser” going out over the eth0 interface will be forwarded to the LOGGING chain where they are dropped and logged to /var/log/messages with the “IPTables-Rule-Dropped:” in the line (so you can easily search for it in your text editor of choice). This will contain info about the packet. I have included a sample line from the log below (local network address in log is 10.0.0.30):
    Mar 22 04:50:47 MACHINE-NAME kernel: IPTables-Rule-Dropped: IN= OUT=eth0 SRC=10.0.0.30 DST=146.255.36.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=35186 DF PROTO=TCP SPT=47708 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 OPT (020405B40402080A009BE7930000000001030306) MARK=0x1 It will look similar to this and as you can see the packet is marked, but the at the time the iptables output rule is applied it doesn’t know yet that this will actually be routed over tun0, it still thinks it’s going out over eth0 if you are using marking like OP, to decide which routing table to use. if you use “ip rule from table vpnuser” then the iptables rule will know where the packet is actually going and apply correctly (not drop packets that will actually be going out over tun0).

  18. I wasn’t able to get ‘gid-owner’ working, and I’m too much of a Linux novice to figure it out. However, duplicating the iptables rules for the different users appears to be working fine. I’m already running the route script using the ‘up’ command in the OpenVPN config. Is there any reason that the iptables and route scripts couldn’t be combined? Or is there a better way to invoke the iptables script?

    Thanks again for your work on this. It’s really quite useful.

  19. Hey dildano, I’m not sure what you tried in order to get gid-owner working, but probably the issue you had when attempting to use gid-owner, is that it is based on the gid of the running process, and not necessarily the gid of a group that user that ran the process is a member of. Typically the gid of a running process will simply be the same number as the uid of the user who ran the process. I.e if user “vpnuser” has a uid of 501, then the default gid of all processes run by “vpnuser” will also be 501, even if “vpnuser” is a member of other specific groups you may have added him to. If however you for example use the chgrp command on the executable transmission file (probably located at /usr/bin/transmission or somewhere similar) to change the group of the transmission file to a group of your choosing, then when transmission is run it should have the uid of the user who ran it, but the gid of the group you changed the file to. you will probably find “ps -eo uid,gid,args” usefull for figuring out whats going on. This will give you a list of all the running processes along with the uid and gid for each process so you can confirm your changes had the desired effect relating to the change in group for the executable file. Once the process is running with your desired gid then gid-owner will probably work as you are expecting it to. More than likely you just added a user to a group, but that doesn’t give processes run by that user a gid; the gid for a process is based on the gid of the actual executable process file, not the gid of a group the user who ran it is in. Hope this helps πŸ™‚

  20. Hello Simon.
    Thank you so much for making this tutorial/soloution. I spend many hours googling trying to figure out a way to solve this problem with no luck until I stumbled upon this page. Everything works like a charm except for one thing. I am running transmission and I would like to connect to port 9091 from an external IP so I can use the web-interface. I have tried to allow port 9091 using this rule: “iptables -A INPUT -i $NETIF -p tcp –dport 9091 -j ACCEPT” but it doesn’t work. It must be the firewall blocking port 9091 because when I disbale the firewall I am able to connect to port 9091 from an externa IP. I am able to connect through port 9091 on my LAN without problems. I have forwarded port 9091 in my router. I don’t think I will be able to solve this by myselv so any help would be much appreciated.
    //Povl

  21. Hi Povl,

    Are you trying to connect to port 9091 from the VPN’s external IP? If so you will need to use this rule instead:

    iptables -A INPUT -i $INTERFACE -p tcp –dport 9091 -j ACCEPT

    Hope this helps

    Simon

  22. I have been trying for days to get this script to work. My VPN is up and running, and it works if I route all my traffic through it.

    Once I run the scripts, however, and su to the vpnuser, I can’t even ping a server. If I start the VPN and switch to vpnuser without running the scripts, I can ping out on the eth0 interface. Likewise, if I take route-nopull out of my VPN config, I can ping out over tun0. But once I run the scripts, I can’t ping out at all!

    I’m on ubuntu, if that makes a difference.

    Any ideas?

  23. Hi John,

    Which version of Ubuntu are you using? Could you also send over the output of these commands ( replacing br0 with your network interface if it is something else )? :


    sudo iptables-save
    sudo ip route show table vpnuser
    cat /proc/sys/net/ipv4/conf/{all,default,br0}/rp_filter

    Thanks

    Simon

  24. Hrm. I just set up the scripts again to send you the output, and now it’s working, all of the sudden! Is it possible that I have been running the two scripts in the wrong order before this successful attempt? Would that actually prevent the rules from working? I’m going to do a hard reset after my backup drive finishes up and let you know how it goes.

    Either way, thanks for your help, and I’ll post the output of those commands if it does indeed fail again.

    John

  25. Hrm. Using these instructions on a fresh install of XBMCbuntu, the vpnuser has no traffic, but other users do. Unsure of what the problem is. Is this writeup still valid?

  26. Hi fingerboxes,

    I’m using the config from the post unchanged currently and it’s still working for me, I’d suggest double-checking everything, and verifying rp-filter is disabled correctly on your VPN network interface, eg:

    cat /proc/sys/net/ipv4/conf/tun0/rp_filter
    

    Should return:

    0
    

    Obviously replacing ‘tun0’ with your own VPN’s interface.

    Simon

  27. It looks like by default, Gentoo has the following set in /etc/sysctl.conf:

    # Enables source route verification
    net.ipv4.conf.default.rp_filter = 1
    # Enable reverse path
    net.ipv4.conf.all.rp_filter = 1

    This will override what ever is put in 9999-vpn.conf. What I did was comment out those two lines and everything worked.

  28. Exactly what I need and want to achieve, but I’m quite a linux n00b, so I would like to ask for some help.
    I configured a home server running Ubuntu Server Trusty following these tow guides more or less: http://www.havetheknowhow.com/default.htm and http://linuxhomeserverguide.com/
    Everything is running excellent, and I’m very much satisfied with the results.
    I have a subscription to a VPN server that I use when running Deluge. Obviously, I woud like to run only Deluge, Flexget, and Sonarr over openVPN connection, and everything else to have direct access to internet.

    I’m somewhat confused with the user vpnuser. On my system, I use user “server” as regular user. But I use user “deluge” for Deluge and Flexget, and atm Sonarr runs under user “server” but will add Sonarr to user “sonarr”. How can I achieve that Deluge, Flexget and Sonarr to run only over openVPN, and everything else not?
    Also, I would like to be able to access webui’s of Deluge and Sonarr while they are inside of VPN, so this would be also nice to know how.

    I realize it is boring, but would help so much of us if you (or somebody here) could create a step-by-step guide for this like on the two linked sites above where each command is described, and not just add this to, configure that:) It might be obvious to most of the users, but for rookies it is not and confusing. In addition, when I can follow a step-by-step guide then I can learn the best as I always check and see what each line does.

    Let’s assume that we have openVPN installed and we are able to connect to VPN and all the traffic is tuned over that VPN and let’s start from there.
    We have a user “deluge” who runs Deluge and Flexget, and we either add Sonarr to user “deluge” or we also have a second user “sonarr”. And how to proceed from here?
    Many thanks!

  29. Hi Simon,

    I seem to be having the same issue as John was having before his setup mysteriously started working. I’m running Ubuntu 14.04.2 LTS.

    And mine just started working again too. I think that the `sysctl.d/9999-vpn.conf` requires more than just a `sysctl -p` to reset. I couldn’t get `cat /proc/sys/net/ipv4/conf/{all,default,eth0,tun0}/rp_filter` to return anything other than `1`s for anything I tried until I rebooted.

    So, if your `rp_filter` is always coming back `1`, try a reboot and a `sysctl -p`… Or maybe I was stupid and should have been running `sudo sysctl -p`. Could be!

    -sean

  30. I have one more question, and I think I’m just not smart enough to figure this out. I would also like access to my webserver (port 443) from an outside IP. I can access it just fine from inside the LAN, but I can’t seem to make it work from outside the IP with these iptables rules. I’m assuming that something uncomplicated is the answer here, but I can’t seem to figure out all of the parts of what I need. Plus I’m not 100% sure how to fully reset iptables so that I’m really testing what I think I’m testing. Here are my iptables right now:

    Chain INPUT (policy ACCEPT 3 packets, 222 bytes)
    pkts bytes target prot opt in out source destination
    0 0 ACCEPT all -- tun0 any anywhere anywhere ctstate ESTABLISHED
    0 0 ACCEPT tcp -- tun0 any anywhere anywhere tcp dpt:59560
    0 0 ACCEPT tcp -- tun0 any anywhere anywhere tcp dpt:6443
    0 0 ACCEPT udp -- tun0 any anywhere anywhere udp dpt:8881
    0 0 ACCEPT udp -- tun0 any anywhere anywhere udp dpt:7881
    0 0 ACCEPT tcp -- eth0 any anywhere anywhere tcp dpt:https
    0 0 REJECT all -- tun0 any anywhere anywhere reject-with icmp-port-unreachable

    Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target prot opt in out source destination

    Chain OUTPUT (policy ACCEPT 2 packets, 154 bytes)
    pkts bytes target prot opt in out source destination
    0 0 ACCEPT all -- any lo anywhere anywhere owner UID match vpnuser
    0 0 ACCEPT all -- any tun0 anywhere anywhere owner UID match vpnuser
    0 0 ACCEPT tcp -- any eth0 anywhere anywhere tcp dpt:https

    You can tell that I’ve messed around a bit with trying to put the https in the INPUT and the OUTPUT chains and removing the last line of the first script iptables -A OUTPUT ! --src $LANIP -o $NETIF -j REJECT. I still can’t seem to make it work. Is something I’m not understanding? Thanks so much!

  31. I am a giant idiot. I recently set up a static IP for my server and forgot to change the information and reset the firewall on my router. This works perfectly.

  32. hi,
    I think I managed to configure everything, and I believe running ok. Excellent guide, many thanks again!

    I do have some questions, please help me on this:

    1) I have user deluge who is running deluge. The script is configured for user deluge, goes over vpn. As I can’t login with ssh as user deluge, how can I check the external ip of user deluge, let’s say with my default user? Running Ubuntu Server Trusty headless. My default user does give me my ISP ip, so it is fine, and judging from Deluge download speeds, it is connected over vpn, as it is slower. And if I stop openVPN, then download stops as well on Deluge. But would be nice if I could get the external ip for vpn user deluge.

    2) I see a few ports in the first script for bittorent. Does that mean that I need to configure those ports for Deluge to be active? Looking at the script:

    # allow bittorrent
    iptables -A INPUT -i $INTERFACE -p tcp –dport 59560 -j ACCEPT
    iptables -A INPUT -i $INTERFACE -p tcp –dport 6443 -j ACCEPT

    iptables -A INPUT -i $INTERFACE -p udp –dport 8881 -j ACCEPT
    iptables -A INPUT -i $INTERFACE -p udp –dport 7881 -j ACCEPT

    This means that Deluge will use tcp port 59560 and 6443 and upd ports 8881 and 7881? Or it is something else?

    Many thanks!

  33. Hello Simon!

    I just wanted to say thank you very much for this post. My linux knowledge is at best extremely rudimentary and I’ve searched, without exaggerating, 6+ months for a solution like this. I’ve tried many guides, and many methods, even including this one but also namespaces, vm’s etc etc. I’m sure all those guides were working but I never got them to work.

    Your guide however just worked, without me having to tweak anything. And on the plus side, I sort of understand how the scripts work and am able to follow what specific lines do. And also, having all traffic stopped for the specific user I am using this for whenever the VPN client is turned off is a bonus (this was propably intended when u wrote the scripts, but it didn’t quite come across from just reading the post).

    Anyways.. I’ve rambled enough. Seriously, thank you so so very much:)

  34. Hey, is there perhaps a trick that would allow to do this thing without disabling rp_filter ?

    I feel very very uneasy about it.

    There appear to be some pointers about how to do so in this serverfault exchange (has to do with PREROUTING -j CONNMARK –restore-mark )
    http://serverfault.com/a/456330

    But I’m apparently too dumb to properly follow through….

  35. Hi LonelyPasserby,

    Not that I’ve managed to get to work properly, I’ll have a look at that serverfault.com page though, it looks useful.

    Thanks

    Simon

  36. Hi Simon,

    I have a question about this section of the script:

    iptables -F -t nat
    iptables -F -t mangle
    iptables -F -t filter

    By doing this when connecting to OpenVPN on startup will any other iptables rules be flushed? I worry about leaving other ports open on my server. If I had to make other rules for other ports (ssh, http, https, etc) would I have to do it in this script, or do I do it elsewhere?

    Right now I have everything working fine on my test box. the vpnuser traffic is routed as expected.

    I’m running Ubuntu Server 14.04 if that helps.

    Any help would be appreciated.

  37. Hi fatgeek,

    Yes that flushes all the iptables rules, I’d suggest if you have other rules to use then add them into the same script.

    Simon

  38. I had issues using the script with openvpn.
    I needed to change the ‘route-nopull’ to ‘route-noexec’ in the openvpn client config, then I grabbed the correct gateway ip from the ‘$route_vpn_gateway’ openvpn environment variable.
    I also added the command ‘sysctl -w “net.ipv4.conf.$VPNIF.rp_filter=2″‘

  39. This works mostly like a charm in Ubuntu Server, but when running a torrent, I’m noticing some leaks somewhere.

    Following traffic with iftop, miscellaneous P2P IPs appear even though all traffic appears to be going through the tunnel. It’s all outgoing data and it’s only 10’s or 100’s of byes (no response data), so I’m not sure what’s being sent, but *something* is going through the ethernet port.

    I’ve tried with all the fancy bittorrent features turned off just in case it’s neighbor stuff, peer finding, etc, but no change. As long as I allow traffic for users that *aren’t* my vpnuser, these show up. The second I lock all traffic down to the VPN (by removing the default gateway), it stops.

    I can’t seem to figure out where the leaks are coming from, but it’s a real hair puller!

    You seem quite a bit more competent than I, so if you have any thoughts or suggestions of what I could do to at least find out the source of these miscellaneous bytes, that would help immensely!

  40. Fixed it! Turns out it was some (presumably retransmitted?) closing ACKs. Nothing to be too worried about, but still I feel more comfortable knowing what it was and cleaning it up.

    I added the following before the mangling in the first script:

    iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

    and the following after the mangling in the first script:

    iptables -t mangle -A OUTPUT -j CONNMARK --save-mark

    That cleaned the stray packets nicely, so now there’s no more apparent leaks!

    Thanks for your great work on this! It’s saved me (and many others) a ton of trouble!! Kudos! πŸ™‚

  41. Hi Simon,

    Thanks for this great tuto.
    On my side, I currently try to set rules according to ports instead of user.
    I have then write the following script using what you provided plus other research:
    ovpn_config_script.sh which is launch during the boot:
    #!/bin/bash

    #This script configure the Linux routing policy such as only the download traffic (Usenet, Direct Download, BitTorrent) goes through the VPN connection

    #Environment variables (need to fit your configuration)
    export VPN_INTERFACE="tun0"
    export NETIF="eth0"
    export LANIP="192.168.0.0/24"
    export VPN_TABLE="VPNTABLE"
    export VPN_PORT_LIST="80,443,119,563,6881-7000"
    #VPN_PORT_LIST (list of ports which traffic shall go through the VPN):
    #80: http, 443: httsp (ports used for Direct Download)
    #119: nntp, 563: nntps (ports used for Usenet)
    #6881 to 7000: ports used for BitTorrent traffic

    # First it is necessary to disable Reverse Path Filtering on all current and future network interfaces:
    for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
    echo 0 > $i
    done

    # Delete table $VPN_TABLE and flush any existing rules if they exist.
    ip route flush table $VPN_TABLE
    ip route del default table $VPN_TABLE
    ip rule del fwmark 1 table $VPN_TABLE
    ip route flush cache
    iptables -t mangle -F PREROUTING

    #Default behavior: MARK = 1 all traffic bypasses VPN, MARK = 0 all traffic goes VPN
    #All traffic for Direct Download, Usenet and BitTorrent goes into the VPN
    iptables -t mangle -I PREROUTING -i $NETIF -p tcp -m multiport --dport $VPN_PORT_LIST -j MARK --set-mark 0
    #The traffic of all ports out of $VPN_PORT_LIST bypass the VPN
    iptables -t mangle -I PREROUTING -i $NETIF -p tcp -m multiport --dport ! $VPN_PORT_LIST -j MARK --set-mark 1
    # All UDP and ICMP traffic will bypass the VPN
    iptables -t mangle -I PREROUTING -i $NETIF -p udp -j MARK --set-mark 1​
    iptables -t mangle -I PREROUTING -i $NETIF -p icmp -j MARK --set-mark 1​

    # all packets on $INTERFACE needs to be masqueraded
    iptables -t nat -A POSTROUTING -o $VPN_INTERFACE -j MASQUERADE

    # reject connections from predator ip going over $NETIF
    iptables -A OUTPUT ! --src $LANIP -o $NETIF -j REJECT

    #Now the Linux routing policy is properly set, let's start the VPN:
    openvpn --config /etc/openvpn/ipvanish-BE-Brussels-bru-b01.ovpn --ca /etc/openvpn/ca.ipvanish.com.crt --up /etc/openvpn/ovpn_up_script.sh

    ovpn_up_script.sh which is launched when the VPN is up:
    #! /bin/bash

    # Script which shall be launch after the VPN is up and running to configure the routing rules for the marked packets

    #Environment variables (need to fit your configuration)
    export VPN_INTERFACE="tun0"
    export VPN_TABLE="VPNTABLE"
    export VPN_GATEWAY_IP=`ifconfig tun0 | grep 'inet addr:' | cut -d: -f3 | awk '{print $1}'`

    # Copy all non-default and non-VPN related routes from the main table into table $VPN_TABLE.
    # Then configure table $VPN_TABLE to route all traffic out the VPN gateway and assign it mark "1"
    ip route show table main | grep -Ev ^default | grep -Ev $VPN_INTERFACE \
    | while read ROUTE ; do
    ip route add table $VPN_TABLE $ROUTE
    done
    ip route add default table $VPN_TABLE via $VPN_GATEWAY_IP
    ip rule add fwmark 1 table $VPN_TABLE
    ip route flush cache

    However, there are style open point for me.
    in the secaond script, you refer to "127.0.0.1 ", what is exactly this IP address?

  42. Hi Simon,

    I hope you still read this but here goes nothing ^^
    I followed your tutorial and it seems to be working. But when using nload while doing a speedtest with the $vpnuser, i can still see traffic on eth0. Is that normal ?
    I’m just looking for a way to be 100% sure that my $vpnuser isn’t using eth0 but only tun0.

    Thx

  43. Dear Sir,

    Thank You for Your wonderful tutorial! It helped me a lot!
    I would like to ask for an advice with following case:

    I have my own VPN network hosted on the very same machine that uses tun7 interface.
    My $VPNUSER is running couple of services that have a web-based admin interfaces.
    My $INTERFACE is tun1

    When I am using any device on my local network from $LANIP range “192.168.1.0/24” (established on eth0 that is also my $NETIF) I can access those services admin panels with no problem at all.

    When I connect to my own VPN network from outside (thus via tun7) those services become unreachable :-/
    I get time-outs all the time.

    Im a total noob in this topic so I tried to add access to tun7 for $VPNUSER by adding a similar line that already exist in Your script
    iptables -A OUTPUT -o tun7 -m owner –uid-owner $VPNUSER -j ACCEPT

    But it did not help.

    Both my scripts look exactly the same besides those two lines that I added to be able to reach outside world while being connected to my private VPN (tun7):
    iptables -t nat -A INPUT -i eth0 -p udp -m udp –dport 1194 -j ACCEPT
    iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT –to-source 192.168.1.XX (local IP of my server).

    Any advice will be much appreciated!

    Best regards,
    Lucas

Leave a Reply

Your email address will not be published. Required fields are marked *