Making all network traffic for a Linux user use a specific network interface

I’ve recently been testing out a VPN service, and normally while running the VPN, all internet traffic goes over the VPN interface. This isn’t really ideal, as I only want traffic from specific applications to use the VPN. IPTables doesn’t seem to have the option to filter specific processes, but it can filter based on a specific user account.

IPTables itself doesn’t really deal with routing packets to interfaces, so we can’t use it to directly route packets. We can however mark packets from the user so they can be routed by the ip routing table. I’ve created a script to flush and apply firewall rules, which does what we need ( obviously set the variables at the beginning of the script to match your details ):

#! /bin/bash

export INTERFACE="tun0"
export VPNUSER="vpnuser"
export LANIP="192.168.1.0/24"
export NETIF="br0"

iptables -F -t nat
iptables -F -t mangle
iptables -F -t filter

# mark packets from $VPNUSER
iptables -t mangle -A OUTPUT ! --dest $LANIP  -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT --dest $LANIP -p udp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT --dest $LANIP -p tcp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1
iptables -t mangle -A OUTPUT ! --src $LANIP -j MARK --set-mark 0x1

# allow responses
iptables -A INPUT -i $INTERFACE -m conntrack --ctstate ESTABLISHED -j ACCEPT

# allow bittorrent
iptables -A INPUT -i $INTERFACE -p tcp --dport 59560 -j ACCEPT
iptables -A INPUT -i $INTERFACE -p tcp --dport 6443 -j ACCEPT

iptables -A INPUT -i $INTERFACE -p udp --dport 8881 -j ACCEPT
iptables -A INPUT -i $INTERFACE -p udp --dport 7881 -j ACCEPT

# block everything incoming on $INTERFACE
iptables -A INPUT -i $INTERFACE -j REJECT

# send DNS to google for $VPNUSER
iptables -t nat -A OUTPUT --dest $LANIP -p udp --dport 53  -m owner --uid-owner $VPNUSER  -j DNAT --to-destination 8.8.4.4
iptables -t nat -A OUTPUT --dest $LANIP -p tcp --dport 53  -m owner --uid-owner $VPNUSER  -j DNAT --to-destination 8.8.8.8

# let $VPNUSER access lo and $INTERFACE
iptables -A OUTPUT -o lo -m owner --uid-owner $VPNUSER -j ACCEPT
iptables -A OUTPUT -o $INTERFACE -m owner --uid-owner $VPNUSER -j ACCEPT

# all packets on $INTERFACE needs to be masqueraded
iptables -t nat -A POSTROUTING -o $INTERFACE -j MASQUERADE

# reject connections from predator ip going over $NETIF
iptables -A OUTPUT ! --src $LANIP -o $NETIF -j REJECT

Now all packets from the user will be marked for the VPN. We also need to add a routing table, by adding the table name to the rt_tables file. In Gentoo this file is in /etc/iproute2/rt_tables , other distros will have it in different places, on my machine the file looks like this, the last line being the one added:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
200     vpnuser

Next we need a script to configure the routing rules for the marked packets:

#! /bin/bash
VPNIF="tun0"
VPNUSER="vpnuser"
GATEWAYIP=`ifconfig $VPNIF | egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' | egrep -v '255|(127\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})' | tail -n1`
if [[ `ip rule list | grep -c 0x1` == 0 ]]; then
ip rule add from all fwmark 0x1 lookup $VPNUSER
fi
ip route replace default via $GATEWAYIP table $VPNUSER 
ip route append default via 127.0.0.1 dev lo table $VPNUSER
ip route flush cache

If you are using OpenVPN, you will need to ensure this line is in your config file, to prevent all traffic from sending over the VPN by default:

route-nopull

You may also need to add these lines into /etc/sysctl.d/9999-vpn.conf to ensure the kernel lets the traffic get routed correctly ( this disables reverse path filtering ):

net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.br0.rp_filter = 0

Then run:

sysctl -p

To apply the new sysctl rules. You may also need to restart your VPN if you are already connected.

Now run the two scripts ( the second script needs to run when the network interface starts – this is in /etc/conf.d/net on Gentoo, or the ‘up’ command in OpenVPN’s config file ) , and the specific user should only be able to access traffic on the VPN, and other users on the system should access the network as normal.

25 thoughts on “Making all network traffic for a Linux user use a specific network interface”

  1. Hi,

    In your script, you are trying to set the mark for all packets which don’t match the Tunnel IP. So won’t this match all packets generated by other users ?
    iptables -t mangle -A OUTPUT ! –src $NETIP -j MARK –set-mark 0×1

    I am trying to do VPN split tunneling, and trying to follow your script. I am facing problem trying to match the last ACK packet, which doesn’t have the UID.

  2. Hi,

    $NETIP is the LAN ip address of the network card, so if the other users send packets from an ip address which is the VPN’s IP it is sent via the vpn.

    I’ve updated the article with the latest version of my script, hope this helps.

    Simon

  3. Hi Simon,

    Thanks for this article – I’m now using your techniques to select a network interface by UID.

    I’m curious about the masquerading line. Is there a reason why you don’t restrict this to the target UID, rather then masquerading everything on that interface? Presumably there is an overhead associated with this.

    Cheers,

    Rob

  4. Hi Rob,

    Glad my article has helped.

    It’s been a while since I wrote the script, but I think it was just put in as an extra safety measure to ensure everything on the interface comes from the correct IP. I don’t know of any reason why it couldn’t match the UID when it does the postrouting rule, but I haven’t tested it. Performance wise, I haven’t noticed any issues masquerading everything, but it might be a worthwhile optimisation to put in.

    Thanks

    Simon

  5. I am trying to do the same thing. I don not know what values to enter for the NETIF and VPNIF in the scripts.
    I am trying to get one application running as user Transmission to use a VPN tunnel (tun0) and all other applications running as user User to use eth) without using the tunnel.
    Any help would be appreciated.

  6. Hi Steve,

    NETIF should be the name of the primary network interface on your machine ( usually eth0, br0, wlan0 or something similar ).

    VPNIF should be the interface for the vpn tunnel ( sounds like tun0 in your case ).

    Hope this helps

    Simon

  7. Hi, thank you for this. :)

    I put the (slightly modified) scripts in my OpenVPN config, so it runs when that starts (I added the second script on the last line on the first script, since ovpn don’t allow multiple scripts).
    When I ping something from my vpnuser when VPN is connected, I get responce. When I disconnect VPN I get 100% packet loss, but all other users work.

    I’m wondering if the other users are supposed to get a non-VPN internet? Or should they just not be restricted when the VPN is down. Because as long as the VPN is up they also get routed through the VPN.

    If that is the case, could you please take a look on my commented files below and see if you spot anything. Thank you!

    Here are my scripts and OpenVPN conf file:
    Script 1: http://pastebin.com/VCxM4N2B
    Script 2: http://pastebin.com/hgt9SLwG
    OpenVPN Conf: http://pastebin.com/qqw7w1S9

  8. Hello Simon!

    Thank you very much for your script. I’m also trying to route a user running transmission only via the tun0 interface, but it’s not working properly.

    I’m on Ubuntu, and first I run:
    sudo service openvpn start
    to create the tun0 interface. Now traffic from all users are routed through tun0.

    Then I run your two scripts:
    sudo ./script1.sh
    sudo ./script1.sh
    But they have no effect. All users are still using tun0.

    What do you think could be wrong? I’ve added the vpnuser to rt_tables. The initial parameters of the first script is defined as follows:

    export INTERFACE="tun0"
    export VPNUSER="vpnuser"
    export LANIP="192.168.19.0/24"
    export NETIF="eth0"

    And the second:

    VPNIF="tun0"
    VPNUSER="vpnuser"

  9. Hi Jon,

    From your openvpn config it looks like you need to add ‘route-nopull’ to your openvpn config, that should stop the non-vpn users getting routed via the VPN.

    You may also need to add these lines into /etc/sysctl.d/9999-vpn.conf to ensure the kernel lets the traffic get routed correctly:

    net.ipv4.conf.all.rp_filter = 0
    net.ipv4.conf.default.rp_filter = 0
    net.ipv4.conf.eth0.rp_filter = 0
    

    And then run:

    sysctl -p

    I’ve also just updated the scripts with a few small changes, so you may want to try the latest script and see if that makes any difference.

    Hope this helps

    Simon

  10. Hi miceagol,

    As with my message to Jon above, you may need to add ‘route-nopull’ to your openvpn config – by default openvpn will pull the remote routes, which usually force all traffic over the vpn.

    Hope this helps

    Simon

  11. I have been looking for a solution for this for a long time, this is the best one I have come across yet – it just worked, so thanks!! I have forwarded rtorrent port through and everything is working well, the only issue is that rutorrent (the rtorrent web interface) says the port is closed, when it is blatantly open and working. I assume that this is because rutorrent is working under the web -server user which is a different user than the rtorrent process. Not really sure how to fix this as dont want to forward my entire web-server through VPN.

  12. In relation to the above, I resolved this issue simply by adding a route. The rutorrent check_port tool uses canyouseeme.org (which has an ip address of 107-20.89.142). I therefore issued at CLI, ip route -p add 107.20.89.142 tun0 and now interface can connect to site, so port is shown as open. Thanks again for the solution provided on this page.

  13. Thank you for this post. I finally had some success in selective routing on my gentoo machine. I have a few follow up questions though:
    # block everything incoming on $INTERFACE
    iptables -A INPUT -i $INTERFACE -j REJECT
    Why are we blocking every input on tun0 (except for those accepted of course) ? Will there ever be traffic sent to that interface that one did not ask for using port forwarding. As i understand openvpn connections are not supposed to initiate connections?

    Regards

  14. Simon, thanks so much for these scripts. I have them working nicely for SABnzbd. It’s great because, if the VPN connection drops, SABnzbd just stops. Nice! I have a question, though. How can I modify the scripts to handle more than one user? For example, what if I want to cover the sabnzbd and transmission users? Is it as simple as duplicating all lines that contain $VPNUSER, changing them to $VPNUSER1 and $VPNUSER2, and doing the same for the routing tables?

    Also, is it necessary to only allow certain ports? I guess my question is similar to maynardj’s. If ALL traffic for the given users if forced through the VPN, then does it matter which ports are used? Just curious.

  15. Hi maynardj,

    The reason for rejecting all incoming packets except those specifically allowed is just a safety thing to prevent accidentally exposing of ports – this guide was originally written while I was using PPTP, and the VPN provider I was using at the time forwarded all ports by default, exposing locally running services over the VPN unless they are firewalled off.

    Thanks

    Simon

  16. Hi dildano,

    Great, glad it helps.

    Yes, duplicating the rules should work ( you will probably just need to duplicate the iptables rules though, the route command shouldnt need to change, as long as you just want to send the data over one connection ), although a better solution might be to use the ‘gid-owner’ iptables parameter instead of ‘uid-owner’ to match a group, and add the vpn users to a single group.

    As with my response to maynardj, the reason for only allowing specific ports is a safety thing to ensure only traffic you want to specifically allow to connect is allowed.

    Hope this helps

    Simon

  17. Thank you very much for this post Simon! It was quite helpful.

    I just wanted to share what solution I came up with based on what you provided plus other research I did. I came up with somewhat of a hybrid based on what you provided. If the program you want to route through vpn supports binding to a specific IP address there is a way to accomplish the same thing (without having to deal with marking packets). This is beneficial as you may have noticed that using the method in the original post, due to the time at which the marking is applied/processed, any iptables OUTPUT rules will still think the packet is going out over the eth0 interface instead of tun0 (at least in my case, perhaps other distros are different). So if you tried to do something like -A OUTPUT -o eth0 -m owner –uid-owner vpnuser -j DROP in iptables with the marking system (method in original post) iptables will actually drop the packets even though they are going out tun0, as it thinks they are going out eth0 still (at the time iptables OUTPUT rules are applied). See example at very end of post if you want more info on this (by logging dropped packets).

    You still use most of the stuff from the original post. EX: You will probably want most of the same iptables rules (other than not needing any of the mangle rules to mark packets). You will also still need to populate the vpnuser routing table using whatever method you want, in my case i use the “route-up” option in the openvpn config file to launch a .sh file plus “route-noexec” so it doesn’t fill the default routing table with the openvpn route. The handy thing about route-up is that it will pass a bunch of environment variables with pretty much all the info/ip addresses you need to create a proper routing table etc. All the variables are explained in the openvpn documentation, but if your are lazy like me, just pipe “env” to your system logging utility (or just redirect it into a file using > ) and then you can see all the variables, and more importantly their values in your system log file. Sample env.sh file to launch with “route-up” that will write all variables to system log (assuming your distro uses logger). Don’t forget to chmod any .sh files you create for this to be executable, otherwise it obviously wont work. Honestly its probably cleaner to just dump it into a file, but I’m lazy and was already digging around in the system log for other info so this is what I did. Feel free to use > instead.
    #!/bin/sh
    env | logger

    Just call env.sh using the route-up command in the openvpn config file and it should write everything to the system log for your perusal. Then you can use the info from that to create proper up.sh/down.sh files that will be called with openvpn’s “route-up” and “down” commands.

    As I’m using this to run rtorrent through my vpn tunnel (tun0) i have rtorrent running as it’s own user for the sake of this argument i’ll call it “vpnuser” as you have.

    I created a VLAN, ex: eth0.4 (or eth0.{whatever # you want}) if you don’t have any VLANs, then eth0.1 is fine.
    I then assigned myself a static ip on that vlan. The following files may be stored in a different location depending on your distribution, so be sure to check where the proper place for these is, I am running CentOS, so this is where they happen to be for me. If you’re not sure just try “ls /etc/sysconfig/network-scripts/” and see if there are already other files located there like “ifcfg-eth0″ and others. If there aren’t, you probably need to find the appropriate directory for your distro.

    sample files to make VLAN eth0.4 with a static ip of 10.0.10.2
    Note: in this case i used a netmask of 255.255.255.240, you may want to use a different netmask depending on your situation, but I’m pretty much just using 1 ip on this vlan so that netmask works fine in this situation
    “/etc/sysconfig/network-scripts/ifcfg-eth0.4″
    DEVICE=eth0.4
    BOOTPROTO=static
    ONBOOT=yes
    TYPE=Ethernet
    IPV6INIT=no
    IPADDR=10.0.10.2
    NETMASK=255.255.255.240
    NETWORK=10.0.10.0
    VLAN=yes

    automatically add a rule for eth0.4 VLAN to use “vpnuser” table for routing. Instead of having to call “ip rule add from 10.0.10.2 table vpnuser” from some script all the time, this will simply apply whenever the eth0.4 VLAN is active.
    “/etc/sysconfig/network-scripts/rule-eth0.4″
    from 10.0.10.2 table vpnuser

    Then in my rtorrent config I set it up to always bind to 10.0.10.2 (ip of VLAN eth0.4) so I can be sure it will using the proper “vpnuser” routing table based on what we did in the 2 files above. obviously every application will have a different way to tell it what ip to bind to so i wont include a sample config for this.

    You may find the following command useful for testing. wget –bind-address=10.0.10.2 -qO- http://ipecho.net/plain
    That uses wget bound to the 10.0.10.2 ip so you can test the routing from that ip, and the rest of the command, assuming it can actually reach the internet, will display the public ip address used to get there (just echoes the data from the ipecho.net/plain website which is simply the ip that accessed that site). If you are seeing your VPN’s public ip address then you’re good to go. If you still see you regular non-vpn public ip address then your “vpnuser” routing table isn’t set up right.

    You still should run whatever programs you want going through the vpn as the “vpnuser” user, and then use an iptables rule to drop all packets from that user going out over eth0. You will have to make sure the programs running as vpnuser are bound to the eth0.4 VLAN or all the packets will simply be dropped as doing it this way doesn’t make all packets from vpnuser go through tun0, just the ones coming from the 10.0.10.2 ip, but this way you can ensure any programs you were running as that user never go out over eth0.

    That should cover most of what you need to know, but I’ll happily clarify anything. No promises how often ill check back here though, i was hoping this post could mainly just be used as a reference.

    side bar on logging specific dropped packets with iptables: instead of dropping packets by forwarding them to the DROP chain, you can forward packets to some other chain you made in iptables for logging instead (that logs the packets and then drops them). You can accomplish this by creating for example a chain named “LOGGING” and then adding the following rules (the limit function below was used to stop it from completely filling the log up if there are tons of packets being dropped, feel free to adjust as necessary):
    -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Rule-Dropped: " --log-tcp-options
    -A LOGGING -j DROP

    Then, using the previous example above just change it to -A OUTPUT -o eth0 -m owner –uid-owner vpnuser -j LOGGING. now all packets from user “vpnuser” going out over the eth0 interface will be forwarded to the LOGGING chain where they are dropped and logged to /var/log/messages with the “IPTables-Rule-Dropped:” in the line (so you can easily search for it in your text editor of choice). This will contain info about the packet. I have included a sample line from the log below (local network address in log is 10.0.0.30):
    Mar 22 04:50:47 MACHINE-NAME kernel: IPTables-Rule-Dropped: IN= OUT=eth0 SRC=10.0.0.30 DST=146.255.36.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=35186 DF PROTO=TCP SPT=47708 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 OPT (020405B40402080A009BE7930000000001030306) MARK=0x1 It will look similar to this and as you can see the packet is marked, but the at the time the iptables output rule is applied it doesn’t know yet that this will actually be routed over tun0, it still thinks it’s going out over eth0 if you are using marking like OP, to decide which routing table to use. if you use “ip rule from table vpnuser” then the iptables rule will know where the packet is actually going and apply correctly (not drop packets that will actually be going out over tun0).

  18. I wasn’t able to get ‘gid-owner’ working, and I’m too much of a Linux novice to figure it out. However, duplicating the iptables rules for the different users appears to be working fine. I’m already running the route script using the ‘up’ command in the OpenVPN config. Is there any reason that the iptables and route scripts couldn’t be combined? Or is there a better way to invoke the iptables script?

    Thanks again for your work on this. It’s really quite useful.

  19. Hey dildano, I’m not sure what you tried in order to get gid-owner working, but probably the issue you had when attempting to use gid-owner, is that it is based on the gid of the running process, and not necessarily the gid of a group that user that ran the process is a member of. Typically the gid of a running process will simply be the same number as the uid of the user who ran the process. I.e if user “vpnuser” has a uid of 501, then the default gid of all processes run by “vpnuser” will also be 501, even if “vpnuser” is a member of other specific groups you may have added him to. If however you for example use the chgrp command on the executable transmission file (probably located at /usr/bin/transmission or somewhere similar) to change the group of the transmission file to a group of your choosing, then when transmission is run it should have the uid of the user who ran it, but the gid of the group you changed the file to. you will probably find “ps -eo uid,gid,args” usefull for figuring out whats going on. This will give you a list of all the running processes along with the uid and gid for each process so you can confirm your changes had the desired effect relating to the change in group for the executable file. Once the process is running with your desired gid then gid-owner will probably work as you are expecting it to. More than likely you just added a user to a group, but that doesn’t give processes run by that user a gid; the gid for a process is based on the gid of the actual executable process file, not the gid of a group the user who ran it is in. Hope this helps :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>