|Linux System Administration|
Two default routes
Linux has very advanced routing, filtering and traffic shaping options. Here is how to configure a system with two default routes.
To activate Linux advanced routing on a Debian GNU/Linux system, install the iproute package:
~# apt-get install iproute
This will create the /etc/iproute2/ directory. It also installs some new executables, including ip.
2. The ip command
From a command line on any Linux system, you can see the existing routing table by simply typing route at the prompt (or /sbin/route if /sbin is not in your path). Your routing table will be similar to this:
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.1.0 * 255.255.255.0 U 0 0 0 eth1 default my.host.com 0.0.0.0 UG 0 0 0 eth1
Advanced routing commands are issued as arguments to the ip command. To see the routing table with iproute2, you can use the long version ip route show table main or the shortcut version ip route like this:
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.20 default via 192.168.1.10 dev eth1
3. (Hot) Potato routing
Typically, a host connected to a network, such as the Internet, will have one default route and one Internet (WAN) interface. However, if a second connection to the Internet is added and both are accessed, you can end up with a situation referred to as (hot) potato routing, or deflection routing.
Normally, when a packet, such as an ICMP ping, arrives at the primary interface, it is examined by the host, after which a reply packet is generated, the routing table is consulted and the packet it sent back via the default route. On the other hand, if a ping packet arrives at the secondary WAN interface, the same thing happens: it is examined by the host, a reply packet is generated, the routing table is consulted and the packet it sent back via the default route. In other words: a packet is received on one interface and the reply is sent back via the other.
In theory such packets can be routed, but are dropped by ISPs. That's because these packets have a source address that is not part of the network that they are being routed from -- they look like they've been forged. Indeed, such forged packet headers are often used in DOS attacks.
However, if you have more than one connection to the Internet and you want to use them all despite the fact that you have only one default route, what can you do? The answer is advanced routing.
4. Configuring two default routes
With advanced routing, you can have as many routing tables as you want. In the example below, we add just one for an extra DSL line from an ISP called "cheapskate."
First add a name for the new routing table to the /etc/iproute2/rt_tables file. This can be appended to it with the command echo 2 cheapskate >> /etc/iproute2/rt_tables. The result looks like this:
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 2 cheapskate
Above I mentioned that the command ip route is actually a shortcut for the longer command ip route show table main. Since there is no shortcut to list the new routing table, you have no choice but to use the long form: ip route show table cheapskate. Entering this command now will reveal that this new table is still empty.
All that is necessary is to add the new default route to the cheapskate table -- the old main table will continue to handle the rest. The reason for this will soon become clear. Here is the existing main table:
~# ip route show table main 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.10 192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.10 default via 192.168.1.1 dev eth0 ~# _
As follows, add the new default route to table cheapskate and then display it:
~# ip route add default via 192.168.2.1 dev eth1 table cheapskate ~# ip route show table cheapskate default via 192.168.2.1 dev eth1 ~# _
As you can see, the entire table consists of a single line. However, it is not yet being used. To implement it, the command, ip rule is required. Routing tables determine packet destinations, but now we need the kernel to use different routing tables depending on their source addresses. The existing set of ip rules is very simple:
~# ip rule 0: from all lookup local 32766: from all lookup main 32767: from all lookup default ~# _
At this point you need to add a new rule:
~# ip rule add from 192.168.2.10 lookup cheapskate prio 1000
This command adds a rule for when a packet has a from pattern of 192.168.2.10 in which case the routing table cheapskate should be used with a priority level of 1000. In this example the pattern only needs to match one address, but you can set patterns in a Linux router to match different sets of addresses.
~# ip rule 0: from all lookup local 1000: from 192.168.2.10 lookup cheapskate 32766: from all lookup main 32767: from all lookup default ~# _
The kernel searches the list of ip rules starting with the lowest priority number, processing each routing table until the packet has been routed successfully.
The default ruleset always has a local table with a match pattern of all. The local table (priority 0) handles traffic that is supposed to stay on the localhost, as well as broadcast traffic.
After the local rule comes our new rule with a priority of 1000. This priority number is arbitrary, but makes it easy to add other rules before and after it later on.
Our new rule comes before the main table, which is the one that is modified by the old route command. The last rule is for the default table. I'm not certain what it's for, as I've always found it to be empty, and seeing as there is a default route in the table main, no traffic ever gets to the table default.** Warning **
When working with more than one routing table, never forget to add the table part of the command. If you do forget, rule changes in the wrong table (main) can seem awfully mysterious. When learning the ropes and working remotely, you will probably lock yourself out a few times this way: the changes happen very quickly, so it may be wise to use a console instead.
Another important point to remember is that routes are cached. In other words, if you update a routing table and nothing seems to happen, it's because the table is still in memory. The solution is simply to flush the cache with ip route flush table cache. In this manner it is possible to first make a number of changes and then flush the cache so that all of the changes will be implemented simultaneously. This is actually convenient when working on an active router.
5. Example configuration
When a secondary WAN interface became available at a client site, I wanted to allow their remote users to use this connection as an alternative in case their primary Internet connection ever went down. With Linux, we now know that this is possible.
First, some background information. The client's router had the following interfaces:
- eth0 Primary Internet connection (Versatel). inet addr: 18.104.22.168 Bcast: 22.214.171.124 Mask: 255.255.255.248 - eth0:0 Private net behind the Versatel ADSL modem/router. inet addr: 192.168.1.1 Bcast: 192.168.1.255 Mask: 255.255.255.0 - eth0:1 Private net behind the Zonnet ADSL modem/router. inet addr: 10.0.0.100 Bcast: 10.255.255.255 Mask: 255.0.0.0 - eth1 Private net -- main internal network segment. inet addr: 192.168.13.1 Bcast: 192.168.13.255 Mask: 255.255.255.0 - eth2 Private net -- wireless internal network segment. inet addr: 192.168.14.1 Bcast: 192.168.14.255 Mask: 255.255.255.0 - lo Loopback interface. inet addr: 127.0.0.1 Mask: 255.0.0.0 - ppp0 Secondary Internet connection (Zonnet). inet addr: 126.96.36.199 P-t-P: 188.8.131.52 Mask: 255.255.255.255
The main routing table displayed with route -n:
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 184.108.40.206 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0 220.127.116.11 0.0.0.0 255.255.255.248 U 0 0 0 eth0 18.104.22.168 22.214.171.124 255.255.255.128 UG 0 0 0 ppp0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.14.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 192.168.13.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 126.96.36.199 188.8.131.52 255.255.248.0 UG 0 0 0 ppp0 10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0 0.0.0.0 184.108.40.206 0.0.0.0 UG 0 0 0 eth0
The route for 220.127.116.11/21 via ppp0 may be unnecessary, but I figured it would be 'cheaper' because the IP address for ppp0 is part of the same network. The route to 18.104.22.168/25 via the ppp0, on the other hand, is a network segment that includes an SMTP relay that is not be available via any other route.
The list of interfaces displayed with ip link list:
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:00:24:c1:10:5c brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:00:24:c1:10:5d brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:00:24:c1:10:5e brd ff:ff:ff:ff:ff:ff 5: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0 6: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1500 qdisc pfifo_fast qlen 3 link/ppp
NB: sit0 is an IPv6-IPv4 tunnel.
The main routing table displayed with ip route show table main:
22.214.171.124 dev ppp0 proto kernel scope link src 126.96.36.199 188.8.131.52/29 dev eth0 proto kernel scope link src 184.108.40.206 220.127.116.11/25 via 18.104.22.168 dev ppp0 scope link 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1 192.168.14.0/24 dev eth2 proto kernel scope link src 192.168.14.1 192.168.13.0/24 dev eth1 proto kernel scope link src 192.168.13.1 22.214.171.124/21 via 126.96.36.199 dev ppp0 scope link 10.0.0.0/8 dev eth0 proto kernel scope link src 10.0.0.100 default via 188.8.131.52 dev eth0
The idea was to create a second routing table for the second Internet connection (ppp0) with its own default route. This can be done in only three steps. First, after installing the necessary software (see above), I created a second routing table (after the existing main routing table) called 'zonnet':
~# echo 2 zonnet >> /etc/iproute2/rt_tables
Second, I added a default route to the zonnet routing table using the ppp0 interface and its IP address:
~# ip route add default via 184.108.40.206 dev ppp0 table zonnet
Third, I added a new rule to the kernel that tell it to use the new routing table when packets (connections) originate from the second interface:
~# ip rule add from 220.127.116.11 lookup zonnet prio 1000
Thus, the new zonnet routing table looks like this (just one line):
~# ip route show table zonnet default via 18.104.22.168 dev ppp0 ~# _
And the routing rule looks like this:
~# ip rule 0: from all lookup local 1000: from 22.214.171.124 lookup zonnet 32766: from all lookup main 32767: from all lookup default ~# _
So far, the result of all this is that all requests destined for the firewall coming in from eth0 are sent back out eth0 (the main default gateway; 126.96.36.199), while requests destined for the firewall coming in from ppp0 are sent back out ppp0 (the secondary default gateway; 188.8.131.52). However, if the server responds to any requests that are forwarded to it, those responses will still be routed out the main default gateway regardless.
The first step towards a solution was to define a second network, 192.168.15.0/24, on the UTP segment that the server is attached to. Luckily, Windows server 2003 allows you to bind additional IP addresses to its interfaces. In this case, only the server and the firewall (via eth1) have addresses on this network.
- eth1:0 Private net -- additional internal network segment. inet addr: 192.168.15.1 Bcast: 192.168.13.255 Mask: 255.255.255.0
On this network, the server is defined as 192.168.15.2 and the firewall is configured to forward all requests for it that arrive via ppp0 on to this address. Naturally, the responses come out this way too.
Second, since all of the packets moving from 192.168/.15.0/24 into the firewall are responses to requests that arrive via the secondary Internet connection (and should be sent back that way) anyway, I could use this one routing rule:
~# ip rule add from 192.168.15.0/24 lookup secnet prio 990
The routing rule now looks like this:
~# ip rule 0: from all lookup local 990: from 192.168.15.0/24 lookup zonnet 1000: from 184.108.40.206 lookup zonnet 32766: from all lookup main 32767: from all lookup default ~# _
Now if a request is sent in via ppp0 and forwarded on to the server (via 192.168.15.0/24), its response will also be sent back via ppp0.