Why Dual Homed Servers Suck

So a friend asked me the other day why some of his clients could get to his dual-homed server (a server with network cards in two different subnets) and some could not. This naturally was followed by the "why the heck are you dual-homing the server" question. It turns out his predecessor was trying to improve the speed of the server by allowing it to have a finger in another network, avoiding a hop. Needless to say, this really didn't solve his problem as the speed of access to the server was never inhibited by the multigigabit network he had nor the gigabit network card in the server.

There are very few reasons to multihome a server or client, but there are right and wrong ways to do it.

1. You are using the device as a router (or actually is a router or firewall)
2. You need to sniff traffic in another network. In which case, the second NIC probably isn't being used for traffic, but as a passive participant without an IP address.
   
3. Your physical server is a Virtual Machine Host and needs physical legs in multiple networks. In this case, each Virtual Machine (VM) probably has access to only one of the NICs anyway.
4. Your server has direct access to a Storage Area Network. In this scenario, no one is addressing to the SAN facing NIC, so it works just fine.
   

The reason why most dual-homed fail is because of improperly routing configurations in the server. A device can only handle 1 default route. Look at the following configuration of a dual-homed setup (assume that the same router is handling both the 10.1.1.0 and 10.2.2.0 networks):

Ethernet adapter Local Area Connection 1:
IP Address: 10.1.1.2
Subnet Mask: 255.255.255.0
Default Gateway: 10.1.1.1

Ethernet adapter Local Area Connection 2:
IP Address: 10.2.2.2
Subnet Mask: 255.255.255.0
Default Gateway: 10.2.2.2

We may have entered two default gateways in the configuration pages, but look what happens in the routing table on the machine:

Active Routes:
Network Destination         Netmask     Gateway     Interface
           0.0.0.0         0.0.0.0    10.1.1.1     10.1.1.2
          10.1.1.0   255.255.255.0     On-Link     10.1.1.2
          10.2.2.0   255.255.255.0     On-Link     10.2.2.2

As you see, any traffic that doesn't match the 10.1.1.0 or 10.2.2.0 network will leave from the 10.1.1.2 interface.

Lets pretend for second that you have DNS setup to point server.mydomain to both 10.1.1.2 and 10.2.2.2. If DNS Round Robin is setup correctly (which is probably is) half of your hosts will go to 10.1.1.2 and the other half will go to 10.2.2.2.
If your hosts are on the 10.1.1.0 network and happen to get pointed to 10.2.2.2, guess which server IP is going to respond: 10.1.1.2. This is kind of cool, but the packet will be dropped because it is coming from a different IP than what the client asked for. Bad news. If you have a request coming from 10.2.2.0 and it's going to 10.1.1.2, then the 10.2.2.2 NIC responds and the packet gets dropped.

To complicate it, lets pretend you have a client on the 10.3.3.0 network and they perform a request to the 10.2.2.2 NIC. Which IP responds? Yep, the 10.1.1.1 NIC, because it has the default gateway defined on it.

So, how do you fix this? Easy! Prevent your 10.2.2.0 network from trying to contact the 10.1.1.2 IP and prevent your 10.1.1.0 network from trying to contact the 10.2.2.2 IP. No, you can't do this by blocking the packets at the firewall/router. You actually have to prevent those clients from ever wanting to contact the wrong interfaces.

There are many ways of doing this, like by setting up DNS to not point the same name to both NICs, but most likely, you'll find it's just easier to get rid of the dual-homed setup, and keep to one IP address on the server.

This is not to say you can't get your server running at multigigabit speeds or in a dual-homed, it's just that there are usually better ways. Like if your network drivers support LACP and your switch supports LACP, you can combine multiple NICs together to form one logical interface that is X times faster.

Now, I've heard people say that they like to put one card of a Mail or Web server in their DMZ and one in their private network. This is a bad idea. If your server gets compromised in this scenario, the attacker now has a direct line into your private, "protected", network. This completely defeats the purpose of having a firewall.

If you want your server in your DMZ, put it in your DMZ. Your inside clients will still have access to your server, but it won't have access to your inside clients. This is a cool trick that firewalls can do and is why so many of us spend thousands of dollars on said firewalls. Even in a scenario where your inside clients have full access to the mail server, the firewall can still prevent some attacks from the inside network.

If you can't figure out how to put your public facing server in your DMZ correctly, please consult a professional who understands your model of firewall before you do something silly like dual-home your mail server.