Note: I don’t use OpenWRT’s DNS or DHCP capabilities anymore.

Why?

I have two routers. One running as a VM under Proxmox which has a Mellanox SFP28 card attached to it and it is my main connection to the Internet, and another running on an old TP-Link Archer C7, which has a 4G modem attached to it and takes over in case my primary connection goes down, using keepalived (read my past post on the subject here).

This works well enough, but there is a small snag. I have to duplicate all DNS assignments & static DHCP leases between the routers. This is normally not a problem, it takes two minutes, but there is a potential for configuration drift. How about we configure this in one place (let’s call it the master router) and have it synchronised to one or more other routers (the target router(s))? And also, make sure that any leases are synchronised between the routers?

Configuration Information

Both DNS and DHCP settings exist in one file: /etc/config/dhcp. DHCP leases exist under /tmp/dhcp.leases.

In order to achieve what we want, we need to copy the dhcp configuration from the master to the target router, however the leases file needs a bit more special handling.

Important: The two DHCP servers must have a split-scope. Split-scope means that they utilise different pools to assign IP addresses for non-statically leases devices. In other words, if you have a phone connecting to your network, you probably don’t care if the assigned IP is 10.0.0.123 or 10.0.0.55, as long as there is no conflict. But if you have a server that it must have the specific IP 10.0.0.100, then both DHCP servers can and should have the same static assignment.

The reason we need to do this, is that since we do not have a real way of synchronisation between the two DHCP servers, there is a potential for a race condition, where two devices are assigned the same IP address from the two routers.

Preparation

Key Authentication

If you don’t have SSH key-based authentication from the master to the target router, you need to configure it.

Login to your main router via ssh and type the following commands:

cd /etc/dropbear
dropbearkey -t ed25519 -f ~/.ssh/id_dropbear

This will create a private/public key pair. The filename.pub contains the public key. You can either use to use ssh-copy-id to copy the public key to the target router, or do it manually. I prefer the manual approach, since I’ve disabled password authentication from all machines (apart from my workstation account). To do it manually, ssh to your target router and type:

cd /etc/dropbear
echo "ssh-ed25519 keyvalue root@router" >> authorized_keys

Keep in mind that the above value is an example, you need to replace it with the proper key, user and host name.

Bash

I chose to install bash in order to work with arrays used in the next script. Bash can be a bit too big to fit some consumer devices running OpenWRT so keep that in mind.

Synchronise DHCP Settings

Warning!

There is an important caveat you must take into account before executing any command shown here. In my case, the dhcp configuration between the two files is almost the same, but I need to provision for two cases:

1. The master router has an extra interface (an OpenVPN one, which will be torn down soon).
2. We need to configure a split scope.

Thankfully, OpenWRT ignores invalid interfaces in the “Devices and Ports” section, so I don’t need to do anything here. We will handle the split scope in the script using sed, but keep in mind that if you need to have additional configuration differences between the two routers, you need to handle it on a case per-case-basis. It’s obviously much easier to have as similar configuration on both routers as possible.

You (probably) can have both routers running as authoritative, but this might cause your devices to swap IP addresses as their leases expire. If you want to avoid this, you can use sed to make the target router non-authoritative.

Now that we’ve gotten the what out of the way, let’s move on to how.

Create a syncing script

I chose to put this under /root; you can choose whichever location you prefer. Create a file name dhcp-sync.sh and add the following content to it:

#!/bin/bash

target_router=ip_or_hostname_of_target_router
dhcp_starts=(1536 512)
dhcp_limits=(150 150)
dhcp_target_starts=(1686 662)
dhcp_target_limits=(100 100)

cp /etc/config/dhcp /tmp/dhcp_temp

for i in ${!dhcp_starts[@]}; do
        sed -i "s/option start '${dhcp_starts[$i]}'/option start '${dhcp_target_starts[$i]}'/" /tmp/dhcp_temp
        sed -i "s/option limit '${dhcp_limits[$i]}'/option limit '${dhcp_target_limits[$i]}'/" /tmp/dhcp_temp
done


scp /tmp/dhcp_temp root@$target_router:/etc/config/dhcp
mv /tmp_dhcp_temp /etc/config/dhcp

rm /tmp/dhcp_temp

ssh root@$target_router -t '/etc/init.d/dnsmasq restart'
logger "DHCP Settings Synced"

In OpenWRT, you define the start and the limit for DHCP. In other words, you define how many addresses to skip from the first possible address in the network (start) and how many addresses will be available for DHCP (limit).

I have 4 separate networks at home, and one of them uses a slightly different pool (one of them is a /21; the rest are /22). While most of my networks use 10.X.0.2.0/22 for DHCP addresses, that one uses 10.Y.0.6.0/21. As such, while most networks have a “start” of 512 addresses, that one has a “start” of 1536.

I’ve defined the following arrays: dhcp_starts, dhcp_limits, dhcp_target_starts, dhcp_target_limits and I am using sed to do a rather hacky find & replace. Sed goes through the dhcp_starts array and uses an indexer to get the elements from the other arrays in a positional manner. It will look for the text option start ‘number‘ and replace it with option start ‘target_number‘ for each column in those arrays. It will do the same for the option limit entries. It goes without saying that all arrays need to have the same number of elements, their columns properly sorted.

As I mentioned, this script uses bash in order to be able to work with arrays, that ash does not support. You can avoid that and still use OpenWRT's ash, but you will have to manually repeat the sed command for each different DHCP pool.

In my example, the master router will have option start at 1536 with a limit of 150 addresses for whatever network matches that, and a start of 512 with a limit of 150 addresses for the rest. The target router will have a start of 1686 (1536+150) and a a limit of 100 for that one network, as well as a start of 662 (512+150) with a limit of 100 for the rest.

This ensures that there are no conflicts and the scope is successfully split. The script then applies the changes to the local and remote configuration file and restarts the dnsmasq service on the target router to apply the changes.

Schedule it

In the /root/crontabs/root file add the following line:

*/30 * * * * /root/dhcp-sync.sh

I chose to sync every 30 minutes. The dnsmasq service will go down for 3-4 seconds, but of course you can choose a different schedule if you want to. Ensure that cron is running by typing

service cron start

You should be able to see a “user.notice root: Leases Synced” message in your System Log.

Synchronise Leases

This is a bit more involved, because we essentially need a 2-way synchronisation, since devices may elect to choose either of the routers to receive a lease. In my configuration, both routers are authoritative and as such they can both assign a DHCP address.

Here’s what a line of a DHCP lease looks like in the /tmp/dhcp.leases file looks like:

1744896252 ac:b4:21:14:f4:a0 10.90.1.100 hostname ff:21:f4:64:80:00:61:d0:01:2d:91:b8:18:bc:24:11:e4:d4:20

The first column is a timestamp, the second is the MAC address, the third is the assigned IP address, the fourth is the hostname for that lease, and the fifth is the DUID (an identifier used for DHCPv6).

The best approach I’ve found to address this is to ignore the first column and take the leases based on the MAC addresses, ignoring any duplicates. This may cause some leases to end earlier (in case of duplicates), but it should not cause any adverse effects. So, create a new file named lease-sync.sh

#!/bin/sh

target_router=ip_or_hostname_of_target_router

scp root@$target_router:/tmp/dhcp.leases /tmp/dhcp_leases_temp
cat /tmp/dhcp.leases >> /tmp/dhcp_leases_temp

awk 'arr[$2]++{next} 1' /tmp/dhcp_leases_temp | sort > /tmp/dhcp_leases_awk
mv /tmp/dhcp_leases_awk /tmp/dhcp.leases
rm /tmp/dhcp_leases_temp

scp /tmp/dhcp.leases root@$target_router:/tmp/dhcp.leases

ssh root@$target_router -t '/etc/init.d/dnsmasq reload'
/etc/init.d/dnsmasq reload

This will retrieve the leases from the second router, combine them with the ones in the master router and filter out the duplicates based on the MAC address (ignoring the timestamp) using awk. We then save the results to the master router’s DHCP leases file as well as the target router’s one.

I have found out through experimentation that you must restart the dnsmasq service for the leases to stick (or at least reload the configuration). I am not 100% sure how dnsmasq works, but based on the behaviour I’ve noticed, it seems to keep the DHCP leases in memory and use the dhcp.leases file as a backup (i.e. in case the service restarts).

Schedule it

In the /root/crontabs/root file add the following line:

*/5 * * * * /root/lease-sync.sh

Since reloading takes a few seconds, I chose to do that every five minutes.

Alternatives

Kea DHCP from ISC is a DHCP server that supports some failover scenarios with (mostly) 2 servers; either load-balancing or active/passive failover. Configuring Kea can be a bit daunting and I would rather not utilise an additional service if I can avoid it.

Windows Server has support for DHCP failover, but that requires setting up a Windows Failover Cluster (and is not free).

Closing thoughts

Seems that DHCP is rather hard to get HA right, since the standard implementations do not really provision for it. There are some caveats with the solution I presented above (the need to restart dnsmasq and a time window of potential non-resolution), but it should work well enough for a home setup.

Feel free to send me any questions or suggestions at blog[at]vtable[dot]org. I had to disable comments here, since the amount of spam was unmanageable.