As part of an interview for a Technical Product Manager job I had to create a video presentation how I would improve a product at home. I have chosen my home broadband and argued for enabling monitoring and mobile backup as an improvement. While I am monitoring my home broadband for quite some time I have always played with the idea to use my mobile data plan in case my fixed broadband is down.
I recently moved default gateway function to my monitoring Raspberry Pi with the help of a VLAN enabled switch - see my previous post about it. The mobile coverage in the electric room where my broadband modem installed is probably the weakest in the building so not an ideal place for a mobile backup solution. On the other hand upstairs at my working desk the mobile coverage is reasonable and I have my other Raspberry Pi that I use as my desktop PC usually. I also regularly charge my phone next to this Raspberry Pi which sparked the idea to extend USB tethering as a mobile backup solution for the entire home. For this I need to change the routing at both Raspberry Pi in case the fixed broadband is down so that all traffic is routed via USB tethering. For this I have found a failover script here that I have customised for my use case as follows on the Raspberry Pi next to my broadband modem:
jordana@pi4b2:~ $ nano failover.sh
#!/bin/bash
# Set defaults if not provided by environment
CHECK_DELAY=${CHECK_DELAY:-1}
CHECK_IP=${CHECK_IP:-8.8.8.8}
PRIMARY_IF=${PRIMARY_IF:-eth0.8}
PRIMARY_GW=${PRIMARY_GW:-192.168.0.1}
BACKUP_IF=${BACKUP_IF:-eth0}
BACKUP_GW=${BACKUP_GW:-192.168.0.12}
# Compare arg with current default gateway interface for route to healthcheck IP
gateway_if() {
[[ "$1" = "$(ip route get "$CHECK_IP" | sed -rn 's/^.*dev ([^ ]*).*$/\1/p')" ]]
}
# Cycle healthcheck continuously with specified delay
while sleep "$CHECK_DELAY"
do
# If healthcheck succeeds from primary interface
if ping -I "$PRIMARY_IF" -c1 "$CHECK_IP" &>/dev/null
then
# Are we using the backup?
if gateway_if "$BACKUP_IF"
then # Switch to primary
ip route delete default via "$BACKUP_GW" dev "$BACKUP_IF"
fi
# If healthcheck fails from primary interface
else
# Are we using the primary?
if gateway_if "$PRIMARY_IF"
then # Switch to backup
ip route add default via "$BACKUP_GW" dev "$BACKUP_IF"
fi
fi
done
On my desk I also have a failover script running on the Raspberry Pi:
jordana@dad-pi4g:~ $ nano failover.sh
#!/bin/bash
# Set defaults if not provided by environment
CHECK_DELAY=${CHECK_DELAY:-1}
CHECK_IP=${CHECK_IP:-8.8.8.8}
PRIMARY_IF=${PRIMARY_IF:-eth0}
PRIMARY_GW=${PRIMARY_GW:-192.168.0.1}
BACKUP_IF=${BACKUP_IF:-usb0}
BACKUP_GW=${BACKUP_GW:-192.168.0.1}
EMAIL=${EMAIL:-jordan.arpad@gmail.com}
# Compare arg with current default gateway interface for route to healthcheck IP
gateway_if() {
[[ "$1" = "$(ip route get "$CHECK_IP" | sed -rn 's/^.*dev ([^ ]*).*$/\1/p')" ]]
}
switch_to_primary() {
ip route delete default via "$PRIMARY_GW" dev "$PRIMARY_IF"
ip route add default via "$PRIMARY_GW" dev "$PRIMARY_IF"
date | mail -s "$HOSTNAME switched to primary" "$EMAIL"
}
switch_to_backup() {
ip route delete default via "$PRIMARY_GW" dev "$PRIMARY_IF"
ip route add default via "$PRIMARY_GW" dev "$PRIMARY_IF" metric 5000
date | mail -s "$HOSTNAME switched to backup" "$EMAIL"
}
# Cycle healthcheck continuously with specified delay
while sleep "$CHECK_DELAY"
do
# If healthcheck succeeds from primary interface
if ping -I "$PRIMARY_IF" -c1 "$CHECK_IP" &>/dev/null
then
# Are we using the backup?
if gateway_if "$BACKUP_IF"
then switch_to_primary
fi
# If healthcheck fails from primary interface
else
sleep "$CHECK_DELAY"
# If healthcheck succeeds from primary interface
if ping -I "$PRIMARY_IF" -c1 "$CHECK_IP" &>/dev/null
then
# Are we using the backup?
if gateway_if "$BACKUP_IF"
then switch_to_primary
fi
# If healthcheck fails from primary interface AGAIN
else
# Are we using the primary?
if gateway_if "$PRIMARY_IF"
then switch_to_backup
fi
fi
fi
done
I added routing and NAT to usb0 on the desktop Raspberry Pi also and whenever I have my phone connected with USB tethering to the mobile network the redundancy switching is working like a charm back and forth :) So I made these scripts running @reboot from crontab and configured notification emails from my desktop Raspberry.
UPDATE
I added email notification to the script as per above after installing ssmtp and mailutils and configured as follows. Please note that this only works with App Password added to your Google account so use it with care, at your own risk and consider creating a separate Google account for this purpose.
jordana@dad-pi4g:~ $ sudo cat /etc/ssmtp/ssmtp.conf
#
# Config file for sSMTP sendmail
#
# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.
root=
# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
mailhub=smtp.gmail.com:587
# Where will the mail seem to come from?
rewriteDomain=
# The full hostname
hostname=dad-pi4g
# Are users allowed to set their own From: address?
# YES - Allow the user to specify their own From: address
# NO - Use the system generated From: address
#FromLineOverride=YES
AuthUser=jordan.arpad73@gmail.com
AuthPass=
UseSTARTTLS=YES
