AnyIP: Bind a whole subnet to your Linux machine

IPv6 Prefix Delegation

In my previous post I wrote how you can use Docker with IPv6 and Prefix Delegation.

A IPv6 subnet routed to a Linux machine can be used with other things than Docker. That’s where the AnyIP feature of the kernel comes in.

Linux Kernel AnyIP

The AnyIP feature of the Linux kernel allows you to bind a complete IPv4 or IPv6 subnet to your system.

Instead of adding all addresses manually to the kernel you can tell it to bind a complete subnet.

Configuring

IPv4

ip -4 route add local 192.168.0.0/24 dev lo

In this case the Linux kernel will now respond to ARP requests for any IPv4 address in the 192.168.0.0/24 subnet.

IPv6

ip -6 route add local 2001:db8:100::/64 dev lo

In this case the kernel will respond for Neigh Sollicitations on any IPv6 address in the 2001:db8:100::/64 subnet.

Example usage

Let’s assume that you have the IPv6 prefix 2001:db8:100::/60 routed to your Linux machine through IPv6 prefix delegation.

From that /60 subnet we take the first /64 subnet and attach it to lo.

ip -6 route add local 2001:db8:100::/64 dev lo

You can now ping any of the addresses in that subnet:

2001:db8:100::1
2001:db8:100::100
2001:db8:100::200
2001:db8:100::dead:b33f

If you would start a webserver which listens on port 80 you can use any of the IPv6 addresses in that subnet and the webserver will respond to it.

Use cases

It could be that you want to to mass-shared hosting on a system where you want to assign each hostname/domainname it’s own IPv6 address. Instead of attaching single IPs to a interface you can simply attach a complete subnet and point traffic to any of the IPs in that subnet.

Demo

On a virtual machine on PCextreme’s Aurora Compute I deployed a Instance with Prefix Delegation enabled.

After running ‘dhclient’ I got the subnet 2a00:f10:500:40::/60 assigned to my Instance.

It was then just one line to attach a /64 subnet:

ip -6 route add local 2a00:f10:500:40::/64 dev lo

Random address generator

I wrote a small piece of Python code to generate a random IPv6 address:

#!/usr/bin/env python3
"""
Generate a random IPv6 address for a specified subnet
"""

from random import seed, getrandbits
from ipaddress import IPv6Network, IPv6Address

subnet = '2a00:f10:500:40::/64'

seed()
network = IPv6Network(subnet)
address = IPv6Address(network.network_address + getrandbits(network.max_prefixlen - network.prefixlen))

print(address)

Using a small loop in Bash I could now ping random addresses in that subnet:

while [ true ]; do ping6 -c 2 `./random-ipv6.py`; done

Some example output:

--- 2a00:f10:500:40:d142:1092:ea84:74b4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 10.252/11.680/13.108/1.428 ms
PING 2a00:f10:500:40:4e50:f264:6ea9:d184(2a00:f10:500:40:4e50:f264:6ea9:d184) 56 data bytes
64 bytes from 2a00:f10:500:40:4e50:f264:6ea9:d184: icmp_seq=1 ttl=56 time=10.0 ms
64 bytes from 2a00:f10:500:40:4e50:f264:6ea9:d184: icmp_seq=2 ttl=56 time=10.0 ms

--- 2a00:f10:500:40:4e50:f264:6ea9:d184 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 10.085/10.087/10.089/0.002 ms
PING 2a00:f10:500:40:d831:1f89:b06d:fe12(2a00:f10:500:40:d831:1f89:b06d:fe12) 56 data bytes
64 bytes from 2a00:f10:500:40:d831:1f89:b06d:fe12: icmp_seq=1 ttl=56 time=9.77 ms
64 bytes from 2a00:f10:500:40:d831:1f89:b06d:fe12: icmp_seq=2 ttl=56 time=10.1 ms

--- 2a00:f10:500:40:d831:1f89:b06d:fe12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1005ms
rtt min/avg/max/mdev = 9.777/9.958/10.140/0.207 ms
PING 2a00:f10:500:40:2c45:26ee:5b93:fa2(2a00:f10:500:40:2c45:26ee:5b93:fa2) 56 data bytes
64 bytes from 2a00:f10:500:40:2c45:26ee:5b93:fa2: icmp_seq=1 ttl=56 time=10.2 ms
64 bytes from 2a00:f10:500:40:2c45:26ee:5b93:fa2: icmp_seq=2 ttl=56 time=10.0 ms

Docker and IPv6 Prefix Delegation

As posted earlier I have IPv6 Prefix Delegation working at our office to test with Docker.

One of the missing links was to automatically configure Docker to use the prefix obtained through DHCPv6+PD. I manually configured the prefix in Docker, but I also had to run dhclient manually.

I figured this could be automated so I gave it a try.

Ubuntu Networking

At first I tried to figure out if Ubuntu’s networking was somehow able to request a prefix through DHCPv6. Long story short: Neither Ubuntu nor CentOS are able to do so. You have to script this manually.

dhclient

To obtain a prefix I had to run dhclient manually. That wasn’t to hard. Simply run:

dhclient -6 -P -d -v eth0

This resulted in obtaining a prefix:

Bound to *:546
Listening on Socket/eth0
Sending on   Socket/eth0
PRC: Confirming active lease (INIT-REBOOT).
XMT: Forming Rebind, 0 ms elapsed.
XMT:  X-- IA_PD d5:68:28:08
XMT:  | X-- Requested renew  +3600
XMT:  | X-- Requested rebind +5400
XMT:  | | X-- IAPREFIX 2001:980:XXXX:140::/60
XMT:  | | | X-- Preferred lifetime +7200
XMT:  | | | X-- Max lifetime +7500
XMT:  V IA_PD appended.
XMT: Rebind on eth0, interval 940ms.
RCV: Reply message on eth0 from fe80::da67:d9ff:fe81:bcec.
RCV:  X-- IA_PD d5:68:28:08
RCV:  | X-- starts 1457617054
RCV:  | X-- t1 - renew  +604800
RCV:  | X-- t2 - rebind +967680
RCV:  | X-- [Options]
RCV:  | | X-- IAPREFIX 2001:980:XXXX:140::/60
RCV:  | | | X-- Preferred lifetime 1209600.
RCV:  | | | X-- Max lifetime 2592000.
RCV:  X-- Server ID: 00:03:00:01:d8:67:d9:81:bc:f0
PRC: Bound to lease 00:03:00:01:d8:67:d9:81:bc:f0.
PRC: Renewal event scheduled in 604800 seconds, to run for 362880 seconds.
PRC: Depreference scheduled in 1209600 seconds.
PRC: Expiration scheduled in 2592000 seconds.

As you can see, I got a /60 prefix. Now I had to somehow get this automated and configure Docker to use it.

Upstart

Since I was testing with Docker 1.10 under Ubuntu 14.04 I had to use Upstart to run dhclient.

The /etc/init/dhclient6-pd.conf Upstart script I created was rather simple:

description     "DHCPv6 Prefix Delegation client"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 30 3
umask 022

console log

exec dhclient -6 -P -d eth0

DHCP hook

dhclient has hooks which it can execute when something happens. I wrote a hook which extracted the delegated IPv6 prefix and restarted Docker.

I placed the hook in the default location for DHCP hooks: /etc/dhcp/dhclient-enter-hooks.d/docker-ipv6:

#!/bin/bash

SUBNET_SIZE=80
DOCKER_ETC_DIR="/etc/docker"
DOCKER_PREFIX_FILE="${DOCKER_ETC_DIR}/ipv6.prefix"

if [ ! -z "$new_ip6_prefix" ]; then
    SUBNET=$(sipcalc -S $SUBNET_SIZE $new_ip6_prefix|grep Network|head -n 1|awk '{print $3}')
    echo "${SUBNET}/${SUBNET_SIZE}" > $DOCKER_PREFIX_FILE

    if [ "$old_ip6_prefix" != "$new_ip6_prefix" ]; then
        service docker restart
    fi
fi

For this to work you need to modify /etc/default/docker so that this line reads:

DOCKER_OPTS="--ipv6 --fixed-cidr-v6=`cat /etc/docker/ipv6.prefix`"

The result

Docker was now running properly with a IPv6 subnet configured and my containers have a IPv6 address as well.

wido@wido-desktop:~$ docker exec -ti 94c8f02 ip addr show dev eth0
13: eth0:  mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.2/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2001:980:XXXX:140:0:242:ac11:2/80 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe11:2/64 scope link 
       valid_lft forever preferred_lft forever
wido@wido-desktop:~$

Native IPv6 in my Docker containers fully automated and dynamic!

All the scripts I used can be found on Github.

MariaDB Galera cluster on IPv6

MariaDB Galera

I try to set as much IPv6-only infrastructure as possible and the same goes for a new MariaDB Galera cluster I had to build.

According to the release notes MariaDB 10.1 should have IPv6 support, but it didn’t work out for me. I wouldn’t get my Galera cluster to work over IPv6-only.

Galera

I tracked the root-cause down to Galera not parsing the addresses properly and it had to be tweaked a bit.

Configuration

With the configuration posted below I was able to get a MariaDB 10.1 setup working on IPv6-only.

[mysqld]
query_cache_size=0
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_doublewrite=1
query_cache_type=0

bind-address = ::

wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so

wsrep_cluster_name="ns01"
wsrep_cluster_address="gcomm://ns011.XXX.eu,ns012.XXX.nl,ns013.XXX.info"

wsrep_sst_method=rsync

wsrep_node_name="ns011"

wsrep_provider_options = "gmcast.listen_addr=tcp://[::]:4567; ist.recv_addr=[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4568"
wsrep_node_address = "[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4567"
wsrep_sst_receive_address = "[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4444"

This resulted in the Galera cluster functioning properly on a IPv6-only network. It’s indeed a bit more configuration then with IPv4, but that will probably be resolved in a future release.

The MariaDB status properly shows being connected over IPv6:

MariaDB [(none)]> show status like 'wsrep_incoming_addresses';
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
| Variable_name            | Value                                                                                                              |
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
| wsrep_incoming_addresses | [2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:3306,[2a00:f10:400:XX:XX:d8ff:fe00:2ef]:3306,[2a00:1d20:3:XX:XX:c01:3:53]:3306 |
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [(none)]>