MariaDB Galera cluster on IPv6

MariaDB Galera

I try to set as much IPv6-only infrastructure as possible and the same goes for a new MariaDB Galera cluster I had to build.

According to the release notes MariaDB 10.1 should have IPv6 support, but it didn’t work out for me. I wouldn’t get my Galera cluster to work over IPv6-only.

Galera

I tracked the root-cause down to Galera not parsing the addresses properly and it had to be tweaked a bit.

Configuration

With the configuration posted below I was able to get a MariaDB 10.1 setup working on IPv6-only.

[mysqld]
query_cache_size=0
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_doublewrite=1
query_cache_type=0

bind-address = ::

wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so

wsrep_cluster_name="ns01"
wsrep_cluster_address="gcomm://ns011.XXX.eu,ns012.XXX.nl,ns013.XXX.info"

wsrep_sst_method=rsync

wsrep_node_name="ns011"

wsrep_provider_options = "gmcast.listen_addr=tcp://[::]:4567; ist.recv_addr=[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4568"
wsrep_node_address = "[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4567"
wsrep_sst_receive_address = "[2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:4444"

This resulted in the Galera cluster functioning properly on a IPv6-only network. It’s indeed a bit more configuration then with IPv4, but that will probably be resolved in a future release.

The MariaDB status properly shows being connected over IPv6:

MariaDB [(none)]> show status like 'wsrep_incoming_addresses';
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
| Variable_name            | Value                                                                                                              |
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
| wsrep_incoming_addresses | [2a00:f10:121:XX:XX:a0ff:fe00:1bc7]:3306,[2a00:f10:400:XX:XX:d8ff:fe00:2ef]:3306,[2a00:1d20:3:XX:XX:c01:3:53]:3306 |
+--------------------------+--------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [(none)]>

IPv6 Prefix Delegation on a Cisco 887VA behind a XS4All VDSL2 connection

XS4All connection

At the PCextreme office we have a XS4All VDSL2 connection which has native IPv6. We get a /48 from XS4All.

I wrote two earlier blogposts about getting the Cisco 887VA router setup which might be of interest before you continue reading:

IPv6 Prefix Delegation

From XS4All we get a /48 routed to our office using DHCPv6 Prefix Delegation. We are experimenting and testing with Docker at the office where we also want to test the IPv6 capabilities of Docker.

The goal was to sub-delegate /60 subnets out of a /56 towards clients internally. I had to figure out how to get this configured on Cisco IOS.

  • We get a /48 delegated from XS4All
  • The first /56 is used for our local networks (LAN, Guest and Servers)
  • The second /56 is used as a pool to delegate /60 subnets from

Sipcalc

To calculate the IPv6 subnets used the tool ‘sipcalc’. I needed to find the second /56 in our /48:

sipcalc -S 56 2001:980:XX::/48

The output is rather long, so I trimmed it a bit:

-[ipv6 : 2001:980:XX::/48] - 0

[Split network]
Network			- 2001:0980:XX:0000:0000:0000:0000:0000 -
			  2001:0980:XX:00ff:ffff:ffff:ffff:ffff
Network			- 2001:0980:XX:0100:0000:0000:0000:0000 -
			  2001:0980:XX:01ff:ffff:ffff:ffff:ffff
Network			- 2001:0980:XX:0200:0000:0000:0000:0000 -
			  2001:0980:XX:02ff:ffff:ffff:ffff:ffff
...
...
Network			- 2001:0980:XX:ff00:0000:0000:0000:0000 -
			  2001:0980:XX:ffff:ffff:ffff:ffff:ffff

-

In this case 2001:0980:XX:0100:0000:0000:0000:0000:/56 is the second /56 in our /48.

Cisco IOS

Some searching brought me to cisco.com which had some examples.

Eventually it was actually quite easy to get it working.

Configuration

You need a DHCPv6 pool inside the Cisco and tell it to start a DHCPv6 server on the proper interface.

ipv6 dhcp pool local-ipv6
 prefix-delegation pool local-ipv6-pd-pool lifetime 3600 1800
 dns-server 2001:888:0:6::66
 dns-server 2001:888:0:9::99
 domain-name pcextreme.nl
interface Vlan1
 ip address 192.168.5.1 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
 ipv6 address xs4all-prefix ::1/64
 ipv6 enable
 ipv6 nd other-config-flag
 ipv6 nd ra interval 30
 ipv6 nd ra dns server 2001:888:0:6::66
 ipv6 nd ra dns server 2001:888:0:9::99
 ipv6 dhcp server local-ipv6 rapid-commit
 ipv6 mld query-interval 60
ipv6 local pool local-ipv6-pd-pool 2001:980:XX:100::/56 60

That’s all!

Asking for a Prefix

On my Ubuntu desktop I could now request a subnet:

wido@wido-desktop:~$ sudo dhclient -6 -P -v eth0
Internet Systems Consortium DHCP Client 4.2.4
Copyright 2004-2012 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Bound to *:546
Listening on Socket/eth0
Sending on   Socket/eth0
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT:  X-- IA_PD d5:68:28:08
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 1060ms.
RCV: Advertise message on eth0 from fe80::da67:d9ff:fe81:bcec.
RCV:  X-- IA_PD d5:68:28:08
RCV:  | X-- starts 1455279332
RCV:  | X-- t1 - renew  +900
RCV:  | X-- t2 - rebind +1440
RCV:  | X-- [Options]
RCV:  | | X-- IAPREFIX 2001:980:XX:100::/60
RCV:  | | | X-- Preferred lifetime 1800.
RCV:  | | | X-- Max lifetime 3600.
RCV:  X-- Server ID: 00:03:00:01:d8:67:d9:81:bc:f0
RCV:  Advertisement recorded.
PRC: Selecting best advertised lease.

As you can see I got 2001:980:XX:100::/60 delegated to my desktop.

IPv6 routes

After I asked for a subnet on my desktop this is how the routes look like. You can see a /60 being routed to my Link-Local Address.

firewall-vdsl-veldzigt#show ipv6 route
IPv6 Routing Table - default - 8 entries
Codes: C - Connected, L - Local, S - Static, U - Per-user Static route
       B - BGP, HA - Home Agent, MR - Mobile Router, R - RIP
       H - NHRP, D - EIGRP, EX - EIGRP external, ND - ND Default
       NDp - ND Prefix, DCE - Destination, NDr - Redirect, O - OSPF Intra
       OI - OSPF Inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2, ON1 - OSPF NSSA ext 1
       ON2 - OSPF NSSA ext 2, la - LISP alt, lr - LISP site-registrations
       ld - LISP dyn-eid, a - Application
S   ::/0 [1/0]
     via Dialer0, directly connected
S   2001:980:XX::/48 [1/0]
     via Null0, directly connected
C   2001:980:XX::/64 [0/0]
     via Vlan1, directly connected
L   2001:980:XX::1/128 [0/0]
     via Vlan1, receive
C   2001:980:XX:1::/64 [0/0]
     via Vlan300, directly connected
L   2001:980:XX:1::1/128 [0/0]
     via Vlan300, receive
S   2001:980:XX:100::/60 [1/0]
     via FE80::C23F:D5FF:FE68:XX, Vlan1
L   FF00::/8 [0/0]
     via Null0, receive
firewall-vdsl-veldzigt#

The subnet is working now and I can use it to hand it out to my Docker containers.

Fully electric to Norway, again!

Last Year

In the beginning of 2015 I drove a 5.500km trip with my father to the most Northern Tesla Motors SuperCharger. For a few reasons:

  • To see the Aurora Borealis
  • To prove it can be done with an electric car
  • Because I like roadtrips and travelling

This winter I’m going it again!

My Model S

I have a ‘classic’ Model S from September 2013. No Auto-Pilot features or All-Wheel drive. It’s a 85kWh RWD model.

The ODO currently displays 110.000km and when we get back it won’t be long before I hit the 120.000km.

Still enjoying this car every time I drive it.

Lofoten

My girlfriend also wants to go to Norway to see the Aurora Borealis. She heard me telling her all the stories for over a year about how great it was and how much I like Norway.

So I said: “Why don’t we go there?”

This year the destination will be the Lofoten Islands. From what I’ve seen and heard it is about the best place to watch the Aurora Borealis!

Route to Lofoten

Our route will take us from Middelburg (Netherlands) to Hirtshals (Denmark) where we take the ferry towards Larvik (Norway). Following the Tesla SuperChargers we will drive to Bodø from where we take the ferry to the Lofoten Islands.

Route Norway 2016

On the Lofoten there are no Tesla SuperChargers, so I’ll be using the CHAdeMO chargers using the CHAdeMO adapter to charge my Model S there.

Tesla CHAdeMO adapter

I found these CHAdeMO chargers on elbil.no’s Hurtigladekartet and on uppladdning.nu.

Route back home

We’ll drive back through Sweden where I want to take Fiskevägen again. What a beautiful route!

Route Norway Back 2016

The total route should be about 6.000km

Preparations

Since I did almost all the preparing last year already I still have about everything I need.

Making sure we have enough water, food, heat and proper winter gear with us. We should be fine!