Using the internet on a IPv6-only network

At home I have native IPv6 via my ISP ZeelandNet since June 2014. Ever since I’ve been using the internet via IPv6 where possible.

Yesterday I thought it was time to create a IPv6-only VLAN + SSID at home and see what parts of the internet I could use while being on a IPv6-only network. No NAT64 or anything, just IPv6.

Linux router

I’m using a Soekris NET6501 with Ubuntu as my router at home. So I created a new VLAN and used that VLAN tag to create a new SSID on my Access Point.

Under Ubuntu I configured:

  • Radvd for Router Advertisements
  • Wide DHCPv6 Server for DNS servers

IPv6-only under iOS 9.1

I have an iPhone 5s and iPad Air 2 both running iOS 9.1 and I thought it was best to use these for testing the IPv6-only network.

They connected just fine! But the WiFi overview didn’t show any IP-Address. Seems that is still IPv4-only.

iOS 9.1 IPv6-only network

And showed that I had IPv6 connectivity only.

IPv6 test iOS 9.1

What works?

You might think that the internet breaks, but I think that already a lot of the large services work. A list of things which work:

  • Facebook / Messenger
  • Google: Search, YouTube, Maps and Gmail
  • NOS (Dutch news
  • Netflix
  • Apple Notifications
  • My own website and E-Mail
  • Various local sites I visit

What does not work?

Well, this could be a very long list. But there are certain services which should be highlighted for not supporting IPv6:

  • Twitter
  • Github
  • Apple App Store
  • Spotify
  • All Dutch Online banking

So yes, the biggest part of the internet does not work over IPv6. But most of the things work for me.

I’ll keep testing the internet using this IPv6-only SSID and I’ll probably keep bugging various admins to turn on IPv6.

Ubuntu and the changing MAC address with bonding

With the ‘new’ style for configuring bonding under Ubuntu your bond device will not always have the same MAC address across reboots.

For example, you configure your bond in the /etc/network/interfaces file:

auto p9p1
iface p9p1 inet manual
        bond-master bond0

auto p10p1
iface p10p1 inet manual
        bond-master bond0

auto bond0
iface bond0 inet manual
        bond-slaves none
        bond-mode 4
        bond-miimon 100
        bond-updelay 5
        bond-downdelay 5

During boot, both interface p9p1 and p10p1 will be hot-plugged under bond0. The first device to be plugged into the bonding device determines which MAC address the bonded device gets.

Due to hardware timing it might be p9p1 OR p10p1 which is the first. This behavior makes the MAC address selection inconsistent between reboots and that might cause problems with:

  • DHCP for IPv4
  • IPv6 with SLAAC (Stateless Auto Configuration)
  • DHCPv6

This has been filed as bug #1288196 with Ubuntu, but no fix from that side so far.

The solutions for now:

auto p9p1
iface p9p1 inet manual
        bond-master bond0

auto p10p1
iface p10p1 inet manual
        pre-up sleep 5
        bond-master bond0

This makes sure p10p1 always comes online 5 seconds after p9p1.

But you can also set a static MAC address for the bonding device:

auto bond0
iface bond0 inet manual
        hwaddress fe:80:12:04:6d:6f
        bond-slaves none
        bond-mode 4
        bond-miimon 100
        bond-updelay 5
        bond-downdelay 5

Choose what you prefer or works best in your situation.

Using the Link-Local Address of IPv6

Link Local

One of the things not know to people is the functionality a Link-Local Address with IPv6 provides.

You might have seen them on your Linux (or any other) system. For example, on my Linux system:

wido@desktop:~$ ip addr show dev eth1
3: eth1:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:8f:9f:af:62 brd ff:ff:ff:ff:ff:ff
    inet brd scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:8fff:fe9f:af62/64 scope link 
       valid_lft forever preferred_lft forever

As you can see, my Link-Local Address in this case is fe80::5054:8fff:fe9f:af62. What can I do with it?

What is it used for?

With IPv6 the Link-Local Address is used for multiple purposes:

  • Finding Routers using a Router Solicitation
  • Performing Duplicate Address Detection
  • Finding Neighbors

The Link-Local is however a fully functional address which you can use for multiple things.

Using Link-Local

Here at the office my colleague has a desktop and his Link-Local Address is fe80::821f:2ff:fed6:5f08.

So can I ping the address?

wido@wido-desktop:~$ ping6 fe80::821f:2ff:fed6:5f08
connect: Invalid argument

No, that doesn’t seem to work. Or does it?

wido@wido-desktop:~$ ping6 -I eth0 -c 2 fe80::821f:2ff:fed6:5f08
PING fe80::821f:2ff:fed6:5f08(fe80::821f:2ff:fed6:5f08) from fe80::c23f:d5ff:fe68:2808 eth0: 56 data bytes
64 bytes from fe80::821f:2ff:fed6:5f08: icmp_seq=1 ttl=64 time=0.566 ms
64 bytes from fe80::821f:2ff:fed6:5f08: icmp_seq=2 ttl=64 time=0.612 ms

--- fe80::821f:2ff:fed6:5f08 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.566/0.589/0.612/0.023 ms

So when I specify the interface I can ping his desktop!

You can also specify the interface this way: fe80::821f:2ff:fed6:5f08%eth0

wido@wido-desktop:~$ ping6 -c 2 fe80::821f:2ff:fed6:5f08%eth0
PING fe80::821f:2ff:fed6:5f08%eth0(fe80::821f:2ff:fed6:5f08) 56 data bytes
64 bytes from fe80::821f:2ff:fed6:5f08: icmp_seq=1 ttl=64 time=0.539 ms
64 bytes from fe80::821f:2ff:fed6:5f08: icmp_seq=2 ttl=64 time=0.481 ms

--- fe80::821f:2ff:fed6:5f08%eth0 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.481/0.510/0.539/0.029 ms

So can I SSH to it or do anything else with it?

wido@wido-desktop:~$ ssh fe80::821f:2ff:fed6:5f08%eth0
The authenticity of host 'fe80::821f:2ff:fed6:5f08%eth0 (fe80::821f:2ff:fed6:5f08%eth0)' can't be established.
ECDSA key fingerprint is d8:d7:d0:bd:3c:6a:18:31:e5:26:b1:13:96:a8:e1:89.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'fe80::821f:2ff:fed6:5f08%eth0' (ECDSA) to the list of known hosts.
wido@fe80::821f:2ff:fed6:5f08%eth0's password: 


Indeed, I can! I can also telnet to the address:

wido@wido-desktop:~$ telnet fe80::821f:2ff:fed6:5f08%eth0 22
Trying fe80::821f:2ff:fed6:5f08%eth0...
Connected to fe80::821f:2ff:fed6:5f08%eth0.
Escape character is '^]'.

telnet> quit
Connection closed.

It is a functional address which you can use on your local network.


Even if you think IPv6 is disabled on your system because you haven’t configured it, it isn’t.

Should you disable IPv6 then? No! Learn to work with it. IPv4 space is running out very quickly, so disabling it is not a wise thing to do.

Just make sure your firewall policies for both IPv4 and IPv6 are up to date. I’ve seen many systems where IPv6 isn’t firewalled at all, which makes them open to anybody on the local network.

Link-Local Addresses are not routed over the internet, so somebody has to gain access to the local Layer 2 LAN before it can connect via Link-Local, but still, keep it in mind.

Ceph with a cluster and public network on IPv6

I’m a big fan of Ceph and IPv6, so I always try to deploy Ceph over IPv6 when possible. Ceph is the future, just like IPv6 is. Why implement legacy?

Recently I did a deployment of Ceph with a public and cluster network running over IPv6. It has a small catch, so I let me explain the cluster and public network first.

Ceph cluster and public network

This image comes from the Ceph documentation and shows the two types of network:

  • Public network for clients and monitors
  • Cluster network for inter-OSD communication (Replication and recovery)

If you want to run your Ceph cluster over IPv6 you have a couple of settings to make:

ms_bind_ipv6 = true
mon_host = [2a00:f10:XX:XX::XX]:6789, [2a00:f10:XX:XX::XY]:6789, [2a00:f10:XX:XX::YY]:6789

As you can see, you have to write the IPv6 address enclosed by [ and ]

When configuring the cluster and/or public network in the ceph.conf you should however not use them:

public_network = 2a00:f10:XX:XX:XX::/64
cluster_network = 2a00:f10:XX:XX:XY::/64

When that is set correctly it should all be working fine and your Ceph cluster will be running over IPv6 with different networks!

Yealink SIP-T20P on a IPv6-only network

At PCextreme we are looking into replacing all our current Cisco, Linksys and Polycom IP phones with new phones. The old phones are worn out and have to be replaced.

We have two demands:

  • IPv6 support
  • TLS support

After some searching I found out that neither Cisco or Polycom support IPv6 in their phones with SIP, so they we off the list.

More searching led us to Yealink and we ended up ordering a SIP-T20P.

A couple of days later I created a IPv6-only VLAN on our XS4All VDSL2 connection to I was sure there was NO IPv4 available for the phone.

It took some time to figure it out, but using the T20 over IPv6 is fairly easy.

  • Start the phone
  • Go to the Advanced Network Settings (password: admin)
  • Set the network type to IPv6

The T20 (Firmware does NOT support DHCPv6 (The T4xx models do), it relies on Router Advertisements. We had to manually enter the auto provisiong URL (over HTTP) and afterwards the phone provisioned itself.

If we choose to go for Yealink it will probably be the T4x models since they support DHCPv6 and we want the auto provisioning to be fully automatic.

Deploying Ceph over IPv6

I like to deploy Ceph clusters over IPv6. I actually think that’s the way forward. IPv4 is legacy just like iSCSI and NFS are.

Last week I was at a customer deploying a new Ceph cluster and they wanted to deploy with IPv6! Most deployment I did with IPv6 were done manually and not with ceph-deploy, but when trying to deploy with ceph-deploy over IPv6 I ran into some issues.

Before going into that I want to make something clear. With Ceph you choose either IPv4 OR IPv6. There is NO dual-stack support. So the whole cluster (including clients) communicates over IPv6 or over IPv4. Switching afterwards is not possible. So that’s why I urge people to deploy with IPv6 since you probably want to have your cluster running for a long time.

All package repos (including the Ceph ones) have IPv6 enabled, so in my opinion there is no good reason to prefer IPv4 with a Ceph deployment when IPv6 is available. I even think it’s easier in large deployment due to the Router Advertisements in IPv6.

Having that said it’s time to go back to the ceph-deploy issue.

In ceph.conf you have to enclose IPv6 addresses for monitors with a [ and ]. This is what ceph-deploy did wrong:

mon_host = 2a00:f10:X:X::X,2a00:f10:X:X::Y,2a00:f10:X:X::Z

While it should have been:

mon_host = [2a00:f10:X:X::X],[2a00:f10:X:X::Y],[2a00:f10:X:X::Z]
ms_bind_ipv6 = true

The ms_bind_ipv6 setting tells the Messenger inside Ceph to bind on IPv6. It’s important that you set that setting on all hosts in the Ceph cluster, otherwise things will go wrong badly. Heartbeats and such will not work.

I wrote a patch for ceph-deploy which fixes it. It writes the ‘mon_host’ setting correctly and also adds the ‘ms_bind_ipv6’ setting when IPv6 is used for the monitors.

Cisco 887VA on a XS4All VDSL connection

I’m going to write the rest of this post in Dutch, since the ISP I’m going to talk about is dutch.

But, for the international visitors: I had troubles getting our brand new Cisco 887VA-SEC-K9 VDSL modem working on a VDSL connection from XS4All (Dutch ISP). It took me about 8 hours in to figure out that ATM was no longer used..


Afgelopen week werd onze ADSL2+ verbinding op kantoor om gezet naar een VDSL verbinding. Vanaf ons kantoor liggen er enkele IPSec tunnels naar een Cisco ASA5510 in het datacenter. Bij de ADSL2+ verbinding hadden we een SpeedTouch ADSL2+ modem in bridge met daar achter een Cisco ASA5505 die de PPP deed.

Bij de upgrade naar VDSL besloten we om net zoals bij de SDSL verbinding die we hebben een Cisco 880 series router te pakken. Lekker makkelijk je modem + router in één en ook direct onder iOS je IPSec tunnels configureren.

Ik kreeg echter onder geen enkele mogelijkheid de Cisco 887VA werkend op de VDSL verbinding. De geleverde Fritz!Box van XS4All werkte prima, maar bij de 887 bleef de interface “ATM0” maar “down”.

XS4All zou de verbinding in de loop van de dag upgraden naar VDSL, dus ik had in de ochtend de Cisco er al tussen geprikt die toen vrolijk ADSL2+ deed. Nadat XS4All in de ochtend de verbinding naar VDSL omzette stopte alles met werken. ATM0 bleef maar down.

Uren gingen voorbij in waarin ik diverse firmwares geprobeerd heb, allerlei ATM settings, DSL modes, noem het maar op, tót ik een blogpost tegen kwam waar iemand aanhaalde dat er geen ATM meer gebruikt wordt bij VDSL, maar het een native Layer 2 verbinding is. Je moet alleen het VLAN nummer weten.

Waar ik het VLAN nummer gevonden heb weet ik niet meer, maar dit is op het KPN netwerk VLAN nummer 6.

Het duurde toen niet lang voordat ik de verbinding werkend had.

De relevante configuratie:

interface Dialer0
 ip address negotiated
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 dialer pool 1
 dialer idle-timeout 0
 dialer-group 1
 ipv6 address autoconfig default
 ipv6 enable
 ipv6 nd ra interval 30
 ipv6 dhcp client pd xs4all-ipv6 rapid-commit
 ipv6 mld query-interval 60
 ipv6 virtual-reassembly in
 ppp authentication pap callin
 ppp pap sent-username password 0 PASSWORD
 no cdp enable
 crypto map vpn
interface Ethernet0
 no ip address
interface Ethernet0.6
 encapsulation dot1Q 6
 pppoe enable group global
 pppoe-client dial-pool-number 1
interface ATM0
 no ip address
 no atm ilmi-keepalive
interface Vlan1
 ip address 192.168.X.1
 ip nat inside
 ip virtual-reassembly in
 ipv6 address 2001:980:XXXX::1/64
 ipv6 enable
 ipv6 nd other-config-flag
 ipv6 nd ra interval 30
 ipv6 dhcp server
 ipv6 mld query-interval 60
access-list 100 permit ip 192.168.X.0 any
ip nat inside source route-map nonat interface Dialer0 overload
ip route Dialer0
ipv6 route ::/0 Dialer0
dialer-list 1 protocol ip permit
no cdp run
route-map nonat permit 10
 match ip address 100

De VDSL verbinding trained op 33Mbit down en 3.4Mbit up, dit zie je op een 887VA in met:

show controllers vDSL 0

Onderaan de output zie je vervolgens:

Firmware	Source		File Name (version)
--------	------		-------------------
VDSL		embedded   	VDSL_LINUX_DEV_01212008 (1)

Modem FW  Version:	110331_1212-4.02L.03.A2pv6C032b.d23f
Modem PHY Version:	A2pv6C032b.d23f
Vender Version:		

 		  DS Channel1	  DS Channel0	US Channel1	  US Channel0
Speed (kbps):	          0	       33021	         0	        3432
SRA Previous Speed:       0	           0	         0	           0
Previous Speed:	          0	           0	         0	           0
Reed-Solomon EC:          0	       79025	         0	           0
CRC Errors:	          0	           0	         0	           0
Header Errors:	          0	           0	         0	           0
Interleave (ms):       0.00	       12.00	      0.00	        4.00
Actual INP:	       0.00	        5.00	      0.00	        2.00

Met deze configuratie werkt de VDSL verbinding van XS4All prima met zowel IPv4 als IPv6 (Het is 2012!).

Het is belangrijk om te weten dat je de 887VA-SEC-K9 nodig hebt om IPv6 werkend te krijgen! De standaard 887VA-K9 doet GEEN IPv6.

Overigens zou het wel handig zijn als XS4All de basis VDSL configuratie parameters op hun website zet. Ookal leveren ze (logisch!) geen support op andere modems zijn de parameters wel handig om te weten.

Canon MP495 supports IPv6!

While we are nearing the end of the IPv4 pool, a lot of consumer electronics (even Enterprise routers) do not support IPv6.

Today I bought a new printer to use at home. It had to be a printer which would work over WiFi, after some time at the local store I choose the Canon Pixma MP495, a simple printer, just what I needed.

After configuring it (which I had to do via Windows), I browsed to the IP of the printer and saw that it supported IPv6! (Even IPsec) Wow, that is something you don’t see often.

Haven’t tested it with my Ubuntu 10.04 laptop yet, but it is nice to see manufacturers start implementing IPv6 in ordinary products!