Category: Internet

Deploying Ceph over IPv6

I like to deploy Ceph clusters over IPv6. I actually think that’s the way forward. IPv4 is legacy just like iSCSI and NFS are.

Last week I was at a customer deploying a new Ceph cluster and they wanted to deploy with IPv6! Most deployment I did with IPv6 were done manually and not with ceph-deploy, but when trying to deploy with ceph-deploy over IPv6 I ran into some issues.

Before going into that I want to make something clear. With Ceph you choose either IPv4 OR IPv6. There is NO dual-stack support. So the whole cluster (including clients) communicates over IPv6 or over IPv4. Switching afterwards is not possible. So that’s why I urge people to deploy with IPv6 since you probably want to have your cluster running for a long time.

All package repos (including the Ceph ones) have IPv6 enabled, so in my opinion there is no good reason to prefer IPv4 with a Ceph deployment when IPv6 is available. I even think it’s easier in large deployment due to the Router Advertisements in IPv6.

Having that said it’s time to go back to the ceph-deploy issue.

In ceph.conf you have to enclose IPv6 addresses for monitors with a [ and ]. This is what ceph-deploy did wrong:

[global]
mon_host = 2a00:f10:X:X::X,2a00:f10:X:X::Y,2a00:f10:X:X::Z

While it should have been:

[global]
mon_host = [2a00:f10:X:X::X],[2a00:f10:X:X::Y],[2a00:f10:X:X::Z]
ms_bind_ipv6 = true

The ms_bind_ipv6 setting tells the Messenger inside Ceph to bind on IPv6. It’s important that you set that setting on all hosts in the Ceph cluster, otherwise things will go wrong badly. Heartbeats and such will not work.

I wrote a patch for ceph-deploy which fixes it. It writes the ‘mon_host’ setting correctly and also adds the ‘ms_bind_ipv6’ setting when IPv6 is used for the monitors.

100% CPU utilization on a Cisco 887VA

Some time ago I wrote a blogpost about using a Cisco 887VA router on a XS4All (dutch ISP) connection. The original article is mostly in Dutch, but I’ll keep this one in English since it will probably help users all over the world.

A couple of days ago I got an e-mail from somebody who read my blogpost and asked me if the 887VA was able to handle more then 25Mbit. I never really tested it since I thought the copper-cable in our office wasn’t that good. During a download I logged into the router and saw that the CPU was 94% utilized!

The VDSL line was however online at 38Mbit, so how could this happen? Was the router underpowered?

I couldn’t wrap my head around it. A brand new VDSL router from Cisco couldn’t handle just 25Mbit? Something had to be wrong.

Some searching brought me to the Cisco Support Forums and one of the suggestions was to turn on CEF. A Cisco technology to improve Layer 3 performance.

Logging in to the router showed me indeed that CEF was disabled for both IPv4 and IPv6.

no ip cef
no ipv6 cef

Enabling CEF was simple:

conf t
ip cef
ipv6 cef

And voila! I suddenly was able to use the full 38Mbit with just ~50% CPU load.

Quassel IRC, never miss anything on IRC!

I was one of those guys who had irssi running inside a screen on a remote Linux box somewhere. It works just fine, but I always forgot to open the SSH session so I missed a lot of IRC conversations. Private messages were a problem as well, most of the times it was a couple of days later before I noticed somebody had actually sent me a PM…

It was time to change my IRC client, with the preference to always be online.

A short search lead me to the website of Quassel IRC, a distributed IRC server/client. Exactly what I was looking for! You just install the “core” on a remote Linux box and use the Linux, Windows, Mac OSX or Android client to participate on IRC.

The core has been running on a Ubuntu 10.04 machine for about one week now and it works like a charm. My IRC conversations are secured by SSL and I never miss a PM or when somebody tags me!

Integration of the client goes well on Ubuntu 12.04 with Unity, it integrates seamlessly with Unity and notifies me whenever I’m tagged or I receive a PM.

Looking for me on IRC? Find me on OFTC @ wido where I hang out in #ceph. Or find me on Freenode @ widodh in #cloudstack

Cisco 887VA on a XS4All VDSL connection

I’m going to write the rest of this post in Dutch, since the ISP I’m going to talk about is dutch.

But, for the international visitors: I had troubles getting our brand new Cisco 887VA-SEC-K9 VDSL modem working on a VDSL connection from XS4All (Dutch ISP). It took me about 8 hours in to figure out that ATM was no longer used..

 

Afgelopen week werd onze ADSL2+ verbinding op kantoor om gezet naar een VDSL verbinding. Vanaf ons kantoor liggen er enkele IPSec tunnels naar een Cisco ASA5510 in het datacenter. Bij de ADSL2+ verbinding hadden we een SpeedTouch ADSL2+ modem in bridge met daar achter een Cisco ASA5505 die de PPP deed.

Bij de upgrade naar VDSL besloten we om net zoals bij de SDSL verbinding die we hebben een Cisco 880 series router te pakken. Lekker makkelijk je modem + router in één en ook direct onder iOS je IPSec tunnels configureren.

Ik kreeg echter onder geen enkele mogelijkheid de Cisco 887VA werkend op de VDSL verbinding. De geleverde Fritz!Box van XS4All werkte prima, maar bij de 887 bleef de interface “ATM0” maar “down”.

XS4All zou de verbinding in de loop van de dag upgraden naar VDSL, dus ik had in de ochtend de Cisco er al tussen geprikt die toen vrolijk ADSL2+ deed. Nadat XS4All in de ochtend de verbinding naar VDSL omzette stopte alles met werken. ATM0 bleef maar down.

Uren gingen voorbij in waarin ik diverse firmwares geprobeerd heb, allerlei ATM settings, DSL modes, noem het maar op, tót ik een blogpost tegen kwam waar iemand aanhaalde dat er geen ATM meer gebruikt wordt bij VDSL, maar het een native Layer 2 verbinding is. Je moet alleen het VLAN nummer weten.

Waar ik het VLAN nummer gevonden heb weet ik niet meer, maar dit is op het KPN netwerk VLAN nummer 6.

Het duurde toen niet lang voordat ik de verbinding werkend had.

De relevante configuratie:

interface Dialer0
 ip address negotiated
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 dialer pool 1
 dialer idle-timeout 0
 dialer-group 1
 ipv6 address autoconfig default
 ipv6 enable
 ipv6 nd ra interval 30
 ipv6 dhcp client pd xs4all-ipv6 rapid-commit
 ipv6 mld query-interval 60
 ipv6 virtual-reassembly in
 ppp authentication pap callin
 ppp pap sent-username USERNAME@xs4all.nl password 0 PASSWORD
 no cdp enable
 crypto map vpn
!
interface Ethernet0
 no ip address
!
interface Ethernet0.6
 encapsulation dot1Q 6
 pppoe enable group global
 pppoe-client dial-pool-number 1
!
interface ATM0
 no ip address
 shutdown
 no atm ilmi-keepalive
!
interface Vlan1
 ip address 192.168.X.1 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
 ipv6 address 2001:980:XXXX::1/64
 ipv6 enable
 ipv6 nd other-config-flag
 ipv6 nd ra interval 30
 ipv6 dhcp server
 ipv6 mld query-interval 60
!
access-list 100 permit ip 192.168.X.0 0.0.0.255 any
ip nat inside source route-map nonat interface Dialer0 overload
ip route 0.0.0.0 0.0.0.0 Dialer0
ipv6 route ::/0 Dialer0
dialer-list 1 protocol ip permit
no cdp run
!
route-map nonat permit 10
 match ip address 100

De VDSL verbinding trained op 33Mbit down en 3.4Mbit up, dit zie je op een 887VA in met:

show controllers vDSL 0

Onderaan de output zie je vervolgens:

Firmware	Source		File Name (version)
--------	------		-------------------
VDSL		embedded   	VDSL_LINUX_DEV_01212008 (1)

Modem FW  Version:	110331_1212-4.02L.03.A2pv6C032b.d23f
Modem PHY Version:	A2pv6C032b.d23f
Vender Version:		


 		  DS Channel1	  DS Channel0	US Channel1	  US Channel0
Speed (kbps):	          0	       33021	         0	        3432
SRA Previous Speed:       0	           0	         0	           0
Previous Speed:	          0	           0	         0	           0
Reed-Solomon EC:          0	       79025	         0	           0
CRC Errors:	          0	           0	         0	           0
Header Errors:	          0	           0	         0	           0
Interleave (ms):       0.00	       12.00	      0.00	        4.00
Actual INP:	       0.00	        5.00	      0.00	        2.00

Met deze configuratie werkt de VDSL verbinding van XS4All prima met zowel IPv4 als IPv6 (Het is 2012!).

Het is belangrijk om te weten dat je de 887VA-SEC-K9 nodig hebt om IPv6 werkend te krijgen! De standaard 887VA-K9 doet GEEN IPv6.

Overigens zou het wel handig zijn als XS4All de basis VDSL configuratie parameters op hun website zet. Ookal leveren ze (logisch!) geen support op andere modems zijn de parameters wel handig om te weten.

Printing over IPv6 to a Canon MP495

Yesterday I posted that my new Canon Pixma MP495 also supports IPv6.

I had to test if I could print over IPv6, so I switched from IPv4 to IPv6 in the printer configuration (Note: You have to select IPv4 or IPv6, there is no Dual-Stack!). Before doing so I wrote down the MAC Address of the printer, I would need that to find it on my network, since the printer would get a IP from the Router Announcements my Linux router send out.

After turning on IPv6 the printer got his address within a few seconds and I was able to browse through the webinterface with Firefox.

Now I wanted to print over IPv6, the first thing I checked was if CUPS under Ubuntu 10.04 supported IPv6. It seems that CUPS supports IPv6 since version 1.2 and Ubuntu 10.04 is shipped with CUPS 1.4, so that was OK.

Then I created a DNS record for my printer, I pointed a AAAA-record to my printer, just so I dind’t have to type the address all the time. And DNS has been developed for NOT typing IP-Addresses, isn’t it?

Now I had to configure CUPS to print over IPv6, my goal was to do this via the GUI and not use any command-line stuff, that was even easier that I thought.

Adding the printer can be done in a few simple steps:

  • Go to System -> Administration -> Printing
  • Add a printer
  • Choose “Network Printer”
  • Choose LPD/LPR Host or Printer
  • In the host field, put the DNS record to your printer (or add the printer in /etc/hosts)
  • Then choose “Probe”
  • At “Queue”, select “ps”
  • Click on “Forward”
  • Choose “Provide a PPD file”
  • Download this PPD file and choose it as the driver
  • Add the printer!

Your printer settings should then look like:

Your are all set, the printer should work over IPv6 after this steps. Happy printing over IPv6!

Canon MP495 supports IPv6!

While we are nearing the end of the IPv4 pool, a lot of consumer electronics (even Enterprise routers) do not support IPv6.

Today I bought a new printer to use at home. It had to be a printer which would work over WiFi, after some time at the local store I choose the Canon Pixma MP495, a simple printer, just what I needed.

After configuring it (which I had to do via Windows), I browsed to the IP of the printer and saw that it supported IPv6! (Even IPsec) Wow, that is something you don’t see often.

Haven’t tested it with my Ubuntu 10.04 laptop yet, but it is nice to see manufacturers start implementing IPv6 in ordinary products!