Getting connection URL from (Flask) SQLAlchemy

While working with Flask‘s SQLAlchemy extension I ran into the case where I wanted to obtain the host I was connected to combined with the username and password.

As I didn’t want to parse the URL I provided earlier to SQLAlchemy (and I didn’t have it available at that part of the code) I looked for a method to obtain it from SQLAlchemy.

It turns out this is not (or poorly) documented, but the Engine object has a attribute called url.

>>> db.engine.url
mysql+mysqlconnector://sqlmanager:***@localhost/sqlmanager?charset=utf8
>>>

This is actually a sqlalchemy.engine.url.URL object which contains attributes with the connection information:

>>> url = db.engine.url
>>> url.host
'localhost'
>>> url.username
'sqlmanager'
>>> url.password
'supersecretpassword'
>>> url.database
'sqlmanager'
>>> 

This was the information I needed and allowed me to use this information elsewhere.

Calculating SLAAC IPv6 Address in Java

SLAAC

With IPv6 a host on a network can use StateLess Address AutoConfiguration (SLAAC) to configure it’s network.

Routers will send out Router Advertisements telling the network which subnet is used in the network.

Based on their MAC address (modified EUI-64) a host will then obtain a IPv6 it will use.

Java

For the Apache CloudStack project I had to write Java code which would take a subnet and a MAC address as an argument and would generate a IPv6 SLAAC address from it.

Combining subnet 2001:db8:100::/64 with MAC address 06:7a:88:00:00:8b yields IPv6 address 2001:db8:100:0:47a:88ff:fe00:8b.

/*
 * Java code using Java-ipv6 from Google Code to convert
 * a given IPv6 subnet and a MAC address into a IPv6 address
 * calculated using SLAAC.
 *
 * Author: Wido den Hollander 
*/
import com.googlecode.ipv6.IPv6Address;
import com.googlecode.ipv6.IPv6Network;

public class IPv6EUI64 {
    public static IPv6Address EUI64Address(final IPv6Network cidr, final String macAddress) {
        if (cidr.getNetmask().asPrefixLength() > 64) {
            throw new IllegalArgumentException("IPv6 subnet " + cidr.toString() + " is not 64 bits or larger in size");
        }

        String mac[] = macAddress.toLowerCase().split(":");

        return IPv6Address.fromString(cidr.getFirst().toString() +
                Integer.toHexString(Integer.parseInt(mac[0], 16) ^ 2) +
                mac[1] + ":" + mac[2] + "ff:fe" + mac[3] +":" + mac[4] + mac[5]);
    }

    public static void main(String[] argv) {
        IPv6Network cidr = IPv6Network.fromString("2001:db8:100::/64");
        String mac = "06:7a:88:00:00:8b";
        IPv6Address eui64addr = EUI64Address(cidr, mac);

        /* This will print 2001:db8:100:0:47a:88ff:fe00:8b */
        System.out.println(eui64addr);
    }
}

The code can also be found on my Github Gist page.

From Middelburg to Trondheim

To Hirtshals

Last Saturday we left at 08:00 from Middelburg for the 1.100km drive to Hirtshals, Denmark. From there we would take the ferry to Larvik, Norway on Sunday morning.

It took us 14 hours to reach Hirtshals. Traffic was bad, very bad starting at Hamburg towards the border. Roadworks and border controls made it stop and go over almost 100km!

A short night followed since our ferry left at 08:00.

Lier South SuperCharger

After arriving in Larvik our first SuperCharger in Norway was Lier South, 100km from Larvik.

It was busy! After we parked all 8 stalls are occupied. Other Model S had to wait in the queue.

Lier South SuperCharger

A queue is bad, but it also shows that the infrastructure is used! It’s not a charger which is rarely used. From what I understood it was also a vacation period, so that might have caused the spike in traffic.

Lillehammer

After charging in Lier we headed to Lillehammer. We would stay the night there and charge again.

Fortum CHAdeMO

While heading to Lillehammer I stopped at a CHAdeMO from Fortum to see if I could charge there. The people from Fortum told me that I could use my Dutch phone and send a SMS to active it.

Well, that didn’t work. I borrowed a RFID tag from somebody else as a backup. On the Lofoten Islands I will need to use a Fortum charger, so I wanted to know if it worked. Lesson learned. It doesn’t.

Fortum CHAdeMO charger

Busy times at Lillehammer

On the E6 to Lillehammer we already spotted a lot of Model S coming from Lillehammer, so I expected the SuperCharger to be crowded.

It was! 9 of the 10 stalls we busy, so we parked at the last stall available.

As we were charging we saw more Model S arrive. We still had 100km left in the battery and we would leave the next morning. We vacated the stall and to decided to charge the next morning for the 155km drive to Dombas and Trondheim.

We checked in at the hotel and went for a dinner in Lillehammer.

SuperCharging with a cold battery

The next morning the car had been in -8C for the night. When I switched to ‘Drive’ a warning indicated that regenerative braking had been disabled. This was due to the battery being cold.

SuperCharging didn’t go very fast. When I just started it would charge with 17kW and slowly climbed to roughly 30kW before we had enough to leave for Dombas.

This was a similar experience as last year at the Krokom SuperCharger in -22C.

The picture below shows that we were charging with 24kW where under normal conditions it should have been about 80kW.

Slow Lillehammer SuperCharger

To Trondheim

From Lillehammer we drove to the Dombas SuperCharger. After a charge and lunch there we headed down to Klett (near Trondheim).

Nothing really special on this part of the trip. The temperature was about -5C and the (road) conditions were good.

To the Lofoten

Our destination is a house we rented through Airbnb on the Lofoten Islands.

From Trondheim we are taking the Hurtigruten ferry to Stamsund on the Lofoten. This will take 2 days.

From Stamsund to the house it is just 21km. Time to relax!

Energy Consumption

The tripmeter shows 1861km and a total usage of 391kWh. That’s 210Wh/km. Not bad at all!

Ceph Monitors are laggy or clock might be skewed

This weekend I got to investigate a Ceph cluster which had issues where the Monitors were constantly performing new elections.

After some investigation on of the three monitors was eating 100% CPU on a single core and kept printing this in the logs:

mon.charlie@2(peon).paxos(paxos updating c 106399655..106400232) lease_expire from mon.0 [2a00:XXX:121:XXX::6789:1]:6789/0 is 2.380296 seconds in the past; mons are probably laggy (or possibly clocks are too skewed)

Digging further I found that the LevelDB store in /var/lib/ceph/mon/X/store.db was 2.5GB in size.

Compact on Start

You can tell the monitor to compact the LevelDB database on start. Add the following to your ceph.conf:

[mon]
mon compact on start = true

Now restart the monitor and it will compact the LevelDB database.

The CPU usage now dropped and the monitors were happy again.

Last SuperCharger in Sweden, going to Norway

Leaving Sweden

In the past two days we visited Stockholm and continued our journey towards Norway.

We both never visited Stockholm before, so yesterday morning we visited Stockholm. We saw Gamla Stan (Old City) and visited the Abba Museum. (My dad is a Abba fan!)

From there on we continued to the Sollentuna and Gävle SuperChargers. We spend the night in Gävle and today we continued towards the Krokum SuperCharger via the Sundsvall.

Nothing really special actually. It started snowing and the temperatures started to drop, it’s -11 Celcius right now and snow is still falling.

Heading to Norway

Tomorrow we are going to drive from Krokum to the Grong SuperCharger in Norway. This is a 250km trip via what it seems some very small towns.

It looks like we have to cross mountains as well, so we’ll do a 100% charge at the Krokum SuperCharger before heading to Grong in Norway.

250km should be doable under the worst conditions possible with a full battery, so it’s looking good. We’ll be in Norway tomorrow!

Energy Consumption

I’m keeping a detailed spreadsheet with the energy consumption during the trip and we are currently at 225Wh/km over 2200km. That is really not bad, I expected higher consumption.

Flash LSI 2308 to IT-mode on a SuperMicro X10SL7-F mainboard

For a new SSD-only Ceph deployment I got my hands on a couple of SuperMicro SYS-1018D-73MTF 1U server with 8 Intel SSDs.

Great machines, but by default SuperMicro ships them with the IR (Integrated RAID) firmware and I wanted to use the IT (JBOD) firmware.

The X10SL7 mainboard has a UEFI bios, so flashing the controller was rather easy.

First, download the latest firmware for the LSI 2308 controller from SuperMicro. At this moment it is the PH19-IT firmware.

Now we have to create a small disk image with a FAT32 filesystem to use as a virtual drive for the IPMI.

I run Ubuntu on my desktops and laptops, so this is how I created that small disk image:

NOTE: You can generate the ISO with the steps below, or download it directly.

mkfs.msdos -C ipmi.iso 2880
sudo mkdir /media/ipmi/
sudo mount -o loop ipmi.iso /media/ipmi/
wget ftp://ftp.supermicro.com/Driver/SAS/LSI/2308/Firmware/IT/PH19-IT.zip
sudo unzip PH19-IT.zip UEFI/* -d /media/ipmi/
sudo umount /media/ipmi/

Now open your IPMI and attach ipmi.iso to your server.

Reboot the machine and hit the F11 key to open a boot menu and start the build in UEFI shell.

When in the UEFI shell enter these commands:

fs0:
cd UEFI
SMC2308T.NSH

At some point it will ask for the 9 remaining digits of the SAS address. Simply enter 000000000 (9 zeros) and you’re fine.

Reboot the system and the controller should now be in IT mode

Cisco 887VA with VDSL2 vectoring on XS4All/KPN

Note: This post is in Dutch since it’s targeted towards a Dutch audience.

Vandaag werd op kantoor onze VDSL2 verbinding van 50Mbit (non-vectoring) naar 65Mbit (vectoring) geupgrade door XS4All. Dat liep niet helemaal lekker. Onze Cisco 887VA router/modem kon daar niet goed mee overweg.

Na wat zoeken (uren) kwam ik er achter dat er een andere firmware nodig is, te weten VA_A_38k1_B_38h_24g1.bin

Om iedereen de moeite te besparen, deze firmware is hier te downloaden. (Hekel aan dat Cisco alles achter logins plaatst!)

Plaats vervolgens de firmware in de router door middel van TFTP of een HTTP-copy en dan is het slechts dit stukje configuratie:

!         
controller VDSL 0
 firmware filename flash:VA_A_38k1_B_38h_24g1.bin
!

Als ik nu in de Cisco kijk zie ik dit:

firewall#show controllers VDSL 0
Controller VDSL 0 is UP

Daemon Status:		 Up 

			XTU-R (DS)		XTU-C (US)
Chip Vendor ID:		'BDCM'			 'BDCM'
Chip Vendor Specific:   0x0000			 0xA45F
Chip Vendor Country:    0xB500			 0xB500
Modem Vendor ID:	'CSCO'			 '    '
Modem Vendor Specific:  0x4602			 0x0000
Modem Vendor Country:   0xB500			 0x0000
Serial Number Near:    FCZ162390P2 887VA-SE 15.3(3)   
Serial Number Far:     AA1250FE43S-05
Modem Version Near:    15.3(3)
Modem Version Far:     0xa45f

Modem Status:		 TC Sync (Showtime!) 
DSL Config Mode:	 AUTO 
Trained Mode:		 G.993.2 (VDSL2) Profile 17a
TC Mode:		 PTM 
Selftest Result:	 0x00 
DELT configuration:	 disabled 
DELT state:		 not running 
Trellis:		 ON			  ON
SRA: 			 disabled			 disabled
 SRA count: 		 0			 0
Bit swap: 		 enabled			 enabled
 Bit swap count:	 1710			 5
Line Attenuation:	  0.0 dB		  0.0 dB
Signal Attenuation:	  0.0 dB		  0.0 dB
Noise Margin:		 12.1 dB		 26.2 dB
Attainable Rate:	90384 kbits/s		 36750 kbits/s
Actual Power:		 12.4 dBm		 - 1.2 dBm
Per Band Status:       	D1 	D2 	D3 	U0 	U1 	U2 	U3
Line Attenuation(dB):   11.7	28.0	44.0	4.0	21.5	33.8	N/A	
Signal Attenuation(dB): 16.3	27.6	44.0	4.0	20.8	33.3	N/A	
Noise Margin(dB):       12.2	12.2	12.1	26.2	26.1	26.2	N/A	
Total FECC:		54			 0
Total ES:		0			 0
Total SES:		0			 0
Total LOSS:		0			 0
Total UAS:		78			 78
Total LPRS:		0			 0
Total LOFS:		0			 0
Total LOLS:		0			 0

Full inits:		1
Failed full inits:	0
Short inits:		0
Failed short inits:	0

Firmware	Source		File Name (version)
--------	------		-------------------
VDSL		user config	flash:VA_A_38k1_B_38h_24g1.bin (10)

Modem FW  Version:	130208_1314-4.02L.03.A2pv6C038k1.d24g1
Modem PHY Version:	A2pv6C038k1.d24g1
Vendor Version:		Ap6v38k1.24g1 68


 		  DS Channel1	  DS Channel0	US Channel1	  US Channel0
Speed (kbps):	          0	       83997	         0	        8399
SRA Previous Speed:       0	           0	         0	           0
Previous Speed:	          0	           0	         0	           0
Reed-Solomon EC:          0	          54	         0	           0
CRC Errors:	          0	           0	         0	           0
Header Errors:	          0	           0	         0	           0
Interleave (ms):       3.00	        0.00	      0.00	        0.00
Actual INP:	       4.00	       55.00	      4.00	       55.00

Training Log :	Stopped
Training Log Filename :	flash:vdsllog.bin

firewall#

USB boot issues on a Soekris net6501-70

After 5 years of great service it was time to replace my Soekris net5501-70 by a new net6501-70.

I tried to install Ubuntu 14.04 via USB, but the Soekris wouldn’t boot from my Kingston Datatraveler USB stick.

It seems the Soekris is rather picky on the USB stick. This is what you don’t want to see:

comBIOS ver. 1.41c  20121115  Copyright (C) 2000-2011 Soekris Engineering.

net6501

2048 Mbyte Memory                        CPU Atom E6xx 1600 Mhz 


SATA AHCI BIOS ver. 0.61 20121115  Copyright (C) 2003-2011 Intel Corporation

Controller Bus#02, Device#06, Function#00: 02 Ports, 02 Devices
  Port-00: Hard Disk, INTEL SSDMCEAC060B3            
  Port-01: Hard Disk, WDC WD10JPVX-80JC3T0           

Soekris USB Expansion ROM ver. 1.01  20111203

82: USB 01                          Xlt -2-32   Mbyte

At 82 you see the USB stick isn’t properly discovered. This is what it should look like:

Soekris USB Expansion ROM ver. 1.01  20111203

82: USB 01  General UDisk           Xlt 994-255-63  7988 Mbyte

That shows t he USB stick is detected properly and you can now install Ubuntu! Simply dd a ISO to the USB stick.

Calculating RADOS objects for RBD images

Ceph’s RBD (RADOS Block Device) is just a thin wrapper on top of RADOS, the object store of Ceph.

It stripes (by default) over 4MB objects in RADOS. It’s very simple to calculate which RADOS object corresponds with which sector on your RBD image/block device.

First you have to find out the block device’s object prefix name and the stripe size:

ceph@daisy:~$ sudo rbd info test
rbd image 'test':
	size 128 MB in 32 objects
	order 22 (4096 KB objects)
	block_name_prefix: rb.0.1066.2ae8944a
	format: 1
ceph@daisy:~$

In this case the stripe size is 4MB (order 2^22) and the object name prefix is rb.0.1066.2ae8944a

With one line of Perl we can calculate the object name in RADOS:

perl -e 'printf "BLOCK_NAME_PREFIX.%012x\n", ((SECTOR_OFFSET * 512) / (4 * 1024 * 1024))'

Let’s say that we want the object for sector 1 of our block device:

perl -e 'printf "rb.0.1066.2ae8944a.%012x\n", ((0 * 512) / (4 * 1024 * 1024))'

This tells us that we need to fetch object rb.0.1066.2ae8944a.000000000000 from RADOS. This can be done using the ‘rados’ command:

sudo rados -p rbd get rb.0.1066.2ae8944a.000000000000 rb.0.1066.2ae8944a.000000000000

Voila, you just fetched 4MB of your drive. Might be useful if you want to do some data recovery or such.

Filesystems on USB devices for Tesla Model S

I’ve seen various posts from users on forums asking how you should format your USB devices for usage in your Tesla Model S.

It all depends on the filesystem you are using. This is what I found out:

Working filesystems:

Not Working filesystems:

The Model S is running a modified version of Linux and it seem that it supports all the filesystems which are supported by the Linux kernel natively. I haven’t tested all filesystems, but it probably supports most filesystems available in the kernel.

For large USB devices I recommend using EXT4 and you can access EXT filesystems under Windows using Ext2Fsd.