Building a iSCSI Target (Server) under Linux is fairly simple, just install Enterprise iSCSI Target (IET) and you are ready. The Initiator (Client) is simple to, just use Open-iSCSI and you are ready to go, but how do you make this redundant?
When i first started using iSCSI i heard about the term “multipath”, i read that you could make a redundant IP link to your iSCSI Target with multipath, but how?
Searching on the web didn’t give me real practical anwsers. After using multipath for about 2 years now, i thought, why don’t i make a blog post about it so other people can have redundant iSCSI to!
For this example i have a iSCSI Target with two IP’s:
- 172.16.0.1/255.255.255.0
- 172.16.1.1/255.255.255.0
These IP’s given to eth0 and eth1, via two switches the connectivity is given to my initiator with the IP’s:
- 172.16.0.101/255.255.255.0
- 172.16.1.101/255.255.255.0
So there is a redundant network connection to the target, now we just have to start using this.
My target has as IQN: “iqn.2009-11-11.widodh.storage:iscsi-001”
I suppose you know how to configure IET and Open-iSCSI, so i’ll just skip the regular configuration. In this example my Target exports one LUN of 10GB.
On the client (Ubuntu 9.04 (Jaunty)) you have to install:
- open-iscsi
- multipath-tools
And that’s it, there is no configuration needed for multipath, this is all done dynamically.
Now we are going to discover the Target on both IP’s and log on to it:
iscsiadm -m discovery -t sendtargets -p 172.16.1.1
iscsiadm -m discovery -t sendtargets -p 172.16.0.1
iscsiadm -m node -T iqn.2009-11-11.widodh.storage:iscsi-001 -p 172.16.0.1 --login
iscsiadm -m node -T iqn.2009-11-11.widodh.storage:iscsi-001 -p 172.16.1.1 --login
The nicest thing about this is, that Multipath itself discovers that there is a redundant connection to a SCSI device and everything is done for you.
In “/dev/mapper” you’ll find (for example) “14945540000000000000000000100000099b2f8000f000000″and that is your multipath device.
You can list your multipath devices with:
multipath -ll
In my example this looked like:
14945540000000000000000000100000099b2f8000f000000dm-0 IET ,VIRTUAL-DISK
[size=35G][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
\_ 4:0:0:0 sdd 8:48 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 3:0:0:0 sdc 8:32 [active][ready]
Multipath detected a redundant path for “sdc” and “sdd” and created a device which i could use.
If one of the connections goes down for what ever reason, you should see this in your dmesg:
[ 2070.285310] device-mapper: multipath: Failing path 8:32.
Multipath will then show:
sdc: checker msg is "directio checker reports path is down"
14945540000000000000000000100000099b2f8000f000000dm-0 IET ,VIRTUAL-DISK
[size=35G][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
\_ 4:0:0:0 sdd 8:48 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:0 sdc 8:32 [failed][faulty]
Yes, you will see a lot of SCSI errors in your dmesg, but since you have a redundant path that is nothing to be worried about.
Just keep in mind, use “/dev/mapper/14945540000000000000000000100000099b2f8000f000000” as your block device for whatever you intent to use it!
Multipath in combination with iSCSI is really great, a simple network hickup will never get your services down and you can keep your network like a basic Layer-2 network, no STP is needed, the redundant paths can go over fully seperated links which decreases the chance for downtime!
Have fun using multipath!
