This week I’m testing a lot with CephFS and one of the things I never tried was re-exporting CephFS using NFS-Ganesha and libcephfs.
NFS-Ganesha is a NFS server which runs in userspace. It has multiple backends (FSALs) it can use and libcephfs is one of them.
libcephfs is a userspace library which you can use to access CephFS. It is written in C/C++ and has Java and Python bindings. NFS-Ganesha however links to the native C++ bindings.
Running NFS-Ganesha on Ubuntu 14.04 is not plug and play, it involves manual compiling which I’ll explain below.
I tested this using:
- Ubuntu 14.04.1
- Ceph 0.89
- NFS-Ganesha 2.1
Building NFS-Ganesha
It starts with installing a couple of packages:
apt-get install git-core cmake build-essential portmap libcephfs-dev bison flex libkrb5-dev libtirpc1
We then clone the Git repository:
cd /usr/src git clone https://github.com/nfs-ganesha/nfs-ganesha.git cd nfs-ganesha git checkout -b V2.1-stable origin/V2.1-stable git submodule update --init
Now we have the sources we can build it:
mkdir build cd build cmake ../src make make install
NFS-Ganesha uses DBus and we have to copy a DBus profile:
cp ../src/scripts/ganeshactl/org.ganesha.nfsd.conf /etc/dbus-1/system.d/
Configuring the NFS export
Now we can create our NFS-Ganesha configuration:
nano /usr/local/etc/ganesha.conf
Add the following:
EXPORT
{
Export_ID = 1;
Path = "/";
Pseudo = "/";
Access_Type = RW;
NFS_Protocols = "3";
Squash = No_Root_Squash;
Transport_Protocols = TCP;
SecType = "none";
FSAL {
Name = CEPH;
}
}
With this configuration we say that we want to export (Path) “/” of our CephFS filesystem as “/” (Pseudo) from our NFS server.
Configuring Ceph
To run NFS-Ganesha you have to make sure that CephFS is up and running and that the server where you are going to run Ganesha on can access the Ceph cluster.
Make sure your ceph.conf and ceph.client.admin.keyring file are both present in /etc/ceph and run:
ceph -s
If that works you can start NFS-Ganesha
Starting NFS-Ganesha
Now Ceph is working we can start the NFS server:
ganesha.nfsd -f /usr/local/etc/ganesha.conf -L /tmp/ganesha.log -N NIV_DEBUG -d
This makes the NFS server log with a DEBUG profile. It gives you a lot of insight on what’s happing. You probably want to disable this when it all works.
Mounting NFS
On a NFS client we can now mount the NFS filesystem which is actually our CephFS:
mkdir /mnt/cephfs-nfs mount -o rw,noatime 1.2.3.4:/ /mnt/cephfs-nfs
Replace 1.2.3.4 with the hostname/IP-Address of the server running NFS-Ganesha.
You should now have a NFS mount which shows you your CephFS filesystem! This way legacy clients can access the most awesome filesystem.
