Ceph version Kraken (11.2.0) has been released and the Release Notes tell us that the new BlueStore backend for the OSDs is now available.
BlueStore
The current backend for the OSDs is the FileStore which mainly uses the XFS filesystem to store it’s data. To overcome several limitations of XFS and POSIX in general the BlueStore backend was developed.
It will provide more performance (mainly writes), data safety due to checksumming and compression.
Users are encouraged to test BlueStore starting with the Kraken release for non-production and non-critical data sets and report back to the community.
Deploying with BlueStore
To deploy OSDs with BlueStore you can use the ceph-deploy by using the –bluestore flag.
I created a simple test cluster with three machines: alpha, bravo and charlie.
Each machine will be running a ceph-mon and ceph-osd proces.
This is the sequence of ceph-deploy commands I used to deploy the cluster
ceph-deploy new alpha bravo charlie ceph-deploy mon create alpha bravo charlie
Now, edit the ceph.conf file in the current directory and add:
[osd] enable_experimental_unrecoverable_data_corrupting_features = bluestore
With this setting we allow the use of BlueStore and we can now deploy our OSDs:
ceph-deploy --overwrite-conf osd create --bluestore alpha:sdb bravo:sdb charlie:sdb
Running BlueStore
This tiny cluster how runs three OSDs with BlueStore:
root@alpha:~# ceph -s
cluster c824e460-2f09-4994-8b2f-108aedc52d19
health HEALTH_OK
monmap e2: 3 mons at {alpha=[2001:db8::100]:6789/0,bravo=[2001:db8::101]:6789/0,charlie=[2001:db8::102]:6789/0}
election epoch 14, quorum 0,1,2 alpha,bravo,charlie
mgr active: charlie standbys: alpha, bravo
osdmap e14: 3 osds: 3 up, 3 in
flags sortbitwise,require_jewel_osds,require_kraken_osds
pgmap v24: 64 pgs, 1 pools, 0 bytes data, 0 objects
43356 kB used, 30374 MB / 30416 MB avail
64 active+clean
root@alpha:~#
root@alpha:~# ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 0.02907 root default -2 0.00969 host alpha 0 0.00969 osd.0 up 1.00000 1.00000 -3 0.00969 host bravo 1 0.00969 osd.1 up 1.00000 1.00000 -4 0.00969 host charlie 2 0.00969 osd.2 up 1.00000 1.00000 root@alpha:~#
On alpha I see that osd.0 only has a small partition for a bit of configuration and the rest is used by BlueStore.
root@alpha:~# df -h /var/lib/ceph/osd/ceph-0 Filesystem Size Used Avail Use% Mounted on /dev/sdb1 97M 5.4M 92M 6% /var/lib/ceph/osd/ceph-0 root@alpha:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 8G 0 disk ├─sda1 8:1 0 7.5G 0 part / ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 510M 0 part [SWAP] sdb 8:16 0 10G 0 disk ├─sdb1 8:17 0 100M 0 part /var/lib/ceph/osd/ceph-0 └─sdb2 8:18 0 9.9G 0 part sdc 8:32 0 10G 0 disk root@alpha:~# cat /var/lib/ceph/osd/ceph-0/type bluestore root@alpha:~#
The OSDs should work just like OSDs running FileStore, but they should perform better.