Tag: crushmap

Sometimes you want to test if changes you are about to make to a CRUSH map will cause data to move or not.

In this case I wanted to change a rule in CRUSH where it would use device classes, but I didn’t want any of the ~1PB of data in that cluster to move.

By swapping IDs I could prevent data to move:

root default {
     id -50      # do not change unnecessarily
     id -53 class hdd        # do not change unnecessarily
     id -122 class ssd       # do not change unnecessarily

root default {
     id -53      # do not change unnecessarily
     id -50 class hdd        # do not change unnecessarily
     id -122 class ssd       # do not change unnecessarily

Notice how I swapped the IDs. After this I updated the rule:

rule rgw {
         id 6
         type replicated
         min_size 1
         max_size 10
         step take ams02-objects class hdd
         step chooseleaf firstn 0 type host
         step emit
 }

I then compiled the CRUSHMap and ran crushtool to see if there were any differences:

root@mon01:~# crushtool -i crushmap --compare crushmap.new 
 rule 0 had 0/10240 mismatched mappings (0)
 rule 1 had 0/10240 mismatched mappings (0)
 rule 2 had 0/10240 mismatched mappings (0)
 rule 3 had 0/10240 mismatched mappings (0)
 rule 4 had 0/10240 mismatched mappings (0)
 rule 5 had 0/3072 mismatched mappings (0)
 rule 6 had 0/10240 mismatched mappings (0)
 maps appear equivalent
 root@mon01:~#

No changes! So it was safe to inject this map:

root@mon01:~# ceph osd setcrushmap -i crushmap.new