At my hosting company PCextreme we are building a cloud offering based on Ceph and CloudStack. We call our cloud services Aurora.
Our cloud services are composed out of two components: Compute and Objects.
For our Aurora Objects service we use the RADOS Gateway from Ceph and we are using the Federated Config to create multiple regions.
At this moment we have one region o.auroraobjects.eu but we soon want to expand to multiple regions.
One of the things we/I wanted is a global namespace for all our regions: o.auroraobjects.com.
By design the RADOS Gateway will return a HTTP-redirect when you connect to the ‘wrong’ region for a specific bucket, but a HTTP-redirect causes extra TCP packets going over the wire causing additional and unneeded latency.
So I came up with the idea of using a custom PowerDNS backend to direct bucket traffic on DNS level.
Imagine having a bucket ceph in the region ‘eu’ and the global namespace o.auroraobjects.com.
Using my custom backend the PowerDNS server will respond with a CNAME pointing the user towards the right hostname:
wido@wido-laptop:~$ host ceph.o.auroraobjects.com ns1.auroraobjects.com
Using domain server:
Name: ns1.auroraobjects.com
Address: 2a00:f10:121:400:48c:2ff:fe00:e6b#53
Aliases:
ceph.o.auroraobjects.com is an alias for ceph.o.auroraobjects.eu.
wido@wido-laptop:~$
As you can see it responded with a CNAME pointing towards ceph.o.auroraobjects.eu.
This allows us to create multiple regions (eu, us, asia, etc) but keep one global namespace to make it easy to consume for our end-users.
Users can create a bucket in the region they like, but they never have to worry about wich hostname to use. We take care of that.
This PowerDNS backend is in the Ceph master branch and can be installed as a WSGI application behind Apache.
I’ve put a small txt file online to show you:
As you can see, both URLs show you the same object.
Deploying the backend for PowerDNS is fairly simply, I recommend you read the README, but here are a few config snippets.
Apache VirtualHost
ServerAdmin webmaster@localhost
DocumentRoot /var/www
Options FollowSymLinks
AllowOverride None
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
WSGIScriptAlias / /var/www/pdns-backend-rgw.py
PowerDNS configuration
local-address=0.0.0.0
local-ipv6=::
cache-ttl=60
default-ttl=60
query-cache-ttl=60
launch=remote
remote-connection-string=http:url=http://localhost/dns
Note: You have to compile PowerDNS manually with –with-modules=remote –enable-remotebackend-http
Don’t forget to put a rgw-pdns.conf in /etc/ceph with the correct configuration.
This is still a work-in-progress on my side and I’ll probably make some commits in the coming months, but feedback is much appreciated!