VXLAN
Virtual Extensible LAN uses encapsulation technique to encapsulate OSI layer 2 Ethernet frames within layer 4 UDP datagrams. More on this can be found on the link provided.
For a Ceph and CloudStack environment I needed to set up a Proof-of-Concept using VXLAN and some refurbished hardware. The main purpose of this PoC is to verify that VXLAN works with CloudStack, Ceph and Ubuntu 18.04
VyOS
VyOS is an open source network operating system based on Debian Linux. It supports VXLAN, so using this we were able to test VXLAN in this setup.
In production a other VXLAN capable router would be used, but for a PoC VyOS works just fine running on a regular server.
Configuration
The VyOS router is connected to ‘the internet’ with one NIC and the other NIC is connected to a switch.
Using static routes a IPv4 subnet (/24) and a IPv6 subnet (/48) are routed towards the VyOS router. These are then splitted and send to multiple VLANs.
As it took me a while to configure VXLAN under VyOS
I’m only posting that configuration.
interfaces {
ethernet eth0 {
address 31.25.96.130/30
address 2a00:f10:100:1d::2/64
duplex auto
hw-id 00:25:90:80:ed:fe
smp-affinity auto
speed auto
}
ethernet eth5 {
duplex auto
hw-id a0:36:9f:0d:ab:be
mtu 9000
smp-affinity auto
speed auto
vif 300 {
address 192.168.0.1/24
description VXLAN
mtu 9000
}
vxlan vxlan1000 {
address 10.0.0.1/23
address 2a00:f10:114:1000::1/64
group 239.0.3.232
ip {
enable-arp-accept
enable-arp-announce
}
ipv6 {
dup-addr-detect-transmits 1
router-advert {
cur-hop-limit 64
link-mtu 1500
managed-flag false
max-interval 600
name-server 2a00:f10:ff04:153::53
name-server 2a00:f10:ff04:253::53
other-config-flag false
prefix 2a00:f10:114:1000::/64 {
autonomous-flag true
on-link-flag true
valid-lifetime 2592000
}
reachable-time 0
retrans-timer 0
send-advert true
}
}
link eth5.300
mtu 1500
vni 1000
}
vxlan vxlan2000 {
address 109.72.91.1/26
address 2a00:f10:114:2000::1/64
group 239.0.7.208
ipv6 {
dup-addr-detect-transmits 1
router-advert {
cur-hop-limit 64
link-mtu 1500
managed-flag false
max-interval 600
name-server 2a00:f10:ff04:153::53
name-server 2a00:f10:ff04:253::53
other-config-flag false
prefix 2a00:f10:114:2000::/64 {
autonomous-flag true
on-link-flag true
valid-lifetime 2592000
}
reachable-time 0
retrans-timer 0
send-advert true
}
}
link eth5.300
mtu 1500
vni 2000
}
}
VLAN 300 on eth5 is used to route VNI 1000 and 2000 in their own multicast groups.
The MTU of eth5 is set to 9000 so that the encapsulated traffic of VXLAN can still be 1500 bytes.
Ubuntu 18.04
To test if VXLAN was actually working on the Ubuntu 18.04 host I made a very simple script:
ip link add vxlan1000 type vxlan id 1000 dstport 4789 group 239.0.3.232 dev vlan300 ttl 5 ip link set up dev vxlan1000 ip addr add 10.0.0.11/23 dev vxlan1000 ip addr add 2a00:f10:114:1000::101/64 dev vxlan1000
That works! I can ping 10.0.0.11 and 2a00:f10:114:1000::1 from my Ubuntu 18.04 machine!