This is a text-only version of the following page on https://raymii.org:
---
Title : Adding IPv6 to a keepalived and haproxy cluster
Author : Remy van Elst
Date : 24-09-2017
URL : https://raymii.org/s/articles/Adding_IPv6_to_a_keepalived_and_haproxy_cluster.html
Format : Markdown/HTML
---
At work I regularly build high-available clusters for customers, where the setup
is distributed over multiple datacenters with failover software. If one
component fails, the service doesn't experience issues or downtime due to the
failure. Recently I was tasked with expanding a cluster setup to be also
reachable via IPv6. This article goes over the settings and configuration
required for haproxy and keepalived for IPv6. The internal cluster will only be
IPv4, the loadbalancer terminates HTTP and HTTPS connections.
<p class="ad"> <b>Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:</b><br><br> <a href="https://leafnode.nl">I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!</a><br><br> <a href="https://github.com/sponsors/RaymiiOrg/">Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.</a><br><br> <a href="https://www.digitalocean.com/?refcode=7435ae6b8212">You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days. </a><br><br> </p>
### Cluster setup
![cluster][2]
This diagram gives a general idea of the clusters I often build at work. The
CloudVPS network is fully redundant over multiple data centers, so I don't have
to worry about that part. A setup often consist out of the following components
per data center: a loadbalancer, multiple application servers (php, apache,
rails, python, java), a database server (mysql/galera or postgresql). So you
have three loadbalancer, three databaseservers and three or more application
servers in total. Often there are extra components like DRBD/NFS for file
storage, Redis as a key/value store, mongodb or elasticsearch. (All which can be
clustered). Because we have three datacenters there is enough for a quorum.
Sometimes customers choose just two datacenters for cost reasons, then we
explain the issues without quorum and make them sign off the risks in the
contract.
The clusters are IPv4 internally, with keepalived or Corosync/Pacemaker handling
the High Available IP addres (VIP). The loadbalancers all have their own IP and
share one or more external VIP's via the cluster software. They also have a
internal VIP because they function as a gateway for the internal servers. If one
loadbalancer fails, VRRP detects that and the VIP becomes active on one of the
other servers.
For complex setups with depencencies and orders we use Corosync, for example
with DRBD/NFS, so make sure the starting order is correct. First DRBD mount,
then the VIP, then NFS. Most of the time keepalived is enough.
Adding IPv6 is suprisingly easy, so this is a short article covering the
following:
* IPv6 on the OS
* keepalived
* haproxy
The internal network stays the same, the load balancer terminates all traffic
and sends it on over IPv4 to the application servers, which do not need to be
configured with IPv6.
In our case all serves come with a /64 IPv6 natively so there is no network
configuration on switches or routers included in this guide.
### Operating System
The clusters run Ubuntu (16.04) most of the time, so in
`/etc/network/interfaces` there must be an IPv6 address:
iface eth0 inet6 static
address 2a02:123:45:67ab::1/48
netmask 48
gateway 2a02:123:45::1
This is not the IPv6 address you'll use as the VIP, but a local IPv6 address for
the machine. You don't configure the VIP on the OS.
We have ACL rules on the backend in our hypervisor environment so I added an
extra IPv6 range to the cluster for use with high-availability:
2a02:123:45:67bb::1/48
(example range in this case, which will be used inside haproxy and keepalived as
IPv6 VIP.)
You also don't need to configure the following sysctl for ipv6:
net.ipv4.ip_nonlocal_bind=1
We handle that inside of haproxy and keepalived.
### Keepalived
This is tested with keepalived in ubuntu 16.04, version 1.2.19. Adding the IPv6
address to the `virtual_ipaddress` section and restarting keepalived is enough:
vrrp_sync_group VG_1 {
group {
EXTERN
INTERN
}
}
vrrp_instance EXTERN {
interface eth0
virtual_router_id 12
state EQUAL
advert_int 1
smtp_alert
notify /usr/local/bin/keepalived-extern.sh
authentication {
auth_type PASS
auth_pass hunter2
}
virtual_ipaddress {
1.2.3.4/32
2a02:123:45:67bb::1/32
}
}
### haproxy
haproxy is suprisingly easy with IPv6. Just add it to your `frontend` section as
a `bind` option:
frontend http-in
mode http
bind 1.2.3.4:80
bind 2a02:123:45:67bb::1:80 transparent
option httplog
option forwardfor
option http-server-close
option httpclose
reqadd X-Forwarded-Proto:\ http
http-request add-header X-Real-IP %[src]
default_backend appserver
You must add the `transparant` option. Otherwise, haproxy will not start if the
VIP is not on the machine itself. (kind of like nonlocal.bind sysctl).
haproxy is intelligent enough to understand the port number in the address. No need to screw around with brackets like `[2a02:123:45:67bb::1]:80` or special options.
Restart haproxy and it's configured:
netstat -tlpn | grep haproxy
Output:
tcp 0 0 1.2.3.4:80 0.0.0.0:* LISTEN 1163/haproxy
tcp 0 0 1.2.3.4:443 0.0.0.0:* LISTEN 1163/haproxy
tcp6 0 0 2a02:123:45:67bb::1:80 :::* LISTEN 1163/haproxy
tcp6 0 0 2a02:124:45:67bb::1:443 :::* LISTEN 1163/haproxy
The version of haproxy is the one from Ubuntu 16.04, 1.6.3.
[1]: https://www.digitalocean.com/?refcode=7435ae6b8212
[2]: https://raymii.org/s/inc/img/cloudvps-cluster.png
---
License:
All the text on this website is free as in freedom unless stated otherwise.
This means you can use it in any way you want, you can copy it, change it
the way you like and republish it, as long as you release the (modified)
content under the same license to give others the same freedoms you've got
and place my name and a link to this site with the article as source.
This site uses Google Analytics for statistics and Google Adwords for
advertisements. You are tracked and Google knows everything about you.
Use an adblocker like ublock-origin if you don't want it.
All the code on this website is licensed under the GNU GPL v3 license
unless already licensed under a license which does not allows this form
of licensing or if another license is stated on that page / in that software:
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Just to be clear, the information on this website is for meant for educational
purposes and you use it at your own risk. I do not take responsibility if you
screw something up. Use common sense, do not 'rm -rf /' as root for example.
If you have any questions then do not hesitate to contact me.
See https://raymii.org/s/static/About.html for details.