This is a text-only version of the following page on https://raymii.org:
---
Title : Set up a Collectd server with web frontend
Author : Remy van Elst
Date : 09-04-2013
URL : https://raymii.org/s/tutorials/Collectd_server_setup_tutorial_with_web_frontend.html
Format : Markdown/HTML
---
This tutorial shows you how to set up a collectd server. It also shows you how
to set up the collectd-web frontend, an interactive gui for collectd and has it
all firewalled. What is collectd? collectd gathers statistics about the system
it is running on and stores this information. Those statistics can then be used
to find current performance bottlenecks (i.e. performance analysis) and predict
future system load (i.e. capacity planning). Or if you just want pretty graphs
of your private server and are fed up with some homegrown solution you're at the
right place, too ;). A collectd server is able to receive data from collectd
clients.
<p class="ad"> <b>Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:</b><br><br> <a href="https://leafnode.nl">I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!</a><br><br> <a href="https://github.com/sponsors/RaymiiOrg/">Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.</a><br><br> <a href="https://www.digitalocean.com/?refcode=7435ae6b8212">You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days. </a><br><br> </p>
[The collectd client tutorial can be found here][2].
We will first set up Collectd on the server, and configure it to listen on the
network. Then we will enable the web GUI, and use NGINX to reverse proxy it.
_Collectd client configuration is not handled by this tutorial._ This tutorial
is tested on Debian 6, Debian 7, Ubuntu 10.04 and Ubuntu 12.04. However, the
configuration works on any other distro.
#### Installing Collectd
First install all the required packages:
sudo apt-get install collectd librrds-perl libconfig-general-perl libhtml-parser-perl libregexp-common-perl liburi-perl libjson-perl restartd python nginx
The collectd web interface has a few perl dependencies. Restartd will be used to
make sure the webinterface is running, and NGINX will be used to proxy the
webinterface to the outside.
#### Configuring Collectd as a network server
Open your favorite editor and edit the `/etc/collectd/collectd.conf` file. Read
it, then remove it all and make sure it looks like the below config file:
## /etc/collectd/collectd.conf
Hostname $HOSTNAME$
FQDNLookup false
## This can be higher if you have a more powerfull box
Interval 30
## This can be higher if you have a more powerfull box
ReadThreads 1
LoadPlugin syslog
<Plugin syslog>
LogLevel info
</Plugin>
LoadPlugin cpu
LoadPlugin df
LoadPlugin disk
LoadPlugin entropy
LoadPlugin interface
LoadPlugin irq
LoadPlugin load
LoadPlugin memory
LoadPlugin processes
LoadPlugin rrdtool
LoadPlugin swap
LoadPlugin users
LoadPlugin network
## Server config
<Plugin "network">
# Can also be "*" "25826" to listen on 0.0.0.0
Listen "$EXTERNAL_IPV4$" "25826"
Listen "$EXTERNAL_IPV6$" "25826"
ReportStats true
SecurityLevel None
</Plugin>
## Extra Plugins
## remove to disable
LoadPlugin nginx
LoadPlugin iptables
LoadPlugin uptime
LoadPlugin dns
LoadPlugin ping
<Plugin rrdtool>
DataDir "/var/lib/collectd/rrd"
</Plugin>
Include "/etc/collectd/filters.conf"
Include "/etc/collectd/plugins.conf"
Include "/etc/collectd/thresholds.conf"
The configuration file is relatively simple. Make sure to replace $VARIABLE$ by
the correct on for your server. You load plugins via `"LoadPlugin $name"`. The
network part is important, this defines the server. Collectd supports both IPv4
and IPv6, I have a few IPv6 IP's in the listen part, and "0.0.0.0" as IPv4
address. _Make sure the file has a blank newline at the end. If it has not,
collectd will fail to start/run correctly_.
#### Configure collectd plugins
Now create the following file: `/etc/collectd/plugins.conf`, it doesn't exist by
default. This will house the plugin config. Add the following content to it, but
make sure it matches your LoadPlugin settings above. If you don't have the ping
plugin, you also don't need the config for it.
## /etc/collectd/plugins.conf
## Static Plugins (every host has them)
<Plugin swap>
ReportByDevice false
</Plugin>
## Dynamic Plugins (loaded by Ansible based on options)
<Plugin nginx>
URL "http://127.0.0.1/nginx_status"
</Plugin>
<Plugin ntpd>
Host "localhost"
Port 123
ReverseLookups false
</Plugin>
<Plugin ping>
Host "google.com"
</Plugin>
<Plugin sensors>
SensorConfigFile "/etc/sensors3.conf"
Sensor "it8712-isa-0290/temperature-temp1"
Sensor "it8712-isa-0290/fanspeed-fan3"
Sensor "it8712-isa-0290/voltage-in8"
IgnoreSelected false
</Plugin>
<Plugin write_graphite>
<Carbon>
Host "$GRAPHITE_HOST$"
Port "2003"
Prefix "collectd"
Postfix "collectd"
StoreRates false
AlwaysAppendDS false
EscapeCharacter "_"
</Carbon>
</Plugin>
Collectd server is now set up in "server" mode. To test it we restart the
service:
/etc/init.d/collectd restart
And then check the `/var/lib/collectd/rrd/` folder and you should see some files
and folders (rrd libraries). If not then your collectd is setup wrong, see
syslog for more info.
#### Set up the web frontend
Clone the git repository to your home directory, or any other folder, but
remember the path:
git://github.com/RaymiiOrg/collectd-web.git
Move into the folder and start the app:
cd collectd-web
python runserver.py
You should now be able to go to "localhost:8888" on the host and see the web
interface. If you need to test it from the outside, use an ssh tunnel:
ssh -t -t -L 8888:localhost:8888 USER@SERVER.COM
You can now view the app in your local web browser via `http://localhost:8888`
#### Set up restartd
We will use restartd to make sure the app works even after reboot or when it
crashes. This could also be done with a nice init script, but this works the
easiest way.
Edit `/etc/restartd.conf` and make sure it has the following in it:
collectd-web ".*runserver.py" "su $USER -l -c 'pushd /home/$USER/collectd-web/ && /usr/bin/python /home/$USER/collectd-web/runserver.py' >> /var/log/$USER-collectd-server.log" "/bin/echo 'collectd-server running' >> /var/log/$USER-collectd-server.log"
Make sure you change $USER to the username which has the application, and check
if the paths are correct. If so then restart restartd:
/etc/init.d/restartd restart
(Funny isn't that? Restarting restartd?)
#### Use NGINX as a reverse proxy
If you want to make your collectd publicly available the you should follow this
part. If you don't want that and you find the SSH port forwarding tunnel works
for you, then use that.
Add the following to your NGINX configuration to set it up as reverse proxy for
the collectd server:
# It should be in a server {} block.
location /collectd {
rewrite ^/collectd(/.*)$ $1 break;
proxy_pass http://127.0.0.1:8888/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
Restart NGINX:
/etc/init.d/nginx configtest
/etc/init.d/nginx restart
Now you can reach the collectd at http://your-server/collectd.
#### Configuring the firewall
You should only allow hosts you set up to connect and send data to collectd.
Collectd supports authentication and singing, but I've had performance issues
with that on <512MB VPS servers, so that's why I firewall. The following
`iptables` and `ip6tables` rules should be added for all the hosts, so change
the IP address every time:
/sbin/iptables -A INPUT -p udp -s 1.2.3.4 --dport 25826 -j ACCEPT
/sbin/ip6tables -A INPUT -p udp -s fe80::feda:6cc1 --dport 25826 -j ACCEPT
If you have set up all the collectd client IP addresses in iptables, close the
gate:
/sbin/iptables -A INPUT -p -udp --dport 25826 -j REJECT --reject-with udp-reset
/sbin/ip6tables -A INPUT -p -udp --dport 25826 -j REJECT --reject-with udp-reset
Now you have a fully working collectd server set up.
[1]: https://www.digitalocean.com/?refcode=7435ae6b8212
[2]: https://raymii.org/s/tutorials/Collectd_client_setup_tutorial.html
---
License:
All the text on this website is free as in freedom unless stated otherwise.
This means you can use it in any way you want, you can copy it, change it
the way you like and republish it, as long as you release the (modified)
content under the same license to give others the same freedoms you've got
and place my name and a link to this site with the article as source.
This site uses Google Analytics for statistics and Google Adwords for
advertisements. You are tracked and Google knows everything about you.
Use an adblocker like ublock-origin if you don't want it.
All the code on this website is licensed under the GNU GPL v3 license
unless already licensed under a license which does not allows this form
of licensing or if another license is stated on that page / in that software:
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Just to be clear, the information on this website is for meant for educational
purposes and you use it at your own risk. I do not take responsibility if you
screw something up. Use common sense, do not 'rm -rf /' as root for example.
If you have any questions then do not hesitate to contact me.
See https://raymii.org/s/static/About.html for details.