Docker - StatsD/Graphite/Grafana

bogotobogo.com site search:

Terminology

Graphite: made up of three components: the graphite web app, the carbon relay daemon and the whisper data format.
Graphite webapp: reads the data from files in Whisper and allows users to query the values and perform statistical operations on them. It's a Django webapp that renders graphs on-demand using Cairo
StatsD: UDP based back-end proxy for the Graphite/Carbon metrics server. With StatsD, applications are to be instrumented by developers using language-specific client libraries. It sends aggregates to one or more pluggable backend services such as Graphite.
StatsD client library: sends each individual call to the StatsD server over a UDP.
StatsD daemon: listens to the UDP traffic from all application libraries, aggregate data over time and flush it at the desired interval to the backend.
Carbon: a daemon that listens on a TCP socket for time-series data which it persists to files in the whisper format.
Whisper: time-series database library.
CollectD: statistics collection daemon that periodically polls various sources for metrics. Tle latest CollectD supports StatsD as an input, so there is no need to run separate daemon.

Modified from this source: https://graphite.readthedocs.io/en/latest/faq.html

Docker container

Let's start a Docker container named: graphite.

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 81:81\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 hopsoft/graphite-statsd

The mapped ports look like this:

As we can see below, this starts a Docker container named: graphite:

$ docker ps
CONTAINER ID        IMAGE                                                     COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                       NAMES
a592790295c4        graphiteapp/graphite-statsd                               "/entrypoint"            6 minutes ago       Up 6 minutes        0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 2013-2014/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp   graphite

To send some stats, let's create stats with a random counter using netcat with 1 sec timeout and UDP to localhost at port 8125:

$ while true; do echo -n "example:$((RANDOM % 100)) " | nc -w 1 -u 127.0.0.1 8125; done

Visuaized (http://localhost:81/render?from=-10mins&until=now&target=stats.example):

Graphite

Graphite is available at: http://localhost:81:

Let's login at http://localhost:81/account/login with root/root:

Then, update the admin user's profile at: http://localhost:81/admin/users/edit/1:

We can visualize the data, open a browser with this url: http://localhost:81/render?from=-10mins&until=now&target=stats.example

Grafana

Let's login at: http://localhost with admin/admin:

Then, we may want to update the admin user's profile at: http://localhost/admin/users/edit/1

Connect Grafana to Graphite

Note that Grafana and graphite are separate applications. Grafana is for visualizing data, especially time series databases. Graphite is a time series databases. Grafana can also visualize data from InfluxDB, Prometheus, ElasticSearch: