Prometheus - Cluster Monitoring

Running on pi1

Clients

To add clients edit /home/baz/docker/prometheus/config (on pi1), and add a new section like:

    - job_name: "metal1"
      scrape_interval: 15s
      static_configs:
        - targets: ['192.168.182.5:9100']

Docker

$ sudo vim /etc/docker/daemon.json
{
  "metrics-addr": "0.0.0.0:9323"
}

$sudo systemctl reload docker

Did not work, might need a full restart.

Linux Host

Download appropriate node_exported from github.

Then follow (https://ourcodeworld.com/articles/read/1686/how-to-install-prometheus-node-exporter-on-ubuntu-2004).

tar xvf node_exporter-1.7.0.linux-amd64.tar.gz
cd ~/bin/node_exporter-1.7.0.linux-amd64
sudo cp node_exporter /usr/local/bin
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
sudo nano /etc/systemd/system/node_exporter.service
sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter

/etc/systemd/system/node_exporter.service:

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target