【Prometheus】4. Monitoring cases

monitor linux The server

node_exporter summary

node_exporter： be used for *NIX System monitoring , Use GO Collector for language development

Using document ：https://prometheus.io/docs/guides/node-exporter/

Github：https://github.com/prometheus/node_exporter

exprot list ：https://prometheus.io/docs/instrumenting/exporters/

Download and run node_exporter:

1. download ：wget https://github.com/prometheus/node_exporter/releases/download/v1.1.2/node_exporter-1.1.2.linux-amd64.tar.gz
2. decompression ：tar -zxvf node_exporter-1.1.2.linux-amd64.tar.gz
3. Move to the working directory ：mv node_exporter-1.1.2.linux-amd64 /usr/local/node_exporter
4. Configure for system service ：
   1. vim /usr/lib/systemd/system/node_exporter.service
   2. systemctl daemon-reload
   3. systemctl start node_exporter
   4. netstat -antp | grep 9100

/usr/lib/systemd/system/node_exporter.service

Configure through service discovery node_exporter

1. modify prometheus.yml

prometheus.yml

2. Add service discovery profile

vim node.yml

3. Check the configuration file and restart prometheus

• ./promtool check config prometheus.yml
• kill -hup [pid]

monitor CPU, Memory , Hard disk

CPU Usage rate ：

100 - (avg(irate(node_cpu_seconds_total{mode="idle"}[5m])) by (instance)*100)

Memory usage ：

100 - (node_memory_MemFree_bytes+node_memory_Cached_bytes+node_memory_Buffers_bytes) / node_memory_MemTotal_bytes * 100

Hard disk usage ：

100 - (node_filesystem_free_bytes{mountpoint="/", fstype=~"ext4|xfs"} /node_filesystem_size_bytes{mountpoint="/", fstype=~"ext4|xfs"} * 100)

Monitoring service status

1. start-up node_exporter Service monitoring vim /usr/lib/systemd/system/node_exporter.service

/usr/lib/systemd/system/node_exporter.service

2. Reload the configuration file systemctl daemon-reload

3. Restart the startup service systemctl restart node_exporter

4. View the restart results ps -ef | grep node_exporter

5. View service status

node_systemd_unit_state{name="sshd.service"}

Use grafana Display monitoring data

grafana： Open source measurement analysis and visualization system , Initial password admin

Download address ：https://grafana.com/grafana/download

More Kanban ：https://grafana.com/dashboards

Usage method

1. docker install ：docker run -d --name=grafana -p 3000:3000 grafana/grafana

2. Add data sources

newly added prometheus data source

add to prometheus data source

Add database address

3. Create dashboard

Add an empty dashboard

To write sql

Multiple dashboards

4. Import other dashboards

----

monitor docker The server

cAdvisor （Container Advisor） Used to collect running container resource usage and performance information ,go Language development

Download address ：https://github.com/google/cadvisor

More Kanban ：https://grafana.com/dashboards/193

Usage method ：

1. install

docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  --privileged \
  --device=/dev/kmsg \
  gcr.io/cadvisor/cadvisor:$VERSION

2. Check whether the startup is successful docker ps -l

3. Add to prometheus monitor

• modify prometheus The configuration file ：

vim prometheus.yml

• Check prometheus Configuration file modification ：./promtool check config prometheus.yml
• restart prometheus
• Import docker grafana The dashboard 193

Import 193 View