31.10.2022

Monitoring Linux Services with Prometheus

server one
HOSTKEY
Rent dedicated and virtual servers with instant deployment in reliable TIER III class data centers in the Netherlands and the USA. Free protection against DDoS attacks included, and your server will be ready for work in as little as 15 minutes. 24/7 Customer Support.

Author: Senior Devops. Hostkey Lead Infrastructure Specialist Nikita Zubarev

In the last article we talked about how to set up metric collection and the display alerts to monitor services using Prometheus. However, during the process, the question arose of how to monitor individual services on Linux servers. For example, in one of our previous articles, we described the case of using Ovirt web consoles using Apache Guacamole. Now let's look at how to organize the monitoring of services on Linux machines.

Guacamole is a cross-platform remote desktop gateway that requires Apache Tomcat and Guacd to run.

Open the service created for node_exporter:

/usr/lib/systemd/system/node_exporter.service

[Unit]

Description=Prometheus exporter for machine metrics, written in Go with pluggable metric collectors.
Documentation=https://github.com/prometheus/node_exporter
After=network.target


[Service]

EnvironmentFile=-/etc/default/node_exporter
User=root
ExecStart=/usr/bin/node_exporter $NODE_EXPORTER_OPTS
Restart=on-failure
RestartSec=5s

[Install]

WantedBy=multi-user.target

by default, $NODE_EXPORTER_OPTS is loaded from the file:
# /etc/default/node_exporter


If necessary, we can either monitor individual services by adding the collector.systemd.unit-whitelist option:

#NODE_EXPORTER_OPTS="--collector.systemd --collector.systemd.unit-whitelist="(tomcat|guacd).service""

After restarting the service, we get the metric on node_exporter:

All you need to do is describe the rule for the alert:

- alert: guacd_service
	expr: node_systemd_unit_state{name="guacd.service",state="active"} == 0
	for: 1s
	annotations:
		summary: "Instance {{ $labels.instance }} is down"
		description: "{{ $labels.instance }} of job {{ $labels.job }} is down."
- alert: tomcat_service
	expr: node_systemd_unit_state{name="tomcat.service",state="active"} == 0
	for: 1s
	annotations:
		summary: "Instance {{ $labels.instance }} is down"
		description: "{{ $labels.instance }} of job {{ $labels.job }} is down."

An example of an alert on the Alertmanager dashboard in Grafana:

Conclusions

In this short article, we looked at how services are monitored on Linux machines.This solution allows you to monitor the work of any system server and greatly simplifies the administration of server hardware. In the next article, we will explain how you should not display the basic parameters that the exporter returns.

Rent dedicated and virtual servers with instant deployment in reliable TIER III class data centers in the Netherlands and the USA. Free protection against DDoS attacks included, and your server will be ready for work in as little as 15 minutes. 24/7 Customer Support.

Other articles

01.03.2023

Creating a Single sign-on service with reminders for users

How to make a simple and reliable password change system and help users remember to change their password timely.

13.02.2023

Which graphics card is better for business - RTX 4090 or server RTX A5000?

We shall compare the new gaming graphics card from Nvidia - the GeForce RTX 4090 and the powerful RTX A5000 server card.

06.02.2023

Migrating virtual servers from oVirt to VMware

We develop and apply a convenient scheme for quickly transferring servers from oVirt to VMware.

13.01.2023

Collecting logs using Go

How to solve some issues with collecting logs using Go

05.12.2022

WindowsPE Live-CD in Jenkins/Foreman Infrastructure

Building a WindowsPE distribution in a Linux infrastructure, automating this process using Jenkins, and deploying MS Windows-based systems through this helper.

HOSTKEY Dedicated servers and cloud solutions Pre-configured and custom dedicated servers. AMD, Intel, GPU cards, Free DDoS protection amd 1Gbps unmetered port 30
4.3 67 67
Upload