Prometheus has become a one of the robust tool for monitoring systems due to its powerful time-series database and flexible query language. One of its key strengths lies in its pull-based model and open ecosystem. While instrumenting your application using Prometheus client libraries gives deep visibility into app-specific metrics, what happens when you need insights from external systems like your operating system, databases, or third-party APIs where instrumentation is not possible?
That’s where Exporters come into play.
1. Instrumenting Your Application with Prometheus
To begin monitoring your application with Prometheus, you typically instrument it using a Prometheus client library that matches the programming language you're using. These libraries expose an HTTP endpoint with your application’s internal metrics.
Here are a few widely used Prometheus client libraries:
· Go: prometheus/client_golang (https://github.com/prometheus/client_golang)
· Java / Scala: prometheus/client_java (https://github.com/prometheus/client_java)
· Python: prometheus/client_python (https://github.com/prometheus/client_python)
· Ruby: prometheus/client_ruby (https://github.com/prometheus/client_ruby)
· Rust: prometheus/client_rust (https://github.com/prometheus/client_rust)
For a complete list of client libraries across different languages, visit the official Prometheus documentation (https://prometheus.io/docs/instrumenting/clientlibs/).
These libraries allow you to define and expose custom metrics like request latency, job queue length, memory usage, and more directly from your code.
The Problem with Instrumentation Alone
But what if you can't instrument your system?
For example,
· You want metrics from the Linux kernel or operating system.
· You need visibility into third-party tools like Kafka, MySQL, or Jenkins.
· You’re dealing with black-box systems where you don’t have code-level access.
In such cases, you can’t use client libraries directly. You need something else…
2. Enter Exporters: Bridging the Gap
An Exporter is a service that acts as a bridge between a third-party system and Prometheus. It collects metrics from these external systems and converts them into Prometheus-compatible format, exposing them via an HTTP endpoint.
2.1 How Exporters Work?
· You deploy the exporter service on the same system you want to monitor (e.g., a Linux server, a database server).
· Prometheus scrapes the exporter endpoint at regular intervals.
· The exporter:
o Accepts the HTTP request from Prometheus,
o Collects the relevant metrics from the target system (via system calls, APIs, logs, or shell commands),
o Converts the data into Prometheus-compatible metrics format
o Returns the metrics response to Prometheus.
Exporters are generally lightweight services and are meant to run on the same machine as the system being monitored to provide accurate and timely metrics.
2.2 Types of Exporters
Exporters exist for a wide range of use cases. Here are some categories and examples:
Operating Systems
· Node Exporter: For hardware and OS metrics (CPU, memory, disk I/O) for Linux systems
· Windows Exporter: For Windows-based systems
Databases
· MySQL Exporter
· PostgreSQL Exporter
· MongoDB Exporter
· CouchDB Exporter
Messaging Systems
· Kafka Exporter
· RabbitMQ Exporter
CI/CD and DevOps Tools
· Jenkins Exporter
· JIRA Exporter
Storage Systems
· HDFS FSImage Exporter
· GPFS Exporter
APIs and Services
· Gmail Exporter
· Rancher Exporter
Hardware and Devices
· NVIDIA GPU Exporter
· IBM Z HMC Exporter
You can explore the complete list of exporters at the official Prometheus docs (https://prometheus.io/docs/instrumenting/exporters/)
2.3 When to Use Exporters
Use an exporter when:
· You don’t control the codebase of the system to be monitored.
· Instrumenting the system directly is infeasible or impractical.
· You need standardized monitoring for common third-party tools.
· You want to bootstrap observability quickly without writing custom instrumentation.
3. Node Exporter (System Metrics) Demo
Node Exporter is used to export CPU usage, memory stats, disk I/O, network stats, etc., from your Linux flavoured systems.
How to install?
brew install node_exporter
node_exporter
Metrics Endpoint:
http://localhost:9100/metrics
Prometheus Config Snippet:
scrape_configs: - job_name: 'node_exporter' static_configs: - targets: ['localhost:9100']
My prometheus.yml file looks like below.
prometheus.yml
global: scrape_interval: 15s scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['localhost:9090'] - job_name: 'node_exporter' static_configs: - targets: ['localhost:9100']
prometheus application.
prometheus --config.file=./prometheus.yml
Navigate to Prometheus targets, you can see two targets (one for promethues, and other for node_exporter).
Navigate to Prometheus Query page and execute the query up, it will show the metrics of two instances.
Execute the query ‘node_memory_free_bytes’ to get the details of total free memory in your system.
Click on Graph tab. You can visualize the free memory.
You can customise the time, for example, I set the time to 5minutes, the graph is changed like below.
One thing I noticed is that the time is not reflecting my local time. I set it by navigating to Settings Icon (Available at top right corner) -> Global settings -> Use local time.
Now the graph is changed like below.
You can observe the time is changed in horizontal axis.
In summary, Exporters are a powerful feature in the Prometheus ecosystem. They help to extend observability to systems you can’t or don’t want to instrument manually. Whether it’s your OS, databases, or external APIs, there’s likely an exporter already available or you can even build your own.
By combining instrumentation and exporters, Prometheus becomes a robust tool that can observe nearly everything in your stack.
Previous Next Home
No comments:
Post a Comment