Saturday, 24 May 2025

Understanding Prometheus Exporters: Extending Metrics Beyond Your Application

Prometheus has become a one of the robust tool for monitoring systems due to its powerful time-series database and flexible query language. One of its key strengths lies in its pull-based model and open ecosystem. While instrumenting your application using Prometheus client libraries gives deep visibility into app-specific metrics, what happens when you need insights from external systems like your operating system, databases, or third-party APIs where instrumentation is not possible?

 

That’s where Exporters come into play.

 

1. Instrumenting Your Application with Prometheus

To begin monitoring your application with Prometheus, you typically instrument it using a Prometheus client library that matches the programming language you're using. These libraries expose an HTTP endpoint with your application’s internal metrics.

 

Here are a few widely used Prometheus client libraries:

 

·      Go: prometheus/client_golang (https://github.com/prometheus/client_golang)

·      Java / Scala: prometheus/client_java (https://github.com/prometheus/client_java)

·      Python: prometheus/client_python (https://github.com/prometheus/client_python)

·      Ruby: prometheus/client_ruby (https://github.com/prometheus/client_ruby)

·      Rust: prometheus/client_rust (https://github.com/prometheus/client_rust)

 

For a complete list of client libraries across different languages, visit the official Prometheus documentation (https://prometheus.io/docs/instrumenting/clientlibs/).

 

These libraries allow you to define and expose custom metrics like request latency, job queue length, memory usage, and more directly from your code.

 

The Problem with Instrumentation Alone

But what if you can't instrument your system?

 

For example,

·      You want metrics from the Linux kernel or operating system.

·      You need visibility into third-party tools like Kafka, MySQL, or Jenkins.

 

·      You’re dealing with black-box systems where you don’t have code-level access.

 

In such cases, you can’t use client libraries directly. You need something else…

 

2. Enter Exporters: Bridging the Gap

An Exporter is a service that acts as a bridge between a third-party system and Prometheus. It collects metrics from these external systems and converts them into Prometheus-compatible format, exposing them via an HTTP endpoint.

 

2.1 How Exporters Work?

·      You deploy the exporter service on the same system you want to monitor (e.g., a Linux server, a database server).

·      Prometheus scrapes the exporter endpoint at regular intervals.

·      The exporter:

o   Accepts the HTTP request from Prometheus,

o   Collects the relevant metrics from the target system (via system calls, APIs, logs, or shell commands),

o   Converts the data into Prometheus-compatible metrics format

o   Returns the metrics response to Prometheus.

 

Exporters are generally lightweight services and are meant to run on the same machine as the system being monitored to provide accurate and timely metrics.

 

2.2 Types of Exporters

Exporters exist for a wide range of use cases. Here are some categories and examples:

 

Operating Systems

·      Node Exporter: For hardware and OS metrics (CPU, memory, disk I/O) for Linux systems

·      Windows Exporter: For Windows-based systems

 

Databases

·      MySQL Exporter

·      PostgreSQL Exporter

·      MongoDB Exporter

·      CouchDB Exporter

 

Messaging Systems

·      Kafka Exporter

·      RabbitMQ Exporter

 

CI/CD and DevOps Tools

 

·      Jenkins Exporter

·      JIRA Exporter

 

Storage Systems

·      HDFS FSImage Exporter

·      GPFS Exporter

 

APIs and Services

·      Gmail Exporter

·      Rancher Exporter

 

Hardware and Devices

·      NVIDIA GPU Exporter

·      IBM Z HMC Exporter

 

You can explore the complete list of exporters at the official Prometheus docs (https://prometheus.io/docs/instrumenting/exporters/)

 

2.3 When to Use Exporters

Use an exporter when:

 

·      You don’t control the codebase of the system to be monitored.

·      Instrumenting the system directly is infeasible or impractical.

·      You need standardized monitoring for common third-party tools.

·      You want to bootstrap observability quickly without writing custom instrumentation.

 

3. Node Exporter (System Metrics) Demo

Node Exporter is used to export CPU usage, memory stats, disk I/O, network stats, etc., from your Linux flavoured systems.

 

How to install?

brew install node_exporter

node_exporter

 

Metrics Endpoint:

http://localhost:9100/metrics

 


Prometheus Config Snippet:

scrape_configs:
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['localhost:9100']

My prometheus.yml file looks like below.

 

prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['localhost:9100']

prometheus application.

 

prometheus --config.file=./prometheus.yml

 

Navigate to Prometheus targets, you can see two targets (one for promethues, and other for node_exporter).



 

Navigate to Prometheus Query page and execute the query up, it will show the metrics of two instances.

 


Execute the query ‘node_memory_free_bytes’ to get the details of total free memory in your system.  



Click on Graph tab. You can visualize the free memory.

 


You can customise the time, for example, I set the time to 5minutes, the graph is changed like below.  



One thing I noticed is that the time is not reflecting my local time. I set it by navigating to Settings Icon (Available at top right corner) -> Global settings -> Use local time.  




Now the graph is changed like below.

 


You can observe the time is changed in horizontal axis.

 

In summary, Exporters are a powerful feature in the Prometheus ecosystem. They help to extend observability to systems you can’t or don’t want to instrument manually. Whether it’s your OS, databases, or external APIs, there’s likely an exporter already available or you can even build your own.

 

By combining instrumentation and exporters, Prometheus becomes a robust tool that can observe nearly everything in your stack.

 

 

Previous                                                    Next                                                    Home

No comments:

Post a Comment