Monday, 19 May 2025

Getting Started with Prometheus Configuration File (prometheus.yml)

If you're new to Prometheus, one of the first things you'll encounter is the prometheus.yml configuration file. This file is the heart of Prometheus, it tells Prometheus what to monitor, where to get metrics from, how frequently to check the rules, and even how to trigger alerts.

In this post, let’s understand a sample prometheus.yml configuration line by line, so you can understand how to set up Prometheus to start scraping metrics.

 

1. What is prometheus.yml?

The prometheus.yml file is the main configuration file for Prometheus. It controls:

 

·      How often to collect metrics

·      Where to collect them from (targets)

·      How to define alerting rules

·      Where to send alerts

·      How to group different monitoring jobs

 

Let’s take a look at a sample config and understand each part.

 

2. Sample prometheus.yml 

# my global config
global:
  scrape_interval: 15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
        labels:
          app: "prometheus"

2.1 global block: This section sets default values for the entire Prometheus instance.

 

global:
  scrape_interval: 15s
  evaluation_interval: 15s

 

·      scrape_interval: How often to collect metrics from each target. In this example, Prometheus scrape the target for every 15 seconds. If you do not set any value to scrape_interval, it defaults to 1 minute.

 

·      evaluation_interval: How often Prometheus evaluates alerting rules.

 

Can we override these?

Yes! You can override scrape_interval and other global settings within each scrape_configs block using scrape_interval, scrape_timeout, etc.

 

2.2 alerting block

alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

This is where you tell Prometheus where to send alerts. The targets under alertmanagers are the hosts running Alertmanager, a separate tool that handles sending out notifications like emails, Slack messages, etc.

 

In this example, the alerting system is commented out, but to enable it, you just remove the # from the line.

- targets:
  - alertmanager:9093

 

You can setup Prometheus alertmanager (https://github.com/prometheus/alertmanager), to handle alerts sent by client applications such as the Prometheus server.

 

2.3 rule_files block

rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

 

This section lists the files that contain alerting and recording rules. You can define alerts like “CPU usage is over 90%” or record commonly-used metrics for reuse. Prometheus checks these rules at the evaluation_interval.

 

To use this, just uncomment and add the paths to your rule files:

rule_files:
  - "alerts.yml"

2.4 scrape_configs block

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
        labels:
          app: "prometheus"

This is the most important block. It defines what to scrape and from where.

 

·      job_name: A name for the group of targets. It will appear as a label in the collected data.

·      static_configs.targets: List of endpoints (host:port) to scrape metrics from.

·      labels: Optional key-value pairs to attach labels to metrics. Useful for filtering or grouping in queries.

 

Note

Prometheus expects metrics to be available at the path /metrics on each target. So in this example:

targets: ["localhost:9090"]

Prometheus will try to scrape "http://localhost:9090/metrics". You can customize this with metrics_path or scheme (e.g., HTTPS), if needed.

 

scrape_configs:
  - job_name: "secure_app"
    scheme: "https"  # Use HTTPS
    metrics_path: "/secure-metrics"  # Optional: change path

    static_configs:
      - targets: ["myapp.example.com:443"]

 

How to View Prometheus Configuration in the Web UI?

Prometheus provides a built-in web interface that not only allows you to query and explore metrics, but also helps you to inspect its internal settings—including its configuration.

 

To view the current configuration of your Prometheus instance:

 

Open your web browser and go to "http://localhost:9090". In the top navigation bar, click on “Status”, then select “Configuration” from the dropdown menu.


 

This will display the entire active configuration that Prometheus is currently using like:

 

·      Global settings (like scrape intervals)

·      Alertmanager targets

·      Rule files

·      Scrape jobs and their targets

·      Any overrides applied within job-specific configs

 

This view is read-only. If you need to change something, you'll have to edit your prometheus.yml file manually and restart Prometheus to apply the changes.

 


References

https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus.yml

 

https://github.com/prometheus/alertmanager


 

Previous                                                    Next                                                    Home

No comments:

Post a Comment