Connect Prometheus

There is no key to create

This is the short answer, and it is worth being blunt about because every other service in this catalog has one: Prometheus has no authentication. No users, no tokens, no roles, no read-only mode. A stock Prometheus answers whoever can reach its port, in full.

So "generate a read-only credential for Prometheus" is not a task you have been putting off — it is not a thing that exists. What you do instead is decide who can reach the port, and that is the rest of this page.

What that means for the Tier

Everywhere else in Prodpeek there are two fences: the credential refuses writes on its own, and the gateway refuses them again. Here there is one. prometheus/read-only is Tier 2, and the profile says so in the file rather than in a footnote.

What makes it defensible is the shape of the connection. This is a native adapter — Prodpeek's own fixed menu of twelve GET endpoints — not a proxy. An agent never supplies a URL path; it names a tool, and the adapter builds the request. The endpoints that would matter are unreachable because nothing constructs them:

A generic "read-only HTTP proxy for Prometheus" would be one path-traversal bug away from the first of those. A fixed menu has no such bug available. That is the whole argument for building the adapter rather than pointing a plain HTTP upstream at the API.

Step 1 — check the two flags

Open Status → Command-Line Flags in the Prometheus UI, or:

curl -s http://PROM:9090/api/v1/status/flags | python3 -m json.tool | grep -E 'admin-api|lifecycle'

Both should be "false":

"web.enable-admin-api": "false",
"web.enable-lifecycle": "false",

If either is true, turn it off unless you have a specific reason. Prodpeek cannot reach those endpoints either way — but a Prometheus that *can* be told to delete its own data by anything on the network is a Prometheus with a bigger problem than this integration.

Step 2 — decide who can reach port 9090

Pick whichever matches your setup.

A. Prodpeek and Prometheus on the same Docker network — the simplest, and the best. Do not publish 9090 to the host at all:

# docker-compose.yml
services:
prometheus:
image: prom/prometheus:v3.1.0
# No `ports:` section. Reachable from other containers, from nothing else.
networks: [obs]
prodpeek:
image: ghcr.io/prodpeek/prodpeek:latest
ports: ["127.0.0.1:8787:8787"]
networks: [obs]

The connection URL is then http://prometheus:9090, and there is no credential at all. This is the recommended shape for EJD.

B. Prometheus on another host, private network. Bind it to the private interface and firewall the port to the Prodpeek host only:

# prometheus.yml is not where this lives — it is a command-line flag:
--web.listen-address=10.0.0.5:9090

# ufw, on the Prometheus host
ufw allow from 10.0.0.9 to any port 9090 proto tcp   # 10.0.0.9 = Prodpeek
ufw deny 9090

C. Prometheus reachable over the public internet. Put a reverse proxy in front with basic auth, and give Prodpeek that login. Caddy makes this two lines:

prom.example.dk {
basic_auth {
prodpeek $2a$14$...   # caddy hash-password
}
reverse_proxy localhost:9090
}

nginx, if you prefer:

location / {
auth_basic           "prometheus";
auth_basic_user_file /etc/nginx/prom.htpasswd;   # htpasswd -c ... prodpeek
proxy_pass           http://127.0.0.1:9090;
}

The connection URL is https://prom.example.dk, the credential is prodpeek:the-password, and the connection's authentication method is Username & password.

Step 3 — add it to Prodpeek

In the console: Services → Add a service → pick prometheus/read-only. Set the upstream to the base URL (http://prometheus:9090, or https://prom.example.dk). Leave the credential empty for A and B; for C, choose Username & password and enter user:password.

From an agent:

add_service  profile=prometheus/read-only  url=http://prometheus:9090  kind=prometheus

With no credential it returns a drop link. For A and B there is nothing to drop — just save it from the console instead, since an empty credential is legitimate here.

Then Test connection. You should see the twelve allowed tools, and nothing under "advertised but absent from the policy" — the adapter's menu and the profile are checked against each other in CI, so a mismatch would be a bug in Prodpeek rather than in your setup.

What your agent can then do

prometheus__targets                     which scrapes are up, and which are failing
prometheus__query    query=up           the classic "is everything alive"
prometheus__query_range query=rate(http_requests_total[5m]) window=6h
prometheus__alerts                      what is firing right now
prometheus__tsdb_status                 why Prometheus is using 40GB
prometheus__label_values label=job      what this Prometheus actually scrapes

Two limits are built into the adapter and will produce a clear refusal rather than a slow Prometheus: a range query may cover at most 31 days, and its step may be no finer than 15 seconds. Both exist because PromQL has no write form — its danger is cost, not mutation — and one unbounded query is how a monitoring system becomes the outage.

What it refuses, and the one that surprises people

/api/v1/status/config is denied, and it is a genuine read. That is the point of the read_leaks classification: prometheus.yml contains your scrape configs, and scrape configs routinely carry basic_auth passwords and bearer tokens for the things being scraped. Reading it is how an agent would collect every credential your monitoring has.

You can still read it yourself. It is not secret from *you*.

If Prometheus is already behind Grafana

Then you may not need this integration at all. Grafana proxies its datasources, so a Grafana service account can query Prometheus through the Grafana connection you already have — one credential, one audit trail, and the Grafana service account's role does the scoping.

Use this profile when you want Prometheus directly: when there is no Grafana in front of it, when you want targets and TSDB cardinality (which Grafana does not expose), or when you want the two audited separately.

Screenshots

Screenshot pending — the steps above stand on their own.

Screenshot pending — the steps above stand on their own.