Energy Metrics via the Prometheus HTTP API#

How to read household consumption and PV generation from outside the cluster — for an agent, script or notebook that has no kubectl access and talks only to https://prometheus.internal.neese-web.de.

Everything below was measured against the running system, not inferred from configuration.

1. Access: it does not work as configured today#

Prometheus sits behind Traefik with the Authelia forward-auth middleware (authelia-forwardauth-authelia@kubernetescrd), and apps/authelia/values.enc.yaml places *.internal.neese-web.de under policy: two_factor.

Measured behaviour against /api/v1/query?query=up:

Two things follow. Authelia does parse the Authorization header — a wrong password yields 401 rather than the redirect. But HTTP Basic Authentication conveys a single factor, and a two_factor policy cannot be satisfied by one factor. A headless client therefore cannot reach the API today: it follows the redirect and receives the login page as HTML instead of JSON.

One of the two changes below is required.

2. Option A — Grafana service account token (recommended)#

Grafana is not behind Authelia. Measured: /api/health returns 200 without credentials, while /api/datasources, /api/search and /api/org return 401 — Grafana enforces its own authentication.

Grafana can proxy Prometheus queries, so an agent needs no Authelia change and gets a revocable, least-privilege token instead of a user password.

Setup, once, in the Grafana UI at https://grafana.internal.neese-web.de:

1. Administration → Users and access → Service accounts → Add service account, role Viewer.
2. Add service account token, copy the glsa_… value.

Find the Prometheus datasource UID:

1
2
3

curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  https://grafana.internal.neese-web.de/api/datasources \
  | jq -r '.[] | select(.type=="prometheus") | "\(.uid)\t\(.name)"'

Then query through the datasource proxy — the path after the proxy prefix is the ordinary Prometheus API:

1
2
3

curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  --get --data-urlencode 'query=home:power_total:watts' \
  "https://grafana.internal.neese-web.de/api/datasources/proxy/uid/$DS_UID/api/v1/query"

Trade-off: the URL is Grafana's, not prometheus.internal.neese-web.de, and the token grants read access to everything that datasource can read. It changes no security policy and can be revoked in one click.

3. Option B — a one_factor rule in Authelia#

If the agent must talk to prometheus.internal.neese-web.de directly, lower only the API paths to one factor. Add this to the access_control.rules list in apps/authelia/values.enc.yaml, above the *.internal.neese-web.de catch-all — Authelia evaluates rules in order and the first match wins, so a rule placed after the wildcard never applies:

1
2
3
4

- domain: prometheus.internal.neese-web.de
  policy: one_factor
  resources:
    - '^/api/v1/(query|query_range|series|labels|label)($|/.*)$'

Create a dedicated account in lldap for the agent rather than reusing a human login. The browser UI keeps two_factor, because it is not matched by the resources regex.

The agent then authenticates with ordinary basic auth:

1
2
3

curl -s -u "$AGENT_USER:$AGENT_PASSWORD" \
  --get --data-urlencode 'query=home:power_total:watts' \
  https://prometheus.internal.neese-web.de/api/v1/query

Not yet verified end to end. The 401/302 measurements above establish that Authelia parses Authorization, but the success path could not be tested without valid credentials. Confirm it returns 200 with JSON after applying the rule; if it still redirects, the rule ordering is the first thing to check.

Trade-off: read access to every metric in the cluster is reduced to a single password. Option A is preferable unless the direct hostname is a requirement.

4. Querying#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# instant value
curl -s --get --data-urlencode 'query=<promql>' "$BASE/api/v1/query"

# instant value at a point in time (RFC3339 or unix seconds)
curl -s --get --data-urlencode 'query=<promql>' \
     --data-urlencode 'time=2026-09-07T12:00:00+02:00' "$BASE/api/v1/query"

# range
curl -s --get --data-urlencode 'query=<promql>' \
     --data-urlencode 'start=…' --data-urlencode 'end=…' \
     --data-urlencode 'step=1d' "$BASE/api/v1/query_range"

Use --data-urlencode; several of the queries below contain +, - and {} which break if pasted raw into a URL. POST works on the same paths and is safer for long expressions.

5. Metric catalogue#

Inverter — Deye SUN-M160G4-EU-Q0#

Read over Solarman V5 from the logger stick, published to MQTT, exposed by mqtt2prometheus in namespace solar. Values shown are examples from a midday scrape.

Only two of the four MPPT inputs are connected; pv3 and pv4 read 0.

Grid meter — FRITZ!Smart Energy 250#

Read over TR-064/AHA from the FRITZ!Box by fritzbox-exporter in namespace monitoring. Select the meter with {ain=~"16000.?0036823.*"}: the device appears both under its own AIN and under the sub-unit …-1, and only the sub-unit carries the power meter.

Derived series (recording rules)#

Defined in apps/monitoring/k8s.prometheusrule.energy.yaml.

6. How the values relate#

The meter is signed, so a single equation holds in both directions:

1

total_load = grid + generation        grid < 0 while exporting

From that:

1
2
3

export            = clamp_min(-grid, 0)          W
self_consumed_pv  = generation - export          W
autarky           = min(generation / total_load, 1)

And over a day:

1
2

consumption = self_consumed_pv + grid_import     kWh
pv_yield    = self_consumed_pv + export          kWh

Those three daily quantities — self-consumed PV, grid import, export — are mutually exclusive. Gross PV yield is not disjoint from export: export is a part of it, so adding yield, import and export double-counts.

7. Ready-made queries#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# current: generation, grid (signed), household load, self-sufficiency in %
home:power_generation:watts
home:power_grid_import:watts
home:power_total:watts
clamp_max(home:autarky:ratio, 1) * 100

# current export in W (0 while importing)
clamp_min(-home:power_grid_import:watts, 0)

# PV yield today, from the inverter's own daily counter
max(deye_production_today_kwh)

# exported energy since midnight, in kWh
# replace 55000 with the seconds elapsed since local midnight
sum_over_time(clamp_min(-home:power_grid_import:watts, 0)[55000s:1m]) / 60 / 1000

# per completed day — query_range with step=1d, aligned to midnight
max(max_over_time(deye_production_today_kwh[1d]))                                  # PV yield
sum(increase(fritz_ha_multimeter_energy_Wh{ain=~"16000.?0036823.*"}[1d])) / 1000   # grid import
sum_over_time(clamp_min(-home:power_grid_import:watts, 0)[1d:1m]) / 60 / 1000      # export

# per MPPT string
max by (string) (deye_dc_power_watts)

# is the data flowing?
up{job=~"mqtt2prometheus|fritzbox-exporter"}
max(deye_logger_online)

8. Pitfalls#

Each of these produced a wrong answer during development.

The grid value is signed. Treating it as import only overstates household load and makes self-sufficiency exceed 100%. Filter with clamp_min(…, 0) when you want import, clamp_min(-…, 0) when you want export.

Aggregate the infrastructure labels away. Every exporter pod restart creates a new series with a different pod/instance. A query over a 24h window then returns several series per device — one per pod that existed. Use max(...), or max by (string) (...) where a label must survive.

The deye_* series disappear at night. The logger powers down with the inverter and mqtt2prometheus evicts the series after its 5-minute cache. Use or vector(0) when a missing generation value would break an expression, and prefer the inverter's own deye_production_today_kwh over reconstructing a day from increase() on the lifetime counter.

Recording rules have no past. A rule records from the moment it is created and never rewrites earlier samples. A query over history must spell the expression out rather than rely on a recently added or recently corrected rule.

A 1d step lands on midnight. Grafana and query_range align steps to multiples of the step, so a daily series produces no point for the current, incomplete day.

fritz_ha_multimeter_energy_Wh counts import only. It never decreases, so exported energy is absent from it. Export has to be integrated from the power series, as in the queries above.