EVOLV/docs/research/dashboardapi-graph-aware-grafana-generator.md

# Research brief: graph-aware Grafana dashboard generator in dashboardAPI

_Date: 2026-05-26_
_Context: follows `/grill-me` session that locked design constraints; feeds into `/prd`._

## Questions
1. Node-RED lifecycle: how does a custom node reliably detect "deploy complete" across deploy types?
2. Prior art: existing Node-RED → Grafana auto-dashboard generators
3. Grafana HTTP API: idempotent dashboard updates by UID, version conflicts, RBAC
4. Dynamic min/max envelope pattern: dashed reference lines that vary over time
5. EVOLV-internal scaffolding already in place

## Design constraints already settled in `/grill-me`
1. dashboardAPI = dashboard **generator**, not just an InfluxDB writer.
2. One dashboardAPI instance = one Grafana dashboard. Multiple instances coexist.
3. Single source of truth: regen on Node-RED deploy **clobbers** manual Grafana edits.
4. Trigger: HTTP API push from dashboardAPI to Grafana, fired on Node-RED deploy.
5. Auth: per-flow Grafana service-account token.
6. Templates centralized in `nodes/dashboardAPI/src/templates/` per node type.
7. Per-instance `_measurement` = node name (already in `influxdbFormatter`).
8. **No data duplication** between parent and child panels (MGC shows group-level only).
9. Predicted-vs-measured = 2 panels side by side; predicted only when no measured registered.
10. Per-pump panel set: %control / flow / delta P / measured-from-children / efficiency / dashed dynamic bounds.
11. Static config bounds → **dashed reference lines** that follow the live operating envelope (top/bottom dashed + act value).

## What's already in this codebase
- **Child registration is fully graph-aware.** `ChildRegistrationUtils` keeps a `Map<id, {child, softwareType, position, registeredAt}>` with type-aware accessors `getAllChildren()`, `getChildById()`, `getChildrenOfType()`. (`nodes/generalFunctions/src/helper/childRegistrationUtils.js:19-106`)
- **dashboardAPI already iterates its children.** `extractChildren()` reads `nodeSource.childRegistrationUtils.registeredChildren.values()`. (`nodes/dashboardAPI/src/specificClass.js:151-163`)
- **Grafana upsert URL is already constructed but not yet dispatched.** `grafanaUpsertUrl()` builds the target URL — the HTTP send is missing. (`nodes/dashboardAPI/src/specificClass.js:107-110`)
- **InfluxDB schema is `measurement: nodeName`, tags from flattened config** (id, softwareType, role, positionVsParent, uuid, tagCode, geoLocation, category, type, model, unit). (`nodes/generalFunctions/src/helper/outputUtils.js:44,99-117`; `formatters/influxdbFormatter.js:12-20`)
- **Lifecycle hooks: only `node.on('close')` and `node.on('input')` are used.** No EVOLV node currently subscribes to `RED.events.on('flows:started')` or similar — net-new wiring. (`nodes/generalFunctions/src/nodered/BaseNodeAdapter.js:164,184`)
- **dashboardAPI's bearer token is stored as a plain `defaults` field, NOT as a Node-RED `credentials:` block** — so it's not encrypted at rest today. (`nodes/dashboardAPI/dashboardAPI.html:15-16`; `src/nodeClass.js:38-42`) **Contradicts the grilling assumption** that "the existing InfluxDB credentials path" is already in place — it isn't.
- **No outbound external HTTPS pattern exists anywhere in EVOLV nodes.** Net-new code path.

## External options
- **Legacy Grafana API (`POST /api/dashboards/db` with `overwrite: true`).** Skips version + uid-uniqueness checks → idempotent. Returns `412 Precondition Failed` on stale version when `overwrite=false`. Minimum RBAC: `dashboards:write` scoped to a folder. ([docs](https://grafana.com/docs/grafana/latest/developers/http_api/dashboard/))
- **Grafana 12 Kubernetes-style API (`/apis/dashboard.grafana.app/v1/...`).** Returns `409 Conflict` instead of `412`. Newer but couples integration to Grafana 12+.
- **`flows:started` runtime event** fires on every deploy (full / nodes / flows) with `{type, diff}` payload. De-dupe by inspecting `diff.added/changed/removed`. Runtime events are undocumented — must read source. (Node-RED `packages/.../runtime/lib/flows/index.js`)
- **`nodes-started` event is deprecated** — use `flows:started`.
- **Dashed-line dynamic bands:** the *only* path that works today is emitting min/max as separate Influx fields + applying `fieldConfig.overrides[].properties[].id = "custom.lineStyle"` with `{fill: "dash", dash: [10,10]}`. Per-series override via `byName` matcher.
- **Grafana thresholds are static-only** (open issue [grafana/grafana#115398](https://github.com/grafana/grafana/issues/115398) — Needs Prioritisation). Dead end for time-varying bands.

## Prior art
- **No relevant prior art found.** Every "node-red + grafana" tutorial puts Influx in the middle and hand-builds dashboards. No npm package pushes Grafana dashboards from Node-RED. Greenfield lane.
- **Grafana Foundation SDK / dashboards-as-code** ([docs](https://grafana.com/docs/grafana/latest/as-code/observability-as-code/foundation-sdk/)) — assumes out-of-band CI generation, not a live Node-RED instance.
- **Operating-envelope plotting in Grafana** — [community thread 57225](https://community.grafana.com/t/how-to-plot-graph-using-upper-and-lower-bound/57225) asks the exact question, no accepted answer.
- **Known Grafana bugs around `custom.lineStyle`:** [#75259](https://github.com/grafana/grafana/issues/75259) (transforms) and [#86546](https://github.com/grafana/grafana/issues/86546) (overlapping dashed → solid).

## Open unknowns
- **(O-1) `flows:started` + `diff` reliability.** Does `diff` cleanly distinguish "this dashboardAPI's flow changed" from "an unrelated flow changed" across all three deploy modes? Source-readable but needs an actual spike to verify edge cases (e.g. a `Modified Nodes` deploy that adds a child measurement to a pumpingStation registered to a dashboardAPI in a different tab). → **Candidate for `/prototype`.**
- **(O-2) Dashed-line rendering against real Influx series.** Two open Grafana bugs ([#75259](https://github.com/grafana/grafana/issues/75259), [#86546](https://github.com/grafana/grafana/issues/86546)) affect `custom.lineStyle`. Untested whether either bites with EVOLV's emission pattern. → **Candidate for `/prototype`.**
- **(O-3) Legacy `/api/dashboards/db` vs v12 K8s API.** Which to commit to? Locks integration to a Grafana version family. Local stack uses `grafana/grafana:latest` — version drifts on `docker compose pull`. → PRD-time decision; pin Grafana image.
- **(O-4) Bearer-token storage migration.** Assumption that "follow existing creds pattern" doesn't hold — dashboardAPI stores it as plain config today. Need to migrate to Node-RED `credentials:` block. Risk: token currently sitting in `flow.json` of users' existing flows. → PRD-time decision; migration step in first issue.

## Recommended next step
`/prd` — commit the design, resolve O-3 and O-4 explicitly, and queue O-1 and O-2 for `/prototype` before the first issue ships.