EVOLV/docs/research/dashboardapi-graph-aware-grafana-generator.md

Research brief: graph-aware Grafana dashboard generator in dashboardAPI

Date: 2026-05-26 Context: follows /grill-me session that locked design constraints; feeds into /prd.

Questions

1. Node-RED lifecycle: how does a custom node reliably detect "deploy complete" across deploy types?
2. Prior art: existing Node-RED → Grafana auto-dashboard generators
3. Grafana HTTP API: idempotent dashboard updates by UID, version conflicts, RBAC
4. Dynamic min/max envelope pattern: dashed reference lines that vary over time
5. EVOLV-internal scaffolding already in place

Design constraints already settled in /grill-me

1. dashboardAPI = dashboard generator, not just an InfluxDB writer.
2. One dashboardAPI instance = one Grafana dashboard. Multiple instances coexist.
3. Single source of truth: regen on Node-RED deploy clobbers manual Grafana edits.
4. Trigger: HTTP API push from dashboardAPI to Grafana, fired on Node-RED deploy.
5. Auth: per-flow Grafana service-account token.
6. Templates centralized in nodes/dashboardAPI/src/templates/ per node type.
7. Per-instance _measurement = node name (already in influxdbFormatter).
8. No data duplication between parent and child panels (MGC shows group-level only).
9. Predicted-vs-measured = 2 panels side by side; predicted only when no measured registered.
10. Per-pump panel set: %control / flow / delta P / measured-from-children / efficiency / dashed dynamic bounds.
11. Static config bounds → dashed reference lines that follow the live operating envelope (top/bottom dashed + act value).

What's already in this codebase

• Child registration is fully graph-aware. ChildRegistrationUtils keeps a Map<id, {child, softwareType, position, registeredAt}> with type-aware accessors getAllChildren(), getChildById(), getChildrenOfType(). (nodes/generalFunctions/src/helper/childRegistrationUtils.js:19-106)
• dashboardAPI already iterates its children. extractChildren() reads nodeSource.childRegistrationUtils.registeredChildren.values(). (nodes/dashboardAPI/src/specificClass.js:151-163)
• Grafana upsert URL is already constructed but not yet dispatched. grafanaUpsertUrl() builds the target URL — the HTTP send is missing. (nodes/dashboardAPI/src/specificClass.js:107-110)
• InfluxDB schema is measurement: nodeName, tags from flattened config (id, softwareType, role, positionVsParent, uuid, tagCode, geoLocation, category, type, model, unit). (nodes/generalFunctions/src/helper/outputUtils.js:44,99-117; formatters/influxdbFormatter.js:12-20)
• Lifecycle hooks: only node.on('close') and node.on('input') are used. No EVOLV node currently subscribes to RED.events.on('flows:started') or similar — net-new wiring. (nodes/generalFunctions/src/nodered/BaseNodeAdapter.js:164,184)
• dashboardAPI's bearer token is stored as a plain defaults field, NOT as a Node-RED credentials: block — so it's not encrypted at rest today. (nodes/dashboardAPI/dashboardAPI.html:15-16; src/nodeClass.js:38-42) Contradicts the grilling assumption that "the existing InfluxDB credentials path" is already in place — it isn't.
• No outbound external HTTPS pattern exists anywhere in EVOLV nodes. Net-new code path.

External options

• Legacy Grafana API (POST /api/dashboards/db with overwrite: true). Skips version + uid-uniqueness checks → idempotent. Returns 412 Precondition Failed on stale version when overwrite=false. Minimum RBAC: dashboards:write scoped to a folder. (docs)
• Grafana 12 Kubernetes-style API (/apis/dashboard.grafana.app/v1/...). Returns 409 Conflict instead of 412. Newer but couples integration to Grafana 12+.
• flows:started runtime event fires on every deploy (full / nodes / flows) with {type, diff} payload. De-dupe by inspecting diff.added/changed/removed. Runtime events are undocumented — must read source. (Node-RED packages/.../runtime/lib/flows/index.js)
• nodes-started event is deprecated — use flows:started.
• Dashed-line dynamic bands: the only path that works today is emitting min/max as separate Influx fields + applying fieldConfig.overrides[].properties[].id = "custom.lineStyle" with {fill: "dash", dash: [10,10]}. Per-series override via byName matcher.
• Grafana thresholds are static-only (open issue grafana/grafana#115398 — Needs Prioritisation). Dead end for time-varying bands.

Prior art

• No relevant prior art found. Every "node-red + grafana" tutorial puts Influx in the middle and hand-builds dashboards. No npm package pushes Grafana dashboards from Node-RED. Greenfield lane.
• Grafana Foundation SDK / dashboards-as-code (docs) — assumes out-of-band CI generation, not a live Node-RED instance.
• Operating-envelope plotting in Grafana — community thread 57225 asks the exact question, no accepted answer.
• Known Grafana bugs around custom.lineStyle: #75259 (transforms) and #86546 (overlapping dashed → solid).

Open unknowns

• (O-1) flows:started + diff reliability. Does diff cleanly distinguish "this dashboardAPI's flow changed" from "an unrelated flow changed" across all three deploy modes? Source-readable but needs an actual spike to verify edge cases (e.g. a Modified Nodes deploy that adds a child measurement to a pumpingStation registered to a dashboardAPI in a different tab). → Candidate for /prototype.
• (O-2) Dashed-line rendering against real Influx series. Two open Grafana bugs (#75259, #86546) affect custom.lineStyle. Untested whether either bites with EVOLV's emission pattern. → Candidate for /prototype.
• (O-3) Legacy /api/dashboards/db vs v12 K8s API. Which to commit to? Locks integration to a Grafana version family. Local stack uses grafana/grafana:latest — version drifts on docker compose pull. → PRD-time decision; pin Grafana image.
• (O-4) Bearer-token storage migration. Assumption that "follow existing creds pattern" doesn't hold — dashboardAPI stores it as plain config today. Need to migrate to Node-RED credentials: block. Risk: token currently sitting in flow.json of users' existing flows. → PRD-time decision; migration step in first issue.

Recommended next step

/prd — commit the design, resolve O-3 and O-4 explicitly, and queue O-1 and O-2 for /prototype before the first issue ships.