RnD/EVOLV
3
0
Fork 0
Code Issues 22 Pull Requests Actions Packages Projects Releases Wiki Activity

wiki: crisp overhaul — no decoration emoji, all 9 master pages refactored

Browse Source 
Source-tree mirror of EVOLV.wiki.git refactor (27a42ee on wiki.git):

- 7 master pages rewritten with clean design (Home, Architecture,
  Topology-Patterns, Topic-Conventions, Telemetry, Getting-Started,
  Glossary). Tables and Mermaid for visuals, gitea alert callouts for
  warnings, shields badges for metadata only. No emoji as decoration.
- Archive.md becomes a removal-changelog pointing readers to git
  history and to the successor pages.
- _Sidebar.md updated to navigate the new flat-name layout.
- Concept / finding / manual pages: uniform mini-header (badges +
  "reference page" callout) added without rewriting domain content.
- Every internal link now uses the flat naming that resolves on the
  live gitea wiki (Concept-ASM-Models, Finding-BEP-..., etc.).

On wiki.git: 29 Archive-* pages hard-deleted (the git history
preserves them; Archive.md documents the removal).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
znetsixe
2026-05-11 22:24:51 +02:00
parent 2ccc8aea9e
commit 5ae8788fd7
33 changed files with 1491 additions and 729 deletions
Download Patch File Download Diff File Expand all files Collapse all files

235
wiki/Telemetry.md
View File

@@ -1,46 +1,68 @@
# Telemetry
> **Reflects code as of `9ab9f6b` · regenerated `2026-05-11`**
![code-ref](https://img.shields.io/badge/code--ref-9ab9f6b-blue)
![source](https://img.shields.io/badge/source-CONTRACTS.md_%C2%A710-orange)
What every EVOLV node emits on each of its three output ports, the InfluxDB line-protocol layout, and how the data reaches Grafana/FlowFuse.
> [!NOTE]
> Every node sends on three output ports: Port 0 (process data), Port 1 (InfluxDB line protocol), Port 2 (registration / control plumbing). All output is formatted by `outputUtils.formatMsg` with delta compression: only changed fields are sent each tick. InfluxDB cardinality discipline: tags = identity (low cardinality), fields = numbers. Source of truth: `.claude/refactor/CONTRACTS.md` §10.
---
## Three-port model
```mermaid
flowchart LR
sc[specificClass<br/>tick or event]:::tier3
nc[nodeClass<br/>outputUtils.formatMsg]:::tier2
p0[(Port 0<br/>process)]:::p0
p1[(Port 1<br/>InfluxDB line)]:::p1
p2[(Port 2<br/>registration)]:::p2
sc["specificClass &mdash; 'output-changed' or tick()"]
ou["outputUtils.formatMsg &mdash; delta-compress"]
p0[("Port 0 &mdash; process")]
p1[("Port 1 &mdash; InfluxDB line")]
p2[("Port 2 &mdash; register / control")]
dl["Downstream Node-RED &mdash; dashboards, functions"]
influx[("InfluxDB")]
parent["Parent EVOLV node"]
sc -->|getOutput| nc
nc --> p0
nc --> p1
nc --> p2
sc -- getOutput() --> ou
ou --> p0 --> dl
ou --> p1 --> influx
sc -. child.register .-> p2 --> parent
p0 -. delta-compressed payload .-> dl[Downstream<br/>Node-RED logic]:::neutral
p1 -. line protocol .-> influx[(InfluxDB)]:::ext
p2 -. child.register .-> parent[Parent EVOLV node]:::neutral
class sc tier3
class ou tier2
class p0 p0c
class p1 p1c
class p2 p2c
class dl,parent dn
class influx ext
classDef tier3 fill:#50a8d9,color:#000
classDef tier2 fill:#86bbdd,color:#000
classDef p0 fill:#86bbdd
classDef p1 fill:#a9daee
classDef p2 fill:#dddddd
classDef neutral fill:#dddddd
classDef ext fill:#fff2cc
classDef p0c fill:#0c99d9,color:#fff
classDef p1c fill:#50a8d9,color:#000
classDef p2c fill:#a9daee,color:#000
classDef dn fill:#dddddd,color:#000
classDef ext fill:#fff2cc,color:#000
```
## Port 0 — Process data (delta-compressed)
### Port summary
**Purpose:** feeds downstream Node-RED logic — dashboards, control functions, alarms.
| Port | Carries | Format | Configured by | Trigger |
|:---|:---|:---|:---|:---|
| 0 (process) | Snapshot of changed measurement / state keys | JSON delta object | `outputUtils.formatMsg(..., 'process')` via `config.output.process` | `'output-changed'` on the emitter, or each tick |
| 1 (telemetry) | Numeric fields only &mdash; time-series | InfluxDB line protocol | `outputUtils.formatMsg(..., 'influxdb')` via `config.output.dbase` | Same trigger as Port 0 |
| 2 (register) | `child.register` plus internal control msgs | Plain object | `BaseNodeAdapter` init | Once 100ms after init; on demand |
**Shape:** `msg.payload` is an object containing **only keys that changed** since the last tick. Consumers cache + merge.
---
**Why delta-compressed:** at 1 Hz ticks with 50 fields per node, full snapshots flood downstream nodes and dashboards. Delta payloads typically carry 0–5 fields per tick.
## Port 0 &mdash; Process data (delta-compressed)
**Example (rotatingMachine, 1 tick):**
Purpose: feed downstream Node-RED logic such as dashboards, alarms, control function nodes.
Shape: `msg.payload` contains only keys that changed since the last tick. Consumers must cache and merge.
> [!IMPORTANT]
> Why delta compression: at 1 Hz with 50+ fields per node, full snapshots flood downstream nodes and dashboards. A typical delta carries 0&ndash;5 fields per tick.
Example output, one rotatingMachine tick:
```json
{
@@ -52,151 +74,192 @@ flowchart LR
}
```
Most other fields (state, pressure, mode, …) didn't change this tick — omitted.
Most other fields (state, pressure, mode, ...) didn't change this tick and are omitted.
**Trigger:** `outputUtils.checkForChanges()` compares the current `getOutput()` against the previous snapshot.
Trigger: `outputUtils.checkForChanges()` compares the current `getOutput()` against the previous snapshot. No change means no send.
## Port 1 — Telemetry (InfluxDB line protocol)
---
**Purpose:** time-series storage in InfluxDB for trending, regulatory reporting, ML training.
## Port 1 &mdash; InfluxDB line protocol
**Shape:** `msg.payload` is a **string** (or array of strings) in InfluxDB line protocol:
Purpose: time-series storage for trending, regulatory reporting, ML training.
Shape: `msg.payload` is a string (or array of strings) in InfluxDB line protocol:
```
<measurement>,<tag-set> <field-set> <timestamp-ns>
```
**Example:**
Example, rotatingMachine telemetry:
```
rotatingMachine,id=pump-A,softwareType=rotatingMachine flow_predicted_downstream=12.4,power_measured_atequipment=18.2 1714752000000000000
```
**Conventions:**
### Conventions
| Element | Rule |
|---|---|
| measurement (table) | The node's `softwareType` (lowercase). |
| tag-set | Low-cardinality identity: `id`, `softwareType`, location-style tags. **Never** raw measurement values. |
| field-set | Numeric values only. Keys flatten `<type>_<variant>_<position>` (underscore, not dot — InfluxDB constraint). |
| timestamp | Nanoseconds. Set by `outputUtils` from the node's clock. |
|:---|:---|
| Measurement (table name) | The node's `softwareType`, lowercase |
| Tag set | Low-cardinality identity: `id`, `softwareType`, location-style tags. Never raw measurement values |
| Field set | Numeric values only. Keys flatten `<type>_<variant>_<position>` (underscore, not dot &mdash; InfluxDB constraint) |
| Timestamp | Nanoseconds. Set by `outputUtils` from the node's clock |
See [InfluxDB schema design](concepts/influxdb-schema-design) for the cardinality discipline.
See [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) for cardinality discipline.
## Port 2 — Registration / control
---
**Purpose:** upward `child.register` at startup; later, internal control msgs (the registry-driven command replies).
## Port 2 &mdash; Registration / control
**Shape:**
Purpose: upward `child.register` at startup; later, internal control msgs.
Shape:
```json
{
"topic": "child.register",
"payload": {
"ref": <node reference>,
"ref": "<node reference>",
"softwareType": "machine",
"config": { ... }
"config": { }
}
}
```
**Trigger:** the nodeClass adapter emits `child.register` on `init` if a `parent` is configured. The parent's `commandRegistry` dispatches into `ChildRouter.onRegister(...)`.
Trigger: the nodeClass adapter emits `child.register` on init if a parent is configured. Parent's `commandRegistry` dispatches into `ChildRouter.onRegister(...)`.
The legacy alias `registerChild` is still accepted; it logs a one-time deprecation warning on first use.
---
## The output composition pipeline
```mermaid
sequenceDiagram
participant tick as Tick (1 Hz)
autonumber
participant tick as Tick (1 Hz) or event source
participant sc as specificClass
participant mc as MeasurementContainer
participant ou as outputUtils
participant ports as Ports 0 / 1
tick->>sc: tick()
tick->>sc: tick() OR emit('output-changed')
sc->>sc: concern modules update mc + state
sc->>ou: getOutput() snapshot
ou->>ou: diff vs last
ou->>ou: diff vs last snapshot
alt no change
ou-->>sc: skip
else change
ou->>ports: Port 0 — JSON delta
ou->>ports: Port 1 — line protocol
ou->>ports: Port 0 &mdash; JSON delta
ou->>ports: Port 1 &mdash; line protocol
end
```
`outputUtils` is the single place the platform serialises state. Never write directly to `node.send` from specificClass — go through `outputUtils.formatMsg`.
> [!CAUTION]
> Never write directly to `node.send` from specificClass. Go through `outputUtils.formatMsg`. Direct sends bypass delta compression and flood downstream nodes.
## InfluxDB layout
---
Recommended schema for EVOLV's data:
## InfluxDB schema layout (recommended)
| InfluxDB element | Maps to |
|---|---|
| Database / bucket | One per plant (or per environment: `evolv_dev`, `evolv_prod`). |
| Measurement (table) | Node softwareType (`rotatingMachine`, `pumpingStation`, …). |
| Tags | `id` (instance id), `softwareType`, `area`, `processCell`, `unit` (for hierarchical drill-down). |
| Fields | Numeric series — every key from `getOutput()` that has a numeric value, flattened with `_`. |
| Retention | Hot bucket: 7 days @ 1 s. Cold bucket: 1 year @ 1 min downsample. |
|:---|:---|
| Database / bucket | One per plant, or per environment: `evolv_dev`, `evolv_prod` |
| Measurement (table) | Node `softwareType` |
| Tags | `id`, `softwareType`, `area`, `processCell`, `unit` (for hierarchical drill-down) |
| Fields | Numeric series &mdash; every numeric key from `getOutput()`, flattened with `_` |
| Retention &mdash; hot | 7 days at 1 s |
| Retention &mdash; cold | 1 year at 1 min downsample |
**Cardinality discipline:** keep tag sets stable. Don't put `state` (string) as a tag — emit it as a field with code (`state_code=2`). High-cardinality tags fragment the index.
> [!WARNING]
> Cardinality discipline. Keep tag sets stable. Do not put `state` (string) as a tag &mdash; emit it as a field with a code (`state_code=2`). High-cardinality tags fragment InfluxDB's index and degrade query performance dramatically.
See [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) for full guidance.
---
## FlowFuse dashboard wiring
If you use FlowFuse `ui-chart` widgets, Port 0 is the natural source — the delta-compressed JSON maps cleanly to `msg.topic` (series label) + `msg.payload` (y-value).
Port 0 is the natural source for FlowFuse `ui-chart` widgets &mdash; the delta-compressed JSON maps cleanly to `msg.topic` (series label) plus `msg.payload` (y-value).
Layout rule (from `.claude/rules/node-red-flow-layout.md` §4):
Layout rule for charts (from `.claude/rules/node-red-flow-layout.md` §4):
- One chart per metric type (one for flow, one for power, one for level).
- A trend-feeder function splits Port-0 deltas into per-series outputs, each output wired to one chart.
- The chart's `category: "topic"` + `categoryType: "msg"` plots one series per unique `msg.topic`.
- A trend-feeder function splits Port-0 deltas into per-series outputs.
- Each output wires to one chart. The chart's `category: "topic"` and `categoryType: "msg"` plot one series per unique `msg.topic`.
```mermaid
flowchart LR
p0[(Port 0)]:::p0
split[trend-feeder<br/>function (N outputs)]:::tier2
chart1[ui-chart: flow]:::neutral
chart2[ui-chart: power]:::neutral
p0[("Port 0")]
split["trend-feeder &mdash; function (N outputs)"]
chart1["ui-chart: flow"]
chart2["ui-chart: power"]
p0 --> split
split --> chart1
split --> chart2
classDef p0 fill:#86bbdd
class p0 p0c
class split tier2
class chart1,chart2 neutral
classDef p0c fill:#0c99d9,color:#fff
classDef tier2 fill:#86bbdd,color:#000
classDef neutral fill:#dddddd
```
See [FlowFuse ui-chart manual](Manual-NodeRED-Flowfuse-Ui-Chart-Manual) for the required chart properties.
---
## Grafana dashboard provisioning
`dashboardAPI` consumes registrations and emits Grafana dashboard JSON via HTTP. Wiring:
`dashboardAPI` consumes registrations and emits Grafana dashboard JSON via HTTP.
```mermaid
flowchart LR
evolv[EVOLV node<br/>any softwareType]:::tier3
dash[dashboardAPI]:::util
grafana[(Grafana HTTP API<br/>POST /api/dashboards/db)]:::ext
evolv["EVOLV node (any softwareType)"]
dash[dashboardAPI]
grafana[("Grafana HTTP API &mdash; POST /api/dashboards/db")]
evolv -->|child.register| dash
dash -->|composed JSON| grafana
evolv -- child.register --> dash
dash -- composed JSON --> grafana
class evolv tier3
class dash util
class grafana ext
classDef tier3 fill:#50a8d9,color:#000
classDef util fill:#dddddd
classDef ext fill:#fff2cc
classDef ext fill:#fff2cc,color:#000
```
dashboardAPI looks up a template per softwareType (in `nodes/dashboardAPI/src/config/templates/`), substitutes the node's id + tags, and POSTs an upsert. Bearer-token auth is supported.
dashboardAPI looks up a template per softwareType (in `nodes/dashboardAPI/src/config/templates/`), substitutes the node's id + tags, and POSTs an upsert. Bearer-token auth is supported via `config.grafanaConnector.bearerToken`.
## Common debug recipes
---
| Symptom | First check |
|---|---|
| InfluxDB rows missing for a node | Confirm Port 1 is wired to an `influxdb out` node. Tap Port 1 with a debug node to verify line-protocol output. |
| Dashboard widgets stuck on `n/a` | Confirm Port 0 is reaching the trend-feeder. Many widgets need `msg.topic` set for series labelling. |
| `child.register` not arriving at parent | Tap Port 2 with debug. Confirm parent's `commandRegistry` accepts `child.register` (or the legacy `registerChild` alias). |
| Too many InfluxDB writes (high write-rate) | Check that `outputUtils.checkForChanges()` is firing. Likely you wired a tick-driven debug node bypassing the delta filter. |
| Grafana dashboard not created on plant boot | Inspect dashboardAPI's HTTP response. Check the bearer token + base URL in its config. |
## Debug recipes
| Symptom | First thing to check |
|:---|:---|
| InfluxDB rows missing for a node | Port 1 wired to an `influxdb out` node? Tap Port 1 with a debug node to verify line-protocol output |
| Dashboard widgets stuck on `n/a` | Port 0 reaching the trend-feeder? Many widgets need `msg.topic` set for series labelling |
| `child.register` not arriving | Tap Port 2 with debug. Confirm parent's `commandRegistry` accepts `child.register` (or `registerChild` alias) |
| Too many InfluxDB writes | Likely a tick-driven debug node bypassed the delta filter. Confirm `outputUtils.checkForChanges()` is firing |
| Grafana dashboard not created on boot | Inspect dashboardAPI's HTTP response. Check bearer token + base URL in its config |
| High cardinality alarm in InfluxDB | A string value is being written as a tag (probably `state` or similar). Move it to a field |
> [!CAUTION]
> Never ship `enableLog: 'debug'` in a demo. Fills the container log within seconds and obscures real errors. Use only for live debugging.
---
## Related pages
- [Architecture](Architecture) — output port wiring in the 3-tier code
- [Topic-Conventions](Topic-Conventions) — what topics map to what fields
- [InfluxDB schema design](concepts/influxdb-schema-design) — cardinality discipline
| Page | Why |
|:---|:---|
| [Architecture](Architecture) | Output port wiring in the three-tier model |
| [Topic Conventions](Topic-Conventions) | What topics map to what fields |
| [Topology Patterns](Topology-Patterns) | Typical telemetry flows |
| [InfluxDB Schema Design](Concept-InfluxDB-Schema-Design) | Cardinality discipline |
| [FlowFuse ui-chart manual](Manual-NodeRED-Flowfuse-Ui-Chart-Manual) | Required chart properties |