EVOLV/.claude/rules/node-red-flow-layout.md

# Node-RED Flow Layout Rules

How to lay out a multi-tab Node-RED demo or production flow so it is readable, debuggable, and trivially extendable. These rules apply to anything you build with `examples/` flows, dashboards, or production deployments.

## 1. Tab boundaries — by CONCERN, not by data

Every node lives on the tab matching its **concern**, never where it happens to be wired:

| Tab | Lives here | Never here |
|---|---|---|
| **🏭 Process Plant** | EVOLV nodes (rotatingMachine, MGC, pumpingStation, measurement, reactor, settler, …) + small per-node output formatters | UI widgets, demo drivers, one-shot setup injects |
| **📊 Dashboard UI** | All `ui-*` widgets, the wrapper functions that turn a button click into a typed `msg`, the trend-feeder split functions | Anything that produces data autonomously, anything that talks to EVOLV nodes directly |
| **🎛️ Demo Drivers** | Random generators, scripted scenarios, schedule injectors, anything that exists only to drive the demo | Real production data sources (those go on Process Plant or are wired in externally) |
| **⚙️ Setup & Init** | One-shot `once: true` injects (setMode, setScaling, auto-startup) | Anything that fires more than once |

**Why these four:** each tab can be disabled or deleted independently. Disable Demo Drivers → demo becomes inert until a real data source is wired. Disable Setup → fresh deploys don't auto-configure (good for debugging). Disable Dashboard UI → headless mode for tests. Process Plant always stays.

If you find yourself wanting a node "between" two tabs, you've named your concerns wrong — re-split.

## 2. Cross-tab wiring — link nodes only, named channels

Never wire a node on tab A directly to a node on tab B. Use **named link-out / link-in pairs**:

```text
[ui-slider] ──► [link out  cmd:demand] ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
                                                                    │
                                                                    ▼
[random gen] ─► [link out  cmd:demand] ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─►  [link in  cmd:demand] ──► [router] ──► [MGC]
                                                                    ▲
                                                                    │
                                  many link-outs may target one link-in
```

### Naming convention

Channels follow `<direction>:<topic>` lowercase, kebab-case after the colon:

- `cmd:` — UI / drivers → process. Carries commands.
- `evt:` — process → UI / external. Carries state events.
- `setup:` — setup tab → wherever. Carries one-shot init.

Examples used in the pumping-station demo:
- `cmd:demand`, `cmd:randomToggle`, `cmd:mode`
- `cmd:station-startup`, `cmd:station-shutdown`, `cmd:station-estop`
- `cmd:setpoint-A`, `cmd:setpoint-B`, `cmd:setpoint-C`
- `cmd:pump-A-seq` (start/stop for pump A specifically)
- `evt:pump-A`, `evt:pump-B`, `evt:pump-C`, `evt:mgc`, `evt:ps`
- `setup:to-mgc`

### Channels are the contract

The list of channel names IS the inter-tab API. Document it in the demo's README. Renaming a channel is a breaking change.

### When to use one channel vs many

- One channel, many emitters: same kind of message from multiple sources (e.g. `cmd:demand` is fired by both the slider and the random generator).
- Different channels: messages with different *meaning* even if they go to the same node (e.g. don't fold `cmd:setpoint-A` into a generic `cmd:pump-A` — keep setpoint and start/stop separate).
- Avoid one mega-channel: a "process commands" channel that the receiver routes-by-topic is harder to read than separate channels per concern.

### Don't use link-call for fan-out

`link call` is for synchronous request/response (waits for a paired `link out` in `return` mode). For fan-out, use plain `link out` (mode=`link`) with multiple targets, or a single link out → single link in → function-node fan-out (whichever is clearer for your case).

## 3. Spacing and visual layout

Nodes need air to be readable. Apply these constants in any flow generator:

```python
LANE_X = [120, 380, 640, 900, 1160, 1420]   # 6 vertical lanes per tab
ROW = 80                                     # standard row pitch
SECTION_GAP = 200                            # extra y-shift between sections
```

### Lane assignment (process plant tab as example)

| Lane | Contents |
|---|---|
| 0 (x=120) | Inputs from outside the tab — link-in nodes, injects |
| 1 (x=380) | First-level transformers — wrappers, fan-outs, routers |
| 2 (x=640) | Mid-level — section comments live here too |
| 3 (x=900) | Target nodes — the EVOLV node itself (pump, MGC, PS) |
| 4 (x=1160) | Output formatters — function nodes that build dashboard-friendly payloads |
| 5 (x=1420) | Outputs to outside the tab — link-out nodes, debug taps |

Inputs flow left → right. Don't loop wires backwards across the tab.

### Section comments

Every logical group within a tab gets a comment header at lane 2 with a `── Section name ──` style label. Use them liberally — every 3-5 nodes deserves a header. The `info` field on the comment carries the multi-line description.

### Section spacing

`SECTION_GAP = 200` between sections, on top of the standard row pitch. Don't pack sections together — when you have 6 measurements on a tab, give each pump 4 rows + a 200 px gap to the next pump. Yes, it makes tabs scroll. Scroll is cheap; visual confusion is expensive.

## 4. Charts — the trend-split rule

ui-chart with `category: "topic"` + `categoryType: "msg"` plots one series per unique `msg.topic`. So:

- One chart per **metric type** (one chart for flow, one for power).
- Each chart receives msgs whose `topic` is the **series label** (e.g. `Pump A`, `Pump B`, `Pump C`).

### The trend-split function pattern

A common bug: feeding both flow and power msgs to a single function output that wires to both charts. Both charts then plot all metrics, garbling the legend.

**Fix:** the trend-feeder function MUST have one output per chart, and split:

```js
// outputs: 2
// wires: [["chart_flow"], ["chart_power"]]
const flowMsg  = p.flowNum  != null ? { topic: 'Pump A', payload: p.flowNum  } : null;
const powerMsg = p.powerNum != null ? { topic: 'Pump A', payload: p.powerNum } : null;
return [flowMsg, powerMsg];
```

A null msg on a given output sends nothing on that output — exactly what we want.

### Chart axis settings to actually configure

- `removeOlder` + `removeOlderUnit`: how much history to keep (e.g. 10 minutes).
- `removeOlderPoints`: cap on points per series (200 is sensible for a demo).
- `ymin` / `ymax`: leave blank for autoscale, or set numeric strings if you want a fixed range.

## 5. Inject node — payload typing

Multi-prop inject must populate `v` and `vt` **per prop**, not just the legacy top-level `payload` + `payloadType`:

```json
{
  "props": [
    {"p": "topic", "vt": "str"},
    {"p": "payload", "v": "{\"action\":\"startup\"}", "vt": "json"}
  ],
  "topic": "execSequence",
  "payload": "{\"action\":\"startup\"}",
  "payloadType": "json"
}
```

If you only fill the top-level fields, `payload_type=json` is silently treated as `str`.

## 6. Dashboard widget rules

- **Widget = display only.** No business logic in `ui-text` formats or `ui-template` HTML.
- **Buttons emit a typed string payload** (`"fired"` or similar). Convert to the real msg shape with a tiny wrapper function on the same tab, before the link-out.
- **Sliders use `passthru: true`** so they re-emit on input messages (useful for syncing initial state from the process side later).
- **One ui-page per demo.** Multiple groups under one page is the natural split.
- **Group widths should sum to a multiple of 12.** The page grid is 12 columns. A row of `4 + 4 + 4` or `6 + 6` works; mixing arbitrary widths leaves gaps.
- **EVERY ui-* node needs `x` and `y` keys.** Without them Node-RED dumps the node at (0,0) — every text widget and chart piles up in the top-left of the editor canvas. The dashboard itself still renders correctly (it lays out by group/order, not editor x/y), but the editor view is unreadable. If you write a flow generator helper, set `x` and `y` on the dict EVERY time. Test with `jq '[.[] | select(.x==0 and .y==0 and (.type|tostring|startswith("ui-")))]'` after generating.

## 7. Do / don't checklist

✅ Do:

- Generate flows from a Python builder (`build_flow.py`) — it's the source of truth.
- Use deterministic IDs (`pump_a`, `meas_pump_a_u`, `lin_demand_to_mgc`) — reproducible diffs across regenerations.
- Tag every channel name with `cmd:` / `evt:` / `setup:`.
- Comment every section, even short ones.
- Verify trends with a `ui-chart` of synthetic data first, before plumbing real data through.

❌ Don't:

- Don't use `replace_all` on a Python identifier that appears in a node's own wires definition — you'll create self-loops (>250k msg/s discovered the hard way).
- Don't wire across tabs directly. The wire IS allowed but it makes the editor unreadable.
- Don't put dashboard widgets next to EVOLV nodes — different concerns.
- Don't pack nodes within 40 px of each other — labels overlap, wires snap to wrong handles.
- Don't ship `enableLog: "debug"` in a demo — fills the container log within seconds and obscures real errors.

## 8. The link-out / link-in JSON shape (cheat sheet)

```json
{
  "id": "lout_demand_dash",
  "type": "link out",
  "z": "tab_ui",
  "name": "cmd:demand",
  "mode": "link",
  "links": ["lin_demand_to_mgc"],
  "x": 380, "y": 140,
  "wires": []
}
```

```json
{
  "id": "lin_demand_to_mgc",
  "type": "link in",
  "z": "tab_process",
  "name": "cmd:demand",
  "links": ["lout_demand_dash", "lout_demand_drivers"],
  "x": 120, "y": 1500,
  "wires": [["demand_fanout_mgc_ps"]]
}
```

Both ends store the paired ids in `links`. The `name` is cosmetic (label only) — Node-RED routes by id. Multiple emitters can target one receiver; one emitter can target multiple receivers.

## 9. Verifying the layout

Before declaring a flow done:

1. **Open the tab in the editor — every wire should run left → right.** No backward loops.
2. **Open each section by section comment — visible in 1 screen height.** If not, raise `SECTION_GAP`.
3. **Hit the dashboard URL — every widget has data.** `n/a` everywhere is a contract failure.
4. **For charts, watch a series populate over 30 s.** A blank chart after 30 s = bug.
5. **Disable each tab one at a time and re-deploy.** Process Plant alone should still load (just inert). Dashboard UI alone should serve a page (just empty). If disabling a tab errors out, the tab boundaries are wrong.

## 10. Hierarchical placement — by S88 level, not by node name

The lane assignment maps to the **S88 hierarchy**, not to specific node names. Any node that lives at a given S88 level goes in the same lane regardless of what kind of equipment it is. New node types added to the platform inherit a lane by their S88 category — no rule change needed.

### 10.1 Lane convention (x-axis = S88 level)

| Lane | x | Purpose | S88 level | Colour | Current EVOLV nodes |
|---:|---:|---|---|---|---|
| **L0** | 120 | Tab inputs | — | (none) | `link in`, `inject` |
| **L1** | 360 | Adapters | — | (none) | `function` (msg-shape wrappers) |
| **L2** | 600 | Control Module | CM | `#a9daee` | `measurement` |
| **L3** | 840 | Equipment Module | EM | `#86bbdd` | `rotatingMachine`, `valve`, `diffuser` |
| **L4** | 1080 | Unit | UN | `#50a8d9` | `machineGroupControl`, `valveGroupControl`, `reactor`, `settler`, `monster` |
| **L5** | 1320 | Process Cell | PC | `#0c99d9` | `pumpingStation` |
| **L6** | 1560 | Output formatters | — | (none) | `function` (build dashboard payload from port 0) |
| **L7** | 1800 | Tab outputs | — | (none) | `link out`, `debug` |

Spacing: **240 px** between lanes. Tab width ≤ 1920 px (fits standard monitors without horizontal scroll in the editor).

**Area level** (`#0f52a5`) is reserved for plant-wide coordination and currently unused — when added, allocate a new lane and shift formatter/output one lane right (i.e. expand to 9 lanes if and when needed).

### 10.2 The group rule (Node-RED `group` boxes anchor each parent + its children)

Use Node-RED's native `group` node (the visual box around a set of nodes — not to be confused with `ui-group`) to anchor every "parent + direct children" cluster. The box makes ownership unambiguous and lets you collapse the cluster in the editor.

**Group rules:**

- **One Node-RED group per parent + its direct children.**
  Example: `Pump A + meas-A-up + meas-A-dn` is one group, named `Pump A`.
- **Group colour = parent's S88 colour.**
  So a Pump-A group is `#86bbdd` (Equipment Module). A reactor group is `#50a8d9` (Unit).
- **Group `style.label = true`** so the box shows the parent's name.
- **Group must contain all the children's adapters / wrappers / formatters** too if those exclusively belong to the parent. The box is the visual anchor for "this is everything that owns / serves Pump A".
- **Utility groups for cross-cutting logic** (mode broadcast, station-wide commands, demand fan-out) use a neutral colour (`#dddddd`).

JSON shape:

```json
{
  "id": "grp_pump_a",
  "type": "group",
  "z": "tab_process",
  "name": "Pump A",
  "style": { "label": true, "stroke": "#000000", "fill": "#86bbdd", "fill-opacity": "0.10" },
  "nodes": ["meas_pump_a_u", "meas_pump_a_d", "pump_a", "format_pump_a", "lin_setpoint_pump_a", "build_setpoint_pump_a", "lin_seq_pump_a", "lout_evt_pump_a"],
  "x": 80, "y": 100, "w": 1800, "h": 200
}
```

`x/y/w/h` is the bounding box of contained nodes + padding — compute it from the children's positions.

### 10.3 The hierarchy rule, restated

> Nodes at the **same S88 level** (siblings sharing one parent) **stack vertically in the same lane**.
>
> Nodes at **different S88 levels** (parent ↔ child) sit **next to each other on different lanes**.

### 10.4 Worked example — pumping station demo

```
                 L0           L1            L2            L3            L4         L5         L6           L7
              (input)      (adapter)       (CM)          (EM)         (Unit)     (PC)     (formatter)  (output)

┌── group: Pump A (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ [lin-set-A]  [build-A]                                                                                            │
│ [lin-seq-A]                                                                                                       │
│                          [meas-A-up]                                                                              │
│                          [meas-A-dn]   →   [Pump A]   →                                                           │
│                                                                                                       [format-A] →[lout-evt-A]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pump B (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ ... same shape ...                                                                                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pump C (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ ... same shape ...                                                                                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: MGC — Pump Group (#50a8d9) ──────────────────────────────────────────────────────────────────────────────┐
│ [lin-demand]  [demand→MGC+PS]                          [MGC]                                          [format-MGC]→[lout-evt-MGC]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pumping Station (#0c99d9) ───────────────────────────────────────────────────────────────────────────────┐
│                                                                              [PS]                     [format-PS]→[lout-evt-PS]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Mode broadcast (#dddddd, neutral) ───────────────────────────────────────────────────────────────────────┐
│ [lin-mode]   [fan-mode]   ─────────────►   to all 3 pumps in the Pump A/B/C groups                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Station-wide commands (#dddddd) ─────────────────────────────────────────────────────────────────────────┐
│ [lin-start] [fan-start]   ─►   to pumps                                                                           │
│ [lin-stop]  [fan-stop]                                                                                            │
│ [lin-estop] [fan-estop]                                                                                           │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

What that buys:

- Search "Pump A" highlights the whole group box (parent + sensors + adapters + formatter).
- S88 colour of the group box tells you the level at a glance.
- Wires are horizontal within a group; cross-group wires (Pump A port 2 → MGC) cross only one band.
- Collapse a group in the editor and it becomes a single tile — clutter disappears during reviews.

### 10.5 Multi-input fan-in rule

Stack link-ins tightly at L0, centred on the destination's y. Merge node one lane right at the same y.

### 10.6 Multi-output fan-out rule

Source at the y-centre of its destinations; destinations stack vertically in the next lane. Wires fork cleanly without jogging.

### 10.7 Link-in placement (within a tab)

- All link-ins on **L0**.
- Order them top-to-bottom by the y of their **first downstream target**.
- Link-ins that feed the same destination share the same y-band as that destination.

### 10.8 Link-out placement (within a tab)

- All link-outs on **L7** (the rightmost lane).
- Each link-out's y matches its **upstream source's** y, so the wire is horizontal.

### 10.9 Cross-tab wire rule

Cross-tab wires use `link out` / `link in` pairs (see Section 2). Direct cross-tab wires are forbidden.

### 10.10 The "no jog" verification

- A wire whose source y == destination y is fine (perfectly horizontal).
- A wire that jogs vertically by ≤ 80 px is fine (one row of slop).
- A wire that jogs by > 80 px means **the destination is in the wrong group y-band**. Move the destination, not the source — the source's position was determined by its own group.

## 11. Dashboard tab variant

Dashboard widgets are stamped to the real grid by the FlowFuse renderer; editor x/y is for the editor's readability.

- Use only **L0, L2, L4, L7**:
  - L0 = `link in` (events from process)
  - L2 = `ui-*` inputs (sliders, switches, buttons)
  - L4 = wrapper / format / trend-split functions
  - L7 = `link out` (commands going back)
- **One Node-RED group per `ui-group`.** Editor group's name matches the `ui-group` name. Colour follows the S88 level of the represented equipment (MGC group = `#50a8d9`, Pump A group = `#86bbdd`, …) so the editor view mirrors the dashboard structure.
- Within the group, widgets stack vertically by their visual order in the dashboard.

## 12. Setup tab variant

Single-column ladder L0 → L7, ordered top-to-bottom by `onceDelay`. Wrap in a single neutral-grey Node-RED group named `Deploy-time setup`.

## 13. Demo Drivers tab variant

Same as Process Plant but typically only L0, L2, L4, L7 are used. Wrap each driver (random gen, scripted scenario, …) in its own neutral Node-RED group.

## 14. Spacing constants (final)

```python
LANE_X = [120, 360, 600, 840, 1080, 1320, 1560, 1800]
SIBLING_PITCH  = 40
GROUP_GAP      = 200
TAB_TOP_MARGIN = 80
GROUP_PADDING  = 20   # extra px around child bounding box for the Node-RED group box

S88_COLORS = {
    "AR": "#0f52a5",   # Area (currently unused)
    "PC": "#0c99d9",   # Process Cell
    "UN": "#50a8d9",   # Unit
    "EM": "#86bbdd",   # Equipment Module
    "CM": "#a9daee",   # Control Module
    "neutral": "#dddddd",
}

# Registry: drop a new node type here to place it automatically.
NODE_LEVEL = {
    "measurement":          "CM",
    "rotatingMachine":      "EM",
    "valve":                "EM",
    "diffuser":             "EM",
    "machineGroupControl":  "UN",
    "valveGroupControl":    "UN",
    "reactor":              "UN",
    "settler":              "UN",
    "monster":              "UN",
    "pumpingStation":       "PC",
    "dashboardAPI":         "neutral",
}
```

Helpers for the build script:

```python
def place(lane, group_index, position_in_group, group_size):
    """Compute (x, y) for a node in a process group."""
    x = LANE_X[lane]
    band_centre = TAB_TOP_MARGIN + group_index * (group_size * SIBLING_PITCH + GROUP_GAP) \
                  + (group_size - 1) * SIBLING_PITCH / 2
    y = band_centre + (position_in_group - (group_size - 1) / 2) * SIBLING_PITCH
    return int(x), int(y)

def wrap_in_group(child_ids, name, s88_color, nodes_by_id, padding=GROUP_PADDING):
    """Compute the Node-RED group box around a set of children."""
    xs = [nodes_by_id[c]["x"] for c in child_ids]
    ys = [nodes_by_id[c]["y"] for c in child_ids]
    return {
        "type": "group", "name": name,
        "style": {"label": True, "stroke": "#000000", "fill": s88_color, "fill-opacity": "0.10"},
        "nodes": list(child_ids),
        "x": min(xs) - padding, "y": min(ys) - padding,
        "w": max(xs) - min(xs) + 160 + 2 * padding,
        "h": max(ys) - min(ys) + 40 + 2 * padding,
    }
```

## 15. Verification checklist (extends Section 9)

After building a tab:

1. **No wire jogs > 80 px vertically within a group.**
2. **Each lane contains nodes of one purpose only** (never an `ui-text` on L3; never a `rotatingMachine` on L2).
3. **Peers share a lane; parents and children sit on adjacent lanes.**
4. **Every parent + direct children sit inside one Node-RED group box, coloured by the parent's S88 level.**
5. **Utility groups** (mode broadcast, station commands, demand fan-out) wrapped in neutral-grey Node-RED groups.
6. **Section comments at the top of each group band.**
7. **Editor scrollable in y but NOT in x** on a normal monitor.
8. **Search test:** typing the parent's name in the editor highlights the whole group box.

## 16. S88 colour cleanup (separate follow-up task)

These nodes don't currently follow the S88 palette. They should be brought in line in a separate session before the placement rule is fully consistent across the editor:

- `settler` (`#e4a363` orange) → should be `#50a8d9` (Unit)
- `monster` (`#4f8582` teal) → should be `#50a8d9` (Unit)
- `diffuser` (no colour set) → should be `#86bbdd` (Equipment Module)
- `dashboardAPI` (no colour set) → utility, no S88 colour needed

Until cleaned up, the placement rule still works — `NODE_LEVEL` (Section 14) already maps these to their semantic S88 level regardless of the node's own colour.