EVOLV/.claude/rules/node-red-flow-layout.md

Node-RED Flow Layout Rules

How to lay out a multi-tab Node-RED demo or production flow so it is readable, debuggable, and trivially extendable. These rules apply to anything you build with examples/ flows, dashboards, or production deployments.

1. Tab boundaries — by CONCERN, not by data

Every node lives on the tab matching its concern, never where it happens to be wired:

Tab	Lives here	Never here
🏭 Process Plant	EVOLV nodes (rotatingMachine, MGC, pumpingStation, measurement, reactor, settler, …) + small per-node output formatters	UI widgets, demo drivers, one-shot setup injects
📊 Dashboard UI	All ui-* widgets, the wrapper functions that turn a button click into a typed msg, the trend-feeder split functions	Anything that produces data autonomously, anything that talks to EVOLV nodes directly
🎛️ Demo Drivers	Random generators, scripted scenarios, schedule injectors, anything that exists only to drive the demo	Real production data sources (those go on Process Plant or are wired in externally)
⚙️ Setup & Init	One-shot once: true injects (setMode, setScaling, auto-startup)	Anything that fires more than once

Why these four: each tab can be disabled or deleted independently. Disable Demo Drivers → demo becomes inert until a real data source is wired. Disable Setup → fresh deploys don't auto-configure (good for debugging). Disable Dashboard UI → headless mode for tests. Process Plant always stays.

If you find yourself wanting a node "between" two tabs, you've named your concerns wrong — re-split.

2. Cross-tab wiring — link nodes only, named channels

Never wire a node on tab A directly to a node on tab B. Use named link-out / link-in pairs:

[ui-slider] ──► [link out  cmd:demand] ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
                                                                    │
                                                                    ▼
[random gen] ─► [link out  cmd:demand] ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─►  [link in  cmd:demand] ──► [router] ──► [MGC]
                                                                    ▲
                                                                    │
                                  many link-outs may target one link-in

Naming convention

Channels follow <direction>:<topic> lowercase, kebab-case after the colon:

• cmd: — UI / drivers → process. Carries commands.
• evt: — process → UI / external. Carries state events.
• setup: — setup tab → wherever. Carries one-shot init.

Examples used in the pumping-station demo:

• cmd:demand, cmd:randomToggle, cmd:mode
• cmd:station-startup, cmd:station-shutdown, cmd:station-estop
• cmd:setpoint-A, cmd:setpoint-B, cmd:setpoint-C
• cmd:pump-A-seq (start/stop for pump A specifically)
• evt:pump-A, evt:pump-B, evt:pump-C, evt:mgc, evt:ps
• setup:to-mgc

Channels are the contract

The list of channel names IS the inter-tab API. Document it in the demo's README. Renaming a channel is a breaking change.

When to use one channel vs many

• One channel, many emitters: same kind of message from multiple sources (e.g. cmd:demand is fired by both the slider and the random generator).
• Different channels: messages with different meaning even if they go to the same node (e.g. don't fold cmd:setpoint-A into a generic cmd:pump-A — keep setpoint and start/stop separate).
• Avoid one mega-channel: a "process commands" channel that the receiver routes-by-topic is harder to read than separate channels per concern.

Don't use link-call for fan-out

link call is for synchronous request/response (waits for a paired link out in return mode). For fan-out, use plain link out (mode=link) with multiple targets, or a single link out → single link in → function-node fan-out (whichever is clearer for your case).

3. Spacing and visual layout

Nodes need air to be readable. Apply these constants in any flow generator:

LANE_X = [120, 380, 640, 900, 1160, 1420]   # 6 vertical lanes per tab
ROW = 80                                     # standard row pitch
SECTION_GAP = 200                            # extra y-shift between sections

Lane assignment (process plant tab as example)

Lane	Contents
0 (x=120)	Inputs from outside the tab — link-in nodes, injects
1 (x=380)	First-level transformers — wrappers, fan-outs, routers
2 (x=640)	Mid-level — section comments live here too
3 (x=900)	Target nodes — the EVOLV node itself (pump, MGC, PS)
4 (x=1160)	Output formatters — function nodes that build dashboard-friendly payloads
5 (x=1420)	Outputs to outside the tab — link-out nodes, debug taps

Inputs flow left → right. Don't loop wires backwards across the tab.

Section comments

Every logical group within a tab gets a comment header at lane 2 with a ── Section name ── style label. Use them liberally — every 3-5 nodes deserves a header. The info field on the comment carries the multi-line description.

Section spacing

SECTION_GAP = 200 between sections, on top of the standard row pitch. Don't pack sections together — when you have 6 measurements on a tab, give each pump 4 rows + a 200 px gap to the next pump. Yes, it makes tabs scroll. Scroll is cheap; visual confusion is expensive.

4. Charts — the trend-split rule

ui-chart with category: "topic" + categoryType: "msg" plots one series per unique msg.topic. So:

• One chart per metric type (one chart for flow, one for power).
• Each chart receives msgs whose topic is the series label (e.g. Pump A, Pump B, Pump C).

Required chart properties (FlowFuse ui-chart renders blank without ALL of these)

Derived from working charts in rotatingMachine/examples/03-Dashboard. Every property listed below is mandatory — omit any one and the chart renders blank with no error message.

{
  "type": "ui-chart",
  "chartType": "line",
  "interpolation": "linear",
  "category": "topic",
  "categoryType": "msg",
  "xAxisType": "time",
  "xAxisProperty": "",
  "xAxisPropertyType": "timestamp",
  "xAxisFormat": "",
  "xAxisFormatType": "auto",
  "yAxisProperty": "payload",
  "yAxisPropertyType": "msg",
  "action": "append",
  "stackSeries": false,
  "pointShape": "circle",
  "pointRadius": 4,
  "showLegend": true,
  "bins": 10,
  "width": 12,
  "height": 6,
  "removeOlder": "15",
  "removeOlderUnit": "60",
  "removeOlderPoints": "",
  "colors": ["#0095FF","#FF0000","#FF7F0E","#2CA02C","#A347E1","#D62728","#FF9896","#9467BD","#C5B0D5"],
  "textColor": ["#666666"],
  "textColorDefault": true,
  "gridColor": ["#e5e5e5"],
  "gridColorDefault": true
}

Key gotchas:

• interpolation MUST be set ("linear", "step", "bezier", "cubic", "cubic-mono"). Without it: no line drawn.
• yAxisProperty: "payload" + yAxisPropertyType: "msg" tells the chart WHERE in the msg to find the y-value. Without these: chart has no data to plot.
• xAxisPropertyType: "timestamp" tells the chart to use msg.timestamp (or auto-generated) for the x-axis.
• width and height are numbers, not strings. width: 12 (correct) vs width: "12" (may break).
• removeOlderPoints: "" (empty string) → retention is controlled by removeOlder + removeOlderUnit only. Set to a number string to additionally cap points per series.
• colors array defines the palette for auto-assigned series colours. Provide at least 3.

The trend-split function pattern

A common bug: feeding both flow and power msgs to a single function output that wires to both charts. Both charts then plot all metrics, garbling the legend.

Fix: the trend-feeder function MUST have one output per chart, and split:

// outputs: 2
// wires: [["chart_flow"], ["chart_power"]]
const flowMsg  = p.flowNum  != null ? { topic: 'Pump A', payload: p.flowNum  } : null;
const powerMsg = p.powerNum != null ? { topic: 'Pump A', payload: p.powerNum } : null;
return [flowMsg, powerMsg];

A null msg on a given output sends nothing on that output — exactly what we want.

Chart axis settings to actually configure

• removeOlder + removeOlderUnit: how much history to keep (e.g. 10 minutes).
• removeOlderPoints: cap on points per series (200 is sensible for a demo).
• ymin / ymax: leave blank for autoscale, or set numeric strings if you want a fixed range.

5. Inject node — payload typing

Multi-prop inject must populate v and vt per prop, not just the legacy top-level payload + payloadType:

{
  "props": [
    {"p": "topic", "vt": "str"},
    {"p": "payload", "v": "{\"action\":\"startup\"}", "vt": "json"}
  ],
  "topic": "execSequence",
  "payload": "{\"action\":\"startup\"}",
  "payloadType": "json"
}

If you only fill the top-level fields, payload_type=json is silently treated as str.

6. Dashboard widget rules

• Widget = display only. No business logic in ui-text formats or ui-template HTML.
• Buttons emit a typed string payload ("fired" or similar). Convert to the real msg shape with a tiny wrapper function on the same tab, before the link-out.
• Sliders use passthru: true so they re-emit on input messages (useful for syncing initial state from the process side later).
• One ui-page per demo. Multiple groups under one page is the natural split.
• Group widths should sum to a multiple of 12. The page grid is 12 columns. A row of 4 + 4 + 4 or 6 + 6 works; mixing arbitrary widths leaves gaps.
• EVERY ui- node needs x and y keys.* Without them Node-RED dumps the node at (0,0) — every text widget and chart piles up in the top-left of the editor canvas. The dashboard itself still renders correctly (it lays out by group/order, not editor x/y), but the editor view is unreadable. If you write a flow generator helper, set x and y on the dict EVERY time. Test with jq '[.[] | select(.x==0 and .y==0 and (.type|tostring|startswith("ui-")))]' after generating.

7. Do / don't checklist

✅ Do:

• Generate flows from a Python builder (build_flow.py) — it's the source of truth.
• Use deterministic IDs (pump_a, meas_pump_a_u, lin_demand_to_mgc) — reproducible diffs across regenerations.
• Tag every channel name with cmd: / evt: / setup:.
• Comment every section, even short ones.
• Verify trends with a ui-chart of synthetic data first, before plumbing real data through.

❌ Don't:

• Don't use replace_all on a Python identifier that appears in a node's own wires definition — you'll create self-loops (>250k msg/s discovered the hard way).
• Don't wire across tabs directly. The wire IS allowed but it makes the editor unreadable.
• Don't put dashboard widgets next to EVOLV nodes — different concerns.
• Don't pack nodes within 40 px of each other — labels overlap, wires snap to wrong handles.
• Don't ship enableLog: "debug" in a demo — fills the container log within seconds and obscures real errors.

8. The link-out / link-in JSON shape (cheat sheet)

{
  "id": "lout_demand_dash",
  "type": "link out",
  "z": "tab_ui",
  "name": "cmd:demand",
  "mode": "link",
  "links": ["lin_demand_to_mgc"],
  "x": 380, "y": 140,
  "wires": []
}

{
  "id": "lin_demand_to_mgc",
  "type": "link in",
  "z": "tab_process",
  "name": "cmd:demand",
  "links": ["lout_demand_dash", "lout_demand_drivers"],
  "x": 120, "y": 1500,
  "wires": [["demand_fanout_mgc_ps"]]
}

Both ends store the paired ids in links. The name is cosmetic (label only) — Node-RED routes by id. Multiple emitters can target one receiver; one emitter can target multiple receivers.

9. Node configuration completeness — ALWAYS set every field

When placing an EVOLV node in a flow (demo or production), configure every config field the node's schema defines — don't rely on schema defaults for operational parameters. Schema defaults exist to make the validator happy, not to represent a realistic plant.

Why this matters: A pumpingStation with basinVolume: 10 but default heightOverflow: 2.5 and default heightOutlet: 0.2 creates an internally inconsistent basin where the fill % exceeds 100%, safety guards fire at wrong thresholds, and the demo looks broken. Every field interacts with every other field.

The rule:

1. Read the node's config schema (generalFunctions/src/configs/<nodeName>.json) before writing the flow.
2. For each section (basin, hydraulics, control, safety, scaling, smoothing, …), set EVERY field explicitly in the flow JSON — even if you'd pick the same value as the default.
3. Add a comment in the flow generator per section explaining WHY you chose each value (e.g. "basin sized so sinus peak takes 6 min to fill from startLevel to overflow").
4. Cross-check computed values: surfaceArea = volume / height, maxVolOverflow = heightOverflow × surfaceArea, gauge max = basin height, fill % denominator = volume (not overflow volume).
5. If a gauge or chart references a config value (basin height, maxVol), derive it from the same source — never hardcode a number that was computed elsewhere.

10. Verifying the layout

Before declaring a flow done:

1. Open the tab in the editor — every wire should run left → right. No backward loops.
2. Open each section by section comment — visible in 1 screen height. If not, raise SECTION_GAP.
3. Hit the dashboard URL — every widget has data. n/a everywhere is a contract failure.
4. For charts, watch a series populate over 30 s. A blank chart after 30 s = bug.
5. Disable each tab one at a time and re-deploy. Process Plant alone should still load (just inert). Dashboard UI alone should serve a page (just empty). If disabling a tab errors out, the tab boundaries are wrong.

10. Hierarchical placement — by S88 level, not by node name

The lane assignment maps to the S88 hierarchy, not to specific node names. Any node that lives at a given S88 level goes in the same lane regardless of what kind of equipment it is. New node types added to the platform inherit a lane by their S88 category — no rule change needed.

10.1 Lane convention (x-axis = S88 level)

Lane	x	Purpose	S88 level	Colour	Current EVOLV nodes
L0	120	Tab inputs	—	(none)	link in, inject
L1	360	Adapters	—	(none)	function (msg-shape wrappers)
L2	600	Control Module	CM	#a9daee	measurement
L3	840	Equipment Module	EM	#86bbdd	rotatingMachine, valve, diffuser
L4	1080	Unit	UN	#50a8d9	machineGroupControl, valveGroupControl, reactor, settler, monster
L5	1320	Process Cell	PC	#0c99d9	pumpingStation
L6	1560	Output formatters	—	(none)	function (build dashboard payload from port 0)
L7	1800	Tab outputs	—	(none)	link out, debug

Spacing: 240 px between lanes. Tab width ≤ 1920 px (fits standard monitors without horizontal scroll in the editor).

Area level (#0f52a5) is reserved for plant-wide coordination and currently unused — when added, allocate a new lane and shift formatter/output one lane right (i.e. expand to 9 lanes if and when needed).

10.2 The group rule (Node-RED group boxes anchor each parent + its children)

Use Node-RED's native group node (the visual box around a set of nodes — not to be confused with ui-group) to anchor every "parent + direct children" cluster. The box makes ownership unambiguous and lets you collapse the cluster in the editor.

Group rules:

• One Node-RED group per parent + its direct children. Example: Pump A + meas-A-up + meas-A-dn is one group, named Pump A.
• Group colour = parent's S88 colour. So a Pump-A group is #86bbdd (Equipment Module). A reactor group is #50a8d9 (Unit).
• Group style.label = true so the box shows the parent's name.
• Group must contain all the children's adapters / wrappers / formatters too if those exclusively belong to the parent. The box is the visual anchor for "this is everything that owns / serves Pump A".
• Utility groups for cross-cutting logic (mode broadcast, station-wide commands, demand fan-out) use a neutral colour (#dddddd).

JSON shape:

{
  "id": "grp_pump_a",
  "type": "group",
  "z": "tab_process",
  "name": "Pump A",
  "style": { "label": true, "stroke": "#000000", "fill": "#86bbdd", "fill-opacity": "0.10" },
  "nodes": ["meas_pump_a_u", "meas_pump_a_d", "pump_a", "format_pump_a", "lin_setpoint_pump_a", "build_setpoint_pump_a", "lin_seq_pump_a", "lout_evt_pump_a"],
  "x": 80, "y": 100, "w": 1800, "h": 200
}

x/y/w/h is the bounding box of contained nodes + padding — compute it from the children's positions.

10.3 The hierarchy rule, restated

Nodes at the same S88 level (siblings sharing one parent) stack vertically in the same lane.

Nodes at different S88 levels (parent ↔ child) sit next to each other on different lanes.

10.4 Worked example — pumping station demo

                 L0           L1            L2            L3            L4         L5         L6           L7
              (input)      (adapter)       (CM)          (EM)         (Unit)     (PC)     (formatter)  (output)

┌── group: Pump A (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ [lin-set-A]  [build-A]                                                                                            │
│ [lin-seq-A]                                                                                                       │
│                          [meas-A-up]                                                                              │
│                          [meas-A-dn]   →   [Pump A]   →                                                           │
│                                                                                                       [format-A] →[lout-evt-A]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pump B (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ ... same shape ...                                                                                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pump C (#86bbdd) ─────────────────────────────────────────────────────────────────────────────────────────┐
│ ... same shape ...                                                                                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: MGC — Pump Group (#50a8d9) ──────────────────────────────────────────────────────────────────────────────┐
│ [lin-demand]  [demand→MGC+PS]                          [MGC]                                          [format-MGC]→[lout-evt-MGC]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Pumping Station (#0c99d9) ───────────────────────────────────────────────────────────────────────────────┐
│                                                                              [PS]                     [format-PS]→[lout-evt-PS]
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Mode broadcast (#dddddd, neutral) ───────────────────────────────────────────────────────────────────────┐
│ [lin-mode]   [fan-mode]   ─────────────►   to all 3 pumps in the Pump A/B/C groups                                │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

┌── group: Station-wide commands (#dddddd) ─────────────────────────────────────────────────────────────────────────┐
│ [lin-start] [fan-start]   ─►   to pumps                                                                           │
│ [lin-stop]  [fan-stop]                                                                                            │
│ [lin-estop] [fan-estop]                                                                                           │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

What that buys:

• Search "Pump A" highlights the whole group box (parent + sensors + adapters + formatter).
• S88 colour of the group box tells you the level at a glance.
• Wires are horizontal within a group; cross-group wires (Pump A port 2 → MGC) cross only one band.
• Collapse a group in the editor and it becomes a single tile — clutter disappears during reviews.

10.5 Multi-input fan-in rule

Stack link-ins tightly at L0, centred on the destination's y. Merge node one lane right at the same y.

10.6 Multi-output fan-out rule

Source at the y-centre of its destinations; destinations stack vertically in the next lane. Wires fork cleanly without jogging.

10.7 Link-in placement (within a tab)

• All link-ins on L0.
• Order them top-to-bottom by the y of their first downstream target.
• Link-ins that feed the same destination share the same y-band as that destination.

10.8 Link-out placement (within a tab)

• All link-outs on L7 (the rightmost lane).
• Each link-out's y matches its upstream source's y, so the wire is horizontal.

10.9 Cross-tab wire rule

Cross-tab wires use link out / link in pairs (see Section 2). Direct cross-tab wires are forbidden.

10.10 The "no jog" verification

• A wire whose source y == destination y is fine (perfectly horizontal).
• A wire that jogs vertically by ≤ 80 px is fine (one row of slop).
• A wire that jogs by > 80 px means the destination is in the wrong group y-band. Move the destination, not the source — the source's position was determined by its own group.

11. Dashboard tab variant

Dashboard widgets are stamped to the real grid by the FlowFuse renderer; editor x/y is for the editor's readability.

• Use only L0, L2, L4, L7:
  • L0 = link in (events from process)
  • L2 = ui-* inputs (sliders, switches, buttons)
  • L4 = wrapper / format / trend-split functions
  • L7 = link out (commands going back)
• One Node-RED group per ui-group. Editor group's name matches the ui-group name. Colour follows the S88 level of the represented equipment (MGC group = #50a8d9, Pump A group = #86bbdd, …) so the editor view mirrors the dashboard structure.
• Within the group, widgets stack vertically by their visual order in the dashboard.

12. Setup tab variant

Single-column ladder L0 → L7, ordered top-to-bottom by onceDelay. Wrap in a single neutral-grey Node-RED group named Deploy-time setup.

13. Demo Drivers tab variant

Same as Process Plant but typically only L0, L2, L4, L7 are used. Wrap each driver (random gen, scripted scenario, …) in its own neutral Node-RED group.

14. Spacing constants (final)

LANE_X = [120, 360, 600, 840, 1080, 1320, 1560, 1800]
SIBLING_PITCH  = 40
GROUP_GAP      = 200
TAB_TOP_MARGIN = 80
GROUP_PADDING  = 20   # extra px around child bounding box for the Node-RED group box

S88_COLORS = {
    "AR": "#0f52a5",   # Area (currently unused)
    "PC": "#0c99d9",   # Process Cell
    "UN": "#50a8d9",   # Unit
    "EM": "#86bbdd",   # Equipment Module
    "CM": "#a9daee",   # Control Module
    "neutral": "#dddddd",
}

# Registry: drop a new node type here to place it automatically.
NODE_LEVEL = {
    "measurement":          "CM",
    "rotatingMachine":      "EM",
    "valve":                "EM",
    "diffuser":             "EM",
    "machineGroupControl":  "UN",
    "valveGroupControl":    "UN",
    "reactor":              "UN",
    "settler":              "UN",
    "monster":              "UN",
    "pumpingStation":       "PC",
    "dashboardAPI":         "neutral",
}

Helpers for the build script:

def place(lane, group_index, position_in_group, group_size):
    """Compute (x, y) for a node in a process group."""
    x = LANE_X[lane]
    band_centre = TAB_TOP_MARGIN + group_index * (group_size * SIBLING_PITCH + GROUP_GAP) \
                  + (group_size - 1) * SIBLING_PITCH / 2
    y = band_centre + (position_in_group - (group_size - 1) / 2) * SIBLING_PITCH
    return int(x), int(y)

def wrap_in_group(child_ids, name, s88_color, nodes_by_id, padding=GROUP_PADDING):
    """Compute the Node-RED group box around a set of children."""
    xs = [nodes_by_id[c]["x"] for c in child_ids]
    ys = [nodes_by_id[c]["y"] for c in child_ids]
    return {
        "type": "group", "name": name,
        "style": {"label": True, "stroke": "#000000", "fill": s88_color, "fill-opacity": "0.10"},
        "nodes": list(child_ids),
        "x": min(xs) - padding, "y": min(ys) - padding,
        "w": max(xs) - min(xs) + 160 + 2 * padding,
        "h": max(ys) - min(ys) + 40 + 2 * padding,
    }

15. Verification checklist (extends Section 9)

After building a tab:

1. No wire jogs > 80 px vertically within a group.
2. Each lane contains nodes of one purpose only (never an ui-text on L3; never a rotatingMachine on L2).
3. Peers share a lane; parents and children sit on adjacent lanes.
4. Every parent + direct children sit inside one Node-RED group box, coloured by the parent's S88 level.
5. Utility groups (mode broadcast, station commands, demand fan-out) wrapped in neutral-grey Node-RED groups.
6. Section comments at the top of each group band.
7. Editor scrollable in y but NOT in x on a normal monitor.
8. Search test: typing the parent's name in the editor highlights the whole group box.

16. S88 colour cleanup (separate follow-up task)

These nodes don't currently follow the S88 palette. They should be brought in line in a separate session before the placement rule is fully consistent across the editor:

• settler (#e4a363 orange) → should be #50a8d9 (Unit)
• monster (#4f8582 teal) → should be #50a8d9 (Unit)
• diffuser (no colour set) → should be #86bbdd (Equipment Module)
• dashboardAPI (no colour set) → utility, no S88 colour needed

Until cleaned up, the placement rule still works — NODE_LEVEL (Section 14) already maps these to their semantic S88 level regardless of the node's own colour.