EVOLV/examples/pumpingstation-complete-example/README.md

# Pumping Station — Complete Example

End-to-end EVOLV stack: 1 pumpingStation + 1 machineGroupControl + 3 rotatingMachine pumps + 12 measurement nodes (4 per pump), wired through Node-RED to InfluxDB and Grafana.

This is the canonical "everything works together" demo. After any cross-node refactor, run this and verify the Node-RED dashboard, the InfluxDB writes, and the Grafana dashboard all populate.

## Quick start

```bash
cd /home/znetsixe/EVOLV
docker compose up -d
# Wait for http://localhost:1880/nodes to return 200, then:
curl -s -X POST http://localhost:1880/flows \
  -H "Content-Type: application/json" \
  -H "Node-RED-Deployment-Type: full" \
  --data-binary @examples/pumpingstation-complete-example/flow.json
```

Then open:

- Node-RED dashboard (realtime + 1h trends): <http://localhost:1880/dashboard>
- Grafana dashboard (realtime gauges + historic graphs): <http://localhost:3000> (anonymous viewer is on; the dashboard is `EVOLV / Pumping Station (complete)`)
- InfluxDB UI: <http://localhost:8086> (user `evolv` / password `evolv-dev-pw`)

## What the flow contains

| Layer | Node(s) | Role |
|---|---|---|
| Process Cell | `pumpingStation` "Pumping Station" | Wet-well basin model. Levelbased control: drives MGC by basin level. Inflow comes from the Drivers tab; outflow is computed from the pumps. |
| Unit | `machineGroupControl` "MGC — Pump Group" | Distributes flow across the 3 pumps via `optimalcontrol`. |
| Equipment | `rotatingMachine` × 3 — Pump A / B / C | Hidrostal H05K-S03R curve. Auto by default; manual setpoint slider per pump when in `virtualControl`. |
| Control Modules | `measurement` × 12 (4 per pump) | Upstream pressure, downstream pressure, flow, power. Each pump's 4 sensors are driven by a per-pump physics function — values are physically coupled to plant state, not random. |
| Telemetry | shared `evt:tlm` link channel → http POST → InfluxDB | Every EVOLV node's port-1 payload is converted to v2 line protocol and POSTed to `telemetry` bucket. |

## Tabs

The flow is split across 5 tabs, by **concern**:

| Tab | Lives here | Why |
|---|---|---|
| 🏭 **Process Plant** | EVOLV nodes (PS, MGC, 3 pumps, 12 sensors) + per-node output formatters + per-pump physics feeders | The deployable plant model. |
| 📊 **Dashboard UI** | All `ui-*` widgets, button/setpoint wrappers, dispatch functions | Display + operator inputs. No business logic. |
| 🎛️ **Demo Drivers** | Inflow generator (Constant / Sine / Diurnal / Storm) + 1Hz tick | Inflow is operator-driven via slider + scenario buttons. Outflow is implicit (the pumps drain the basin). |
| ⚙️ **Setup & Init** | One-shot `once: true` injects (MGC scaling/mode, pumps mode, initial inflow scenario) | Runs at deploy time only. |
| 📈 **Telemetry** | link-in `evt:tlm` → line-protocol function → http POST | InfluxDB writer. |

Cross-tab wiring uses **named link-out / link-in pairs**, never direct cross-tab wires.

### Channel contract

| Channel | Direction | What it carries |
|---|---|---|
| `cmd:inflow-baseline` | UI → Drivers | numeric m³/h baseline |
| `cmd:inflow-scenario` | UI → Drivers | `'constant' \| 'sine' \| 'diurnal' \| 'storm'` |
| `cmd:q_in` | Drivers → process | computed inflow in m³/s |
| `cmd:Qd` | UI → process | manual demand m³/h (manual mode only) |
| `cmd:ps-mode` | UI → process | `'levelbased' \| 'manual'` |
| `cmd:mode` | Setup → process | per-pump `setMode` broadcast |
| `cmd:station-startup / -shutdown / -estop` | UI → process | station-wide command, fanned to all 3 pumps |
| `cmd:setpoint-A / -B / -C` | UI → process | per-pump setpoint slider value |
| `cmd:pump-A-seq / -B-seq / -C-seq` | UI → process | per-pump start/stop |
| `evt:pump-A / -B / -C` | process → UI | formatted per-pump status |
| `evt:mgc` | process → UI | MGC totals |
| `evt:ps` | process → UI | basin state, level, fill |
| `evt:inflow` | Drivers → UI | live inflow value + active scenario |
| `evt:tlm` | every EVOLV node → Telemetry | port-1 payload in `{measurement, fields, tags}` shape |
| `setup:to-mgc` | Setup → process | one-shot MGC scaling/mode init |

## Per-pump physics feeder

Each pump has a `physics_<pump>` function node on the Process Plant tab. It receives:

1. The pump's own port-0 stream (state, predicted flow, predicted power).
2. PS port-0 stream (basin level), fanned out by `ps_to_physics`.

It computes physically-coupled values for each sensor and emits them to the 4 measurement nodes:

| Sensor | Computation |
|---|---|
| Upstream pressure | `ρ g h` where `h = max(0, basinLevel − outflowLevel)`; pump suction sees the basin's hydrostatic head. |
| Downstream pressure | Idle → static head only (12 m → 1177 mbar). Running → static + flow²-scaled dynamic head (up to ~2354 mbar at q=200 m³/h). |
| Flow | Mirrors rotatingMachine's predicted flow with 1% Gaussian noise. Zero when the pump is idle. |
| Power | Mirrors rotatingMachine's predicted power with 0.5% Gaussian noise. Zero when the pump is idle. |

Gaussian noise uses a 12-uniform-sum approximation (no external libs).

## Inflow scenarios

Pick a scenario on the **Realtime** dashboard page (group "Inflow"):

| Scenario | Behaviour |
|---|---|
| Constant | `q_h = baseline` (no modulation) |
| Sine | `baseline · (1 + 0.5 · sin(2πt/240))` — period 4 min |
| Diurnal | `baseline · (1 + 0.6 · sin(2πt/480 − π/2))` — period 8 min, peak offset |
| Storm | 4-min cycle: rapid 5× ramp, then linear decay back to baseline |

Slider sets `baseline` in m³/h (0–250). The generator emits `q_in` to PS every second.

## Dashboard map

### Node-RED — `/dashboard`

Realtime page (`/dashboard/realtime`):

1. Inflow — slider, 4 scenario buttons, live value + active scenario label
2. Station mode + commands — Auto/Manual switch, manual Qd slider, Start All / Stop All / Emergency Stop
3. Basin realtime — direction, level, volume, fill %, net flow, time-to-full/empty, inflow, outflow, safety state, gauges (level + fill)
4. MGC — total flow + power (text + gauges), efficiency
5. Pump A / B / C — state, mode, controller %, flow, power, up/dn pressure (text), setpoint slider, Startup / Shutdown buttons

Trends page (`/dashboard/trends`) — 1-hour rolling windows:

- Basin level + fill %
- Inflow / Outflow / Per-pump flow (one chart, multi-series)
- Per-pump power
- Per-pump up/dn pressure

### Grafana — `EVOLV / Pumping Station (complete)`

Two rows:

- **Realtime** — gauges for basin level + fill, stat panels for total flow / total power / per-pump state.
- **Historic** — line charts for level + fill, inflow/outflow/net, per-pump flow + power (predicted), per-pump pressure, per-pump sensor flow + power (measured).

Default time range: last 15 minutes. Adjust with the Grafana picker for longer history.

## Verification

```bash
# 1. Bring up the stack
docker compose up -d
sleep 10  # wait for Node-RED ready

# 2. Deploy the flow
curl -s -X POST http://localhost:1880/flows \
  -H 'Content-Type: application/json' \
  -H 'Node-RED-Deployment-Type: full' \
  --data-binary @examples/pumpingstation-complete-example/flow.json | jq .

# 3. Quick sanity check on Influx writes
curl -s -X POST 'http://localhost:8086/api/v2/query?org=evolv' \
  -H 'Authorization: Token evolv-dev-token' \
  -H 'Accept: application/csv' \
  -H 'Content-type: application/vnd.flux' \
  --data 'from(bucket:"telemetry") |> range(start: -1m) |> count() |> group(columns: ["_measurement"])'
```

You should see counts per measurement (`Pumping Station`, `Pump A`, `MGC — Pump Group`, the per-pump sensors, …) growing in real time.

## Regenerating `flow.json`

`flow.json` is generated from `build_flow.py`. Edit the Python (cleaner diff) and regenerate:

```bash
cd examples/pumpingstation-complete-example
python3 build_flow.py > flow.json
```

The Python is the source of truth.

After regenerating, push the new flow into the running runtime:

```bash
./scripts/sync-example.sh pumpingstation-complete-example
```

## Projects + persistence (Node-RED)

The Docker stack uses a named volume (`evolv_nodered_data`) for `/data`, and Node-RED's **Projects** feature is enabled. Each folder under `examples/` is bootstrapped into `/data/projects/<name>/` on first container start with its own `git init` and a synthesized `package.json`. Switching between projects is two clicks in the editor: **menu → Projects → Open Project**.

| What you do | Where it lives | What persists |
|---|---|---|
| `docker compose down && up` | Container is recreated; named volume survives | Active flow + project list survive |
| Edit a flow in the Node-RED editor | `/data/projects/<name>/flow.json` (in volume) | Until `docker compose down -v` |
| Edit `examples/<name>/build_flow.py` then regenerate | `examples/<name>/flow.json` (in repo) | Always — it's in Git |
| Run `scripts/sync-example.sh <name>` | Copies repo's `flow.json` → volume's project + reloads | Volume copy now matches repo |

### Adding a new example as a project

1. Create `examples/<your-name>/flow.json` (build it however you like — `build_flow.py` is one way).
2. Restart the Node-RED container: `docker compose restart nodered`.
3. Editor → Projects → Open Project → pick `<your-name>`.

The bootstrap is idempotent: existing projects in the volume aren't overwritten. To force a refresh from the repo: delete the project in the volume (`docker exec evolv-nodered rm -rf /data/projects/<name>`) and restart, or use `scripts/sync-example.sh` for a flow-only refresh.

To start fresh (wipe all volume state including flows, sessions, project history): `docker compose down -v`.

## Notable design choices

- **PS in `levelbased` mode** with `manual` mode toggleable from the UI. Levelbased = PS commands MGC by basin level; manual = operator drives MGC via the Qd slider.
- **Inflow is operator-driven**, outflow is implicit (computed from pump activity). Single steerable knob (the Inflow group) keeps the demo focused.
- **Sensors driven externally**, not by the measurement node's built-in simulator. The physics feeder is a function node on the Process Plant tab — disable it and sensors freeze, which is a useful failure mode to demonstrate.
- **All EVOLV port 1 → one shared telemetry channel** (`evt:tlm`) → one writer. Adding a new EVOLV node anywhere in the flow only needs a new `lout_tlm_<id>` link-out + appending the id to `_all_tlm_lout_ids()` in `build_flow.py`.
- **Dashboard pages split by concern, not data**: realtime widgets never share a page with historical charts.