feat: architecture refactor — validators, positions, menuUtils, ESLint, tests, CI
Major improvements across the codebase: - Extract validationUtils.js (548→217 lines) into strategy pattern validators - Extract menuUtils.js (543→35 lines) into 6 focused menu modules - Adopt POSITIONS constants across 23 files (183 replacements) - Eliminate all 71 ESLint warnings (0 errors, 0 warnings) - Add 158 unit tests for ConfigManager, MeasurementContainer, ValidationUtils - Add architecture documentation with Mermaid diagrams - Add CI pipeline (Docker, ESLint, Jest, Makefile) - Add E2E infrastructure (docker-compose.e2e.yml) Test results: 377 total (230 Jest + 23 node:test + 124 legacy), all passing Lint: 0 errors, 0 warnings Closes #2, #3, #9, #13, #14, #18 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Rene De Ren
418
docs/ARCHITECTURE.md
Normal file
418
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,418 @@
|
||||
# EVOLV Architecture
|
||||
|
||||
## 1. System Overview
|
||||
|
||||
High-level view of how EVOLV fits into the wastewater treatment automation stack.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
NR[Node-RED Runtime] <-->|msg objects| EVOLV[EVOLV Nodes]
|
||||
EVOLV -->|InfluxDB line protocol| INFLUX[(InfluxDB)]
|
||||
INFLUX -->|queries| GRAFANA[Grafana Dashboards]
|
||||
EVOLV -->|process output| NR
|
||||
EVOLV -->|parent output| NR
|
||||
|
||||
style NR fill:#b22222,color:#fff
|
||||
style EVOLV fill:#0f52a5,color:#fff
|
||||
style INFLUX fill:#0c99d9,color:#fff
|
||||
style GRAFANA fill:#50a8d9,color:#fff
|
||||
```
|
||||
|
||||
Each EVOLV node produces three outputs:
|
||||
| Port | Name | Purpose |
|
||||
|------|------|---------|
|
||||
| 0 | process | Process data forwarded to downstream nodes |
|
||||
| 1 | dbase | InfluxDB-formatted measurement data |
|
||||
| 2 | parent | Control messages to parent nodes (e.g. registerChild) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Node Architecture (Three-Layer Pattern)
|
||||
|
||||
Every node follows a consistent three-layer design that separates Node-RED wiring from domain logic.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Node-RED Runtime"
|
||||
REG["RED.nodes.registerType()"]
|
||||
end
|
||||
|
||||
subgraph "Layer 1 — Wrapper (valve.js)"
|
||||
W[wrapper .js]
|
||||
W -->|"new nodeClass(config, RED, this, name)"| NC
|
||||
W -->|MenuManager| MENU[HTTP /name/menu.js]
|
||||
W -->|configManager| CFG[HTTP /name/configData.js]
|
||||
end
|
||||
|
||||
subgraph "Layer 2 — Node Adapter (src/nodeClass.js)"
|
||||
NC[nodeClass]
|
||||
NC -->|_loadConfig| CFGM[configManager]
|
||||
NC -->|_setupSpecificClass| SC
|
||||
NC -->|_attachInputHandler| INPUT[onInput routing]
|
||||
NC -->|_startTickLoop| TICK[1s tick loop]
|
||||
NC -->|_tick → outputUtils| OUT[formatMsg]
|
||||
end
|
||||
|
||||
subgraph "Layer 3 — Domain Logic (src/specificClass.js)"
|
||||
SC[specificClass]
|
||||
SC -->|measurements| MC[MeasurementContainer]
|
||||
SC -->|state machine| ST[state]
|
||||
SC -->|hydraulics / biology| DOMAIN[domain models]
|
||||
end
|
||||
|
||||
subgraph "generalFunctions"
|
||||
GF[shared library]
|
||||
end
|
||||
|
||||
REG --> W
|
||||
GF -.->|logger, outputUtils, configManager,\nMeasurementContainer, validation, ...| NC
|
||||
GF -.->|MeasurementContainer, state,\nconvert, predict, ...| SC
|
||||
|
||||
style W fill:#0f52a5,color:#fff
|
||||
style NC fill:#0c99d9,color:#fff
|
||||
style SC fill:#50a8d9,color:#fff
|
||||
style GF fill:#86bbdd,color:#000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. generalFunctions Module Map
|
||||
|
||||
The shared library (`nodes/generalFunctions/`) provides all cross-cutting concerns.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
GF[generalFunctions/index.js]
|
||||
|
||||
subgraph "Core Helpers (src/helper/)"
|
||||
LOGGER[logger]
|
||||
OUTPUT[outputUtils]
|
||||
CHILD[childRegistrationUtils]
|
||||
CFGUTIL[configUtils]
|
||||
ASSERT[assertionUtils]
|
||||
VALID[validationUtils]
|
||||
end
|
||||
|
||||
subgraph "Validators (src/helper/validators/)"
|
||||
TV[typeValidators]
|
||||
CV[collectionValidators]
|
||||
CURV[curveValidator]
|
||||
end
|
||||
|
||||
subgraph "Domain Modules (src/)"
|
||||
MC[MeasurementContainer]
|
||||
CFGMGR[configManager]
|
||||
MENUMGR[MenuManager]
|
||||
STATE[state]
|
||||
CONVERT[convert / Fysics]
|
||||
PREDICT[predict / interpolation]
|
||||
NRMSE[nrmse / errorMetrics]
|
||||
COOLPROP[coolprop]
|
||||
end
|
||||
|
||||
subgraph "Data (datasets/)"
|
||||
CURVES[assetData/curves]
|
||||
ASSETS[assetData/assetData.json]
|
||||
UNITS[unitData.json]
|
||||
end
|
||||
|
||||
subgraph "Constants (src/constants/)"
|
||||
POS[POSITIONS / POSITION_VALUES]
|
||||
end
|
||||
|
||||
GF --> LOGGER
|
||||
GF --> OUTPUT
|
||||
GF --> CHILD
|
||||
GF --> CFGUTIL
|
||||
GF --> ASSERT
|
||||
GF --> VALID
|
||||
VALID --> TV
|
||||
VALID --> CV
|
||||
VALID --> CURV
|
||||
GF --> MC
|
||||
GF --> CFGMGR
|
||||
GF --> MENUMGR
|
||||
GF --> STATE
|
||||
GF --> CONVERT
|
||||
GF --> PREDICT
|
||||
GF --> NRMSE
|
||||
GF --> COOLPROP
|
||||
GF --> CURVES
|
||||
GF --> POS
|
||||
|
||||
style GF fill:#0f52a5,color:#fff
|
||||
style LOGGER fill:#86bbdd,color:#000
|
||||
style OUTPUT fill:#86bbdd,color:#000
|
||||
style VALID fill:#86bbdd,color:#000
|
||||
style MC fill:#50a8d9,color:#fff
|
||||
style CFGMGR fill:#50a8d9,color:#fff
|
||||
style MENUMGR fill:#50a8d9,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Data Flow (Message Lifecycle)
|
||||
|
||||
Sequence diagram showing a typical input message and the periodic tick output cycle.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant NR as Node-RED
|
||||
participant W as wrapper.js
|
||||
participant NC as nodeClass
|
||||
participant SC as specificClass
|
||||
participant OU as outputUtils
|
||||
|
||||
Note over W: Node startup
|
||||
W->>NC: new nodeClass(config, RED, node, name)
|
||||
NC->>NC: _loadConfig (configManager.buildConfig)
|
||||
NC->>SC: new specificClass(config, stateConfig, options)
|
||||
NC->>NR: send([null, null, {topic: registerChild}])
|
||||
|
||||
Note over NC: Every 1 second (tick loop)
|
||||
NC->>SC: getOutput()
|
||||
SC-->>NC: raw measurement data
|
||||
NC->>OU: formatMsg(raw, config, 'process')
|
||||
NC->>OU: formatMsg(raw, config, 'influxdb')
|
||||
NC->>NR: send([processMsg, influxMsg])
|
||||
|
||||
Note over NR: Incoming control message
|
||||
NR->>W: msg {topic: 'execMovement', payload: {...}}
|
||||
W->>NC: onInput(msg)
|
||||
NC->>SC: handleInput(source, action, setpoint)
|
||||
SC->>SC: update state machine & measurements
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Node Types
|
||||
|
||||
| Node | S88 Level | Purpose |
|
||||
|------|-----------|---------|
|
||||
| **measurement** | Control Module | Generic measurement point — reads, validates, and stores sensor values |
|
||||
| **valve** | Control Module | Valve simulation with hydraulic model, position control, flow/pressure prediction |
|
||||
| **rotatingMachine** | Control Module | Pumps, blowers, mixers — rotating equipment with speed control and efficiency curves |
|
||||
| **diffuser** | Control Module | Aeration diffuser — models oxygen transfer and pressure drop |
|
||||
| **settler** | Equipment | Sludge settler — models settling behavior and sludge blanket |
|
||||
| **reactor** | Equipment | Hydraulic tank and biological process simulator (activated sludge, digestion) |
|
||||
| **monster** | Equipment | MONitoring and STrEam Routing — complex measurement aggregation |
|
||||
| **pumpingStation** | Unit | Coordinates multiple pumps as a pumping station |
|
||||
| **valveGroupControl** | Unit | Manages multiple valves as a coordinated group — distributes flow, monitors pressure |
|
||||
| **machineGroupControl** | Unit | Group control for rotating machines — load balancing and sequencing |
|
||||
| **dashboardAPI** | Utility | Exposes data and unit conversion endpoints for external dashboards |
|
||||
# EVOLV Architecture
|
||||
|
||||
## Node Hierarchy (S88)
|
||||
|
||||
EVOLV follows the ISA-88 (S88) batch control standard. Each node maps to an S88 level and uses a consistent color scheme in the Node-RED editor.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef area fill:#0f52a5,color:#fff,stroke:#0a3d7a
|
||||
classDef processCell fill:#0c99d9,color:#fff,stroke:#0977aa
|
||||
classDef unit fill:#50a8d9,color:#fff,stroke:#3d89b3
|
||||
classDef equipment fill:#86bbdd,color:#000,stroke:#6a9bb8
|
||||
classDef controlModule fill:#a9daee,color:#000,stroke:#87b8cc
|
||||
classDef standalone fill:#f0f0f0,color:#000,stroke:#999
|
||||
|
||||
%% S88 Levels
|
||||
subgraph "S88: Area"
|
||||
PS[pumpingStation]
|
||||
end
|
||||
|
||||
subgraph "S88: Equipment"
|
||||
MGC[machineGroupControl]
|
||||
VGC[valveGroupControl]
|
||||
end
|
||||
|
||||
subgraph "S88: Control Module"
|
||||
RM[rotatingMachine]
|
||||
V[valve]
|
||||
M[measurement]
|
||||
R[reactor]
|
||||
S[settler]
|
||||
end
|
||||
|
||||
subgraph "Standalone"
|
||||
MON[monster]
|
||||
DASH[dashboardAPI]
|
||||
DIFF[diffuser - not implemented]
|
||||
end
|
||||
|
||||
%% Parent-child registration relationships
|
||||
PS -->|"accepts: measurement"| M
|
||||
PS -->|"accepts: machine"| RM
|
||||
PS -->|"accepts: machineGroup"| MGC
|
||||
PS -->|"accepts: pumpingStation"| PS2[pumpingStation]
|
||||
|
||||
MGC -->|"accepts: machine"| RM
|
||||
|
||||
RM -->|"accepts: measurement"| M2[measurement]
|
||||
RM -->|"accepts: reactor"| R
|
||||
|
||||
VGC -->|"accepts: valve"| V
|
||||
VGC -->|"accepts: machine / rotatingmachine"| RM2[rotatingMachine]
|
||||
VGC -->|"accepts: machinegroup / machinegroupcontrol"| MGC2[machineGroupControl]
|
||||
VGC -->|"accepts: pumpingstation / valvegroupcontrol"| PS3["pumpingStation / valveGroupControl"]
|
||||
|
||||
R -->|"accepts: measurement"| M3[measurement]
|
||||
R -->|"accepts: reactor"| R2[reactor]
|
||||
|
||||
S -->|"accepts: measurement"| M4[measurement]
|
||||
S -->|"accepts: reactor"| R3[reactor]
|
||||
S -->|"accepts: machine"| RM3[rotatingMachine]
|
||||
|
||||
%% Styling
|
||||
class PS,PS2,PS3 area
|
||||
class MGC,MGC2 equipment
|
||||
class VGC equipment
|
||||
class RM,RM2,RM3 controlModule
|
||||
class V controlModule
|
||||
class M,M2,M3,M4 controlModule
|
||||
class R,R2,R3 controlModule
|
||||
class S controlModule
|
||||
class MON,DASH,DIFF standalone
|
||||
```
|
||||
|
||||
### Registration Summary
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
classDef parent fill:#0c99d9,color:#fff
|
||||
classDef child fill:#a9daee,color:#000
|
||||
|
||||
PS[pumpingStation] -->|measurement| LEAF1((leaf))
|
||||
PS -->|machine| RM1[rotatingMachine]
|
||||
PS -->|machineGroup| MGC1[machineGroupControl]
|
||||
PS -->|pumpingStation| PS1[pumpingStation]
|
||||
|
||||
MGC[machineGroupControl] -->|machine| RM2[rotatingMachine]
|
||||
|
||||
VGC[valveGroupControl] -->|valve| V1[valve]
|
||||
VGC -->|source| SRC["machine, machinegroup,<br/>pumpingstation, valvegroupcontrol"]
|
||||
|
||||
RM[rotatingMachine] -->|measurement| LEAF2((leaf))
|
||||
RM -->|reactor| R1[reactor]
|
||||
|
||||
R[reactor] -->|measurement| LEAF3((leaf))
|
||||
R -->|reactor| R2[reactor]
|
||||
|
||||
S[settler] -->|measurement| LEAF4((leaf))
|
||||
S -->|reactor| R3[reactor]
|
||||
S -->|machine| RM3[rotatingMachine]
|
||||
|
||||
class PS,MGC,VGC,RM,R,S parent
|
||||
class LEAF1,LEAF2,LEAF3,LEAF4,RM1,RM2,RM3,MGC1,PS1,V1,SRC,R1,R2,R3 child
|
||||
```
|
||||
|
||||
## Node Types
|
||||
|
||||
| Node | S88 Level | softwareType | role | Accepts Children | Outputs |
|
||||
|------|-----------|-------------|------|-----------------|---------|
|
||||
| **pumpingStation** | Area | `pumpingstation` | StationController | measurement, machine (rotatingMachine), machineGroup, pumpingStation | [process, dbase, parent] |
|
||||
| **machineGroupControl** | Equipment | `machinegroupcontrol` | GroupController | machine (rotatingMachine) | [process, dbase, parent] |
|
||||
| **valveGroupControl** | Equipment | `valvegroupcontrol` | ValveGroupController | valve, machine, rotatingmachine, machinegroup, machinegroupcontrol, pumpingstation, valvegroupcontrol | [process, dbase, parent] |
|
||||
| **rotatingMachine** | Control Module | `rotatingmachine` | RotationalDeviceController | measurement, reactor | [process, dbase, parent] |
|
||||
| **valve** | Control Module | `valve` | controller | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||
| **measurement** | Control Module | `measurement` | Sensor | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||
| **reactor** | Control Module | `reactor` | Biological reactor | measurement, reactor (upstream chaining) | [process, dbase, parent] |
|
||||
| **settler** | Control Module | `settler` | Secondary settler | measurement, reactor (upstream), machine (return pump) | [process, dbase, parent] |
|
||||
| **monster** | Standalone | - | - | dual-parent, standalone | - |
|
||||
| **dashboardAPI** | Standalone | - | - | accepts any child (Grafana integration) | - |
|
||||
| **diffuser** | Standalone | - | - | _(not implemented)_ | - |
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Measurement Data Flow (upstream to downstream)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Sensor as measurement (sensor)
|
||||
participant Machine as rotatingMachine
|
||||
participant Group as machineGroupControl
|
||||
participant Station as pumpingStation
|
||||
|
||||
Note over Sensor: Sensor reads value<br/>(pressure, flow, level, temp)
|
||||
|
||||
Sensor->>Sensor: measurements.type(t).variant("measured").position(p).value(v)
|
||||
Sensor->>Sensor: emitter.emit("type.measured.position", eventData)
|
||||
|
||||
Sensor->>Machine: Event: "pressure.measured.upstream"
|
||||
Machine->>Machine: Store in own MeasurementContainer
|
||||
Machine->>Machine: getMeasuredPressure() -> calcFlow() -> calcPower()
|
||||
Machine->>Machine: emitter.emit("flow.predicted.downstream", eventData)
|
||||
|
||||
Machine->>Group: Event: "flow.predicted.downstream"
|
||||
Group->>Group: handlePressureChange()
|
||||
Group->>Group: Aggregate flows across all machines
|
||||
Group->>Group: Calculate group totals and efficiency
|
||||
|
||||
Machine->>Station: Event: "flow.predicted.downstream"
|
||||
Station->>Station: Store predicted flow in/out
|
||||
Station->>Station: _updateVolumePrediction()
|
||||
Station->>Station: _calcNetFlow(), _calcTimeRemaining()
|
||||
```
|
||||
|
||||
### Control Command Flow (downstream to upstream)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Station as pumpingStation
|
||||
participant Group as machineGroupControl
|
||||
participant Machine as rotatingMachine
|
||||
participant Machine2 as rotatingMachine (2)
|
||||
|
||||
Station->>Group: handleInput("parent", action, param)
|
||||
|
||||
Group->>Group: Determine scaling strategy
|
||||
Group->>Group: Calculate setpoints per machine
|
||||
|
||||
Group->>Machine: handleInput("parent", "execMovement", setpoint)
|
||||
Group->>Machine2: handleInput("parent", "execMovement", setpoint)
|
||||
|
||||
Machine->>Machine: setpoint() -> state.moveTo(pos)
|
||||
Machine->>Machine: updatePosition() -> calcFlow(), calcPower()
|
||||
Machine->>Machine: emitter.emit("flow.predicted.downstream")
|
||||
|
||||
Machine2->>Machine2: setpoint() -> state.moveTo(pos)
|
||||
Machine2->>Machine2: updatePosition() -> calcFlow(), calcPower()
|
||||
Machine2->>Machine2: emitter.emit("flow.predicted.downstream")
|
||||
```
|
||||
|
||||
### Wastewater Treatment Process Flow
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
classDef process fill:#50a8d9,color:#fff
|
||||
classDef equipment fill:#86bbdd,color:#000
|
||||
|
||||
PS_IN[pumpingStation<br/>Influent] -->|flow| R1[reactor<br/>Anoxic]
|
||||
R1 -->|effluent| R2[reactor<br/>Aerated]
|
||||
R2 -->|effluent| SET[settler]
|
||||
SET -->|effluent out| PS_OUT[pumpingStation<br/>Effluent]
|
||||
SET -->|sludge return| RM_RET[rotatingMachine<br/>Return pump]
|
||||
RM_RET -->|recirculation| R1
|
||||
|
||||
PS_IN --- MGC_IN[machineGroupControl]
|
||||
MGC_IN --- RM_IN[rotatingMachine<br/>Influent pumps]
|
||||
|
||||
class PS_IN,PS_OUT process
|
||||
class R1,R2,SET process
|
||||
class MGC_IN,RM_IN,RM_RET equipment
|
||||
```
|
||||
|
||||
### Event-Driven Communication Pattern
|
||||
|
||||
All parent-child communication uses Node.js `EventEmitter`:
|
||||
|
||||
1. **Registration**: Parent calls `childRegistrationUtils.registerChild(child, position)` which stores the child and calls the parent's `registerChild(child, softwareType)` method.
|
||||
2. **Event binding**: The parent's `registerChild()` subscribes to the child's `measurements.emitter` events (e.g., `"flow.predicted.downstream"`).
|
||||
3. **Data propagation**: When a child updates a measurement, it emits an event. The parent's listener stores the value in its own `MeasurementContainer` and runs its domain logic.
|
||||
4. **Three outputs**: Every node sends data to three Node-RED outputs: `[process, dbase, parent]` -- process data for downstream nodes, InfluxDB for persistence, and parent aggregation data.
|
||||
|
||||
### Position Convention
|
||||
|
||||
Children register with a position relative to their parent:
|
||||
- `upstream` -- before the parent in the flow direction
|
||||
- `downstream` -- after the parent in the flow direction
|
||||
- `atEquipment` -- physically located at/on the parent equipment
|
||||
Reference in New Issue
Block a user