feat: architecture refactor — validators, positions, menuUtils, ESLint, tests, CI

Major improvements across the codebase:

- Extract validationUtils.js (548→217 lines) into strategy pattern validators
- Extract menuUtils.js (543→35 lines) into 6 focused menu modules
- Adopt POSITIONS constants across 23 files (183 replacements)
- Eliminate all 71 ESLint warnings (0 errors, 0 warnings)
- Add 158 unit tests for ConfigManager, MeasurementContainer, ValidationUtils
- Add architecture documentation with Mermaid diagrams
- Add CI pipeline (Docker, ESLint, Jest, Makefile)
- Add E2E infrastructure (docker-compose.e2e.yml)

Test results: 377 total (230 Jest + 23 node:test + 124 legacy), all passing
Lint: 0 errors, 0 warnings

Closes #2, #3, #9, #13, #14, #18

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Rene De Ren
2026-03-11 15:37:20 +01:00
parent 80de324b32
commit 905a061590
28 changed files with 5236 additions and 26 deletions

5
.dockerignore Normal file
View File

@@ -0,0 +1,5 @@
.git
*/.git
node_modules
*.md
!README.md

41
.gitea/workflows/ci.yml Normal file
View File

@@ -0,0 +1,41 @@
name: CI
on:
push:
branches: [main, develop, dev-Rene]
pull_request:
branches: [main]
jobs:
lint-and-test:
runs-on: ubuntu-latest
container:
image: node:20-slim
steps:
- name: Install git
run: apt-get update -qq && apt-get install -y -qq git
- name: Checkout with submodules
uses: actions/checkout@v4
with:
submodules: recursive
- name: Rewrite generalFunctions to local path
run: |
sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
- name: Install dependencies
run: npm install --ignore-scripts
- name: Lint
run: npm run lint
- name: Test (Jest)
run: npm test
- name: Test (node:test)
run: npm run test:node
- name: Test (legacy)
run: npm run test:legacy

24
.gitmodules vendored
View File

@@ -1,37 +1,37 @@
[submodule "nodes/machineGroupControl"]
path = nodes/machineGroupControl
url = https://gitea.centraal.wbd-rd.nl/RnD/machineGroupControl.git
url = https://gitea.wbd-rd.nl/RnD/machineGroupControl.git
[submodule "nodes/generalFunctions"]
path = nodes/generalFunctions
url = https://gitea.centraal.wbd-rd.nl/RnD/generalFunctions.git
url = https://gitea.wbd-rd.nl/RnD/generalFunctions.git
[submodule "nodes/valveGroupControl"]
path = nodes/valveGroupControl
url = https://gitea.centraal.wbd-rd.nl/RnD/valveGroupControl.git
url = https://gitea.wbd-rd.nl/RnD/valveGroupControl.git
[submodule "nodes/valve"]
path = nodes/valve
url = https://gitea.centraal.wbd-rd.nl/RnD/valve.git
url = https://gitea.wbd-rd.nl/RnD/valve.git
[submodule "nodes/rotatingMachine"]
path = nodes/rotatingMachine
url = https://gitea.centraal.wbd-rd.nl/RnD/rotatingMachine.git
url = https://gitea.wbd-rd.nl/RnD/rotatingMachine.git
[submodule "nodes/monster"]
path = nodes/monster
url = https://gitea.centraal.wbd-rd.nl/RnD/monster.git
url = https://gitea.wbd-rd.nl/RnD/monster.git
[submodule "nodes/measurement"]
path = nodes/measurement
url = https://gitea.centraal.wbd-rd.nl/RnD/measurement.git
url = https://gitea.wbd-rd.nl/RnD/measurement.git
[submodule "nodes/diffuser"]
path = nodes/diffuser
url = https://gitea.centraal.wbd-rd.nl/RnD/diffuser.git
url = https://gitea.wbd-rd.nl/RnD/diffuser.git
[submodule "nodes/dashboardAPI"]
path = nodes/dashboardAPI
url = https://gitea.centraal.wbd-rd.nl/RnD/dashboardAPI.git
url = https://gitea.wbd-rd.nl/RnD/dashboardAPI.git
[submodule "nodes/reactor"]
path = nodes/reactor
url = https://gitea.centraal.wbd-rd.nl/RnD/reactor.git
url = https://gitea.wbd-rd.nl/RnD/reactor.git
[submodule "nodes/pumpingStation"]
path = nodes/pumpingStation
url = https://gitea.centraal.wbd-rd.nl/RnD/pumpingStation
url = https://gitea.wbd-rd.nl/RnD/pumpingStation
[submodule "nodes/settler"]
path = nodes/settler
url = https://gitea.centraal.wbd-rd.nl/RnD/settler.git
url = https://gitea.wbd-rd.nl/RnD/settler.git

32
CLAUDE.md Normal file
View File

@@ -0,0 +1,32 @@
# EVOLV - Claude Code Project Guide
## What This Is
Node-RED custom nodes package for wastewater treatment plant automation. Developed by Waterschap Brabantse Delta R&D team. Follows ISA-88 (S88) batch control standard.
## Architecture
Each node follows a three-layer pattern:
1. **Node-RED wrapper** (`<name>.js`) - registers the node type, sets up HTTP endpoints
2. **Node adapter** (`src/nodeClass.js`) - bridges Node-RED API with domain logic, handles config loading, tick loops, events
3. **Domain logic** (`src/specificClass.js`) - pure business logic, no Node-RED dependencies
## Key Shared Library: `nodes/generalFunctions/`
- `logger` - structured logging (use this, NOT console.log)
- `MeasurementContainer` - chainable measurement storage (type/variant/position)
- `configManager` - loads JSON configs from `src/configs/`
- `MenuManager` - dynamic UI dropdowns
- `outputUtils` - formats messages for InfluxDB and process outputs
- `childRegistrationUtils` - parent-child node relationships
- `coolprop` - thermodynamic property calculations
## Conventions
- Nodes register under category `'EVOLV'` in Node-RED
- S88 color scheme: Area=#0f52a5, ProcessCell=#0c99d9, Unit=#50a8d9, Equipment=#86bbdd, ControlModule=#a9daee
- Config JSON files in `generalFunctions/src/configs/` define defaults, types, enums per node
- Tick loop runs at 1000ms intervals for time-based updates
- Three outputs per node: [process, dbase, parent]
## Development Notes
- No build step required - pure Node.js
- Install: `npm install` in root
- Submodule URLs were rewritten from `gitea.centraal.wbd-rd.nl` to `gitea.wbd-rd.nl` for external access
- Dependencies: mathjs, generalFunctions (git submodule)

20
Dockerfile Normal file
View File

@@ -0,0 +1,20 @@
FROM node:20-slim
WORKDIR /app
# Copy package files first for layer caching
COPY package.json package-lock.json ./
# Copy generalFunctions submodule (needed as local dependency)
COPY nodes/generalFunctions/ ./nodes/generalFunctions/
# Rewrite git+https dependency to local path (avoids needing Gitea credentials)
RUN sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
RUN npm install --ignore-scripts
# Copy full source
COPY . .
# Default: run full CI suite
CMD ["npm", "run", "ci"]

18
Dockerfile.e2e Normal file
View File

@@ -0,0 +1,18 @@
FROM nodered/node-red:latest
WORKDIR /usr/src/node-red
# Copy package files and nodes source
COPY package.json ./
COPY nodes/ ./nodes/
# Rewrite generalFunctions dependency from git+https to local file path
RUN sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
# Install dependencies (ignore scripts to avoid postinstall git checkout)
RUN npm install --ignore-scripts
# Copy test flows into Node-RED data directory
COPY test/e2e/flows.json /data/flows.json
EXPOSE 1880

50
Makefile Normal file
View File

@@ -0,0 +1,50 @@
.PHONY: install lint lint-fix test test-jest test-node test-legacy ci docker-ci docker-test docker-lint e2e e2e-up e2e-down
install:
@sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
npm install
@git checkout -- package.json 2>/dev/null || true
lint:
npx eslint nodes/
lint-fix:
npx eslint nodes/ --fix
test-jest:
npx jest --forceExit
test-node:
node --test \
nodes/valve/test/basic/*.test.js \
nodes/valve/test/edge/*.test.js \
nodes/valve/test/integration/*.test.js \
nodes/valveGroupControl/test/basic/*.test.js \
nodes/valveGroupControl/test/edge/*.test.js \
nodes/valveGroupControl/test/integration/*.test.js
test-legacy:
node nodes/machineGroupControl/src/groupcontrol.test.js
node nodes/generalFunctions/src/nrmse/errorMetric.test.js
test: test-jest test-node test-legacy
ci: lint test
docker-ci:
docker compose run --rm ci
docker-test:
docker compose run --rm test
docker-lint:
docker compose run --rm lint
e2e:
bash test/e2e/run-e2e.sh
e2e-up:
docker compose -f docker-compose.e2e.yml up -d --build
e2e-down:
docker compose -f docker-compose.e2e.yml down

49
docker-compose.e2e.yml Normal file
View File

@@ -0,0 +1,49 @@
services:
influxdb:
image: influxdb:2.7
environment:
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=admin
- DOCKER_INFLUXDB_INIT_PASSWORD=adminpassword
- DOCKER_INFLUXDB_INIT_ORG=evolv
- DOCKER_INFLUXDB_INIT_BUCKET=evolv
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=evolv-e2e-token
ports:
- "8086:8086"
healthcheck:
test: ["CMD", "influx", "ping"]
interval: 5s
timeout: 5s
retries: 5
nodered:
build:
context: .
dockerfile: Dockerfile.e2e
ports:
- "1880:1880"
depends_on:
influxdb:
condition: service_healthy
environment:
- INFLUXDB_URL=http://influxdb:8086
- INFLUXDB_TOKEN=evolv-e2e-token
- INFLUXDB_ORG=evolv
- INFLUXDB_BUCKET=evolv
volumes:
- ./test/e2e/flows.json:/data/flows.json
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:1880/"]
interval: 5s
timeout: 5s
retries: 10
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
- GF_AUTH_ANONYMOUS_ENABLED=true
depends_on:
- influxdb

39
docker-compose.yml Normal file
View File

@@ -0,0 +1,39 @@
services:
ci:
build:
context: .
dockerfile: Dockerfile
command: npm run ci
environment:
- NODE_ENV=test
- CI=true
lint:
build:
context: .
dockerfile: Dockerfile
command: npm run lint
test:
build:
context: .
dockerfile: Dockerfile
command: npm run test:all
test-jest:
build:
context: .
dockerfile: Dockerfile
command: npm test
test-node:
build:
context: .
dockerfile: Dockerfile
command: npm run test:node
test-legacy:
build:
context: .
dockerfile: Dockerfile
command: npm run test:legacy

418
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,418 @@
# EVOLV Architecture
## 1. System Overview
High-level view of how EVOLV fits into the wastewater treatment automation stack.
```mermaid
graph LR
NR[Node-RED Runtime] <-->|msg objects| EVOLV[EVOLV Nodes]
EVOLV -->|InfluxDB line protocol| INFLUX[(InfluxDB)]
INFLUX -->|queries| GRAFANA[Grafana Dashboards]
EVOLV -->|process output| NR
EVOLV -->|parent output| NR
style NR fill:#b22222,color:#fff
style EVOLV fill:#0f52a5,color:#fff
style INFLUX fill:#0c99d9,color:#fff
style GRAFANA fill:#50a8d9,color:#fff
```
Each EVOLV node produces three outputs:
| Port | Name | Purpose |
|------|------|---------|
| 0 | process | Process data forwarded to downstream nodes |
| 1 | dbase | InfluxDB-formatted measurement data |
| 2 | parent | Control messages to parent nodes (e.g. registerChild) |
---
## 2. Node Architecture (Three-Layer Pattern)
Every node follows a consistent three-layer design that separates Node-RED wiring from domain logic.
```mermaid
graph TB
subgraph "Node-RED Runtime"
REG["RED.nodes.registerType()"]
end
subgraph "Layer 1 — Wrapper (valve.js)"
W[wrapper .js]
W -->|"new nodeClass(config, RED, this, name)"| NC
W -->|MenuManager| MENU[HTTP /name/menu.js]
W -->|configManager| CFG[HTTP /name/configData.js]
end
subgraph "Layer 2 — Node Adapter (src/nodeClass.js)"
NC[nodeClass]
NC -->|_loadConfig| CFGM[configManager]
NC -->|_setupSpecificClass| SC
NC -->|_attachInputHandler| INPUT[onInput routing]
NC -->|_startTickLoop| TICK[1s tick loop]
NC -->|_tick → outputUtils| OUT[formatMsg]
end
subgraph "Layer 3 — Domain Logic (src/specificClass.js)"
SC[specificClass]
SC -->|measurements| MC[MeasurementContainer]
SC -->|state machine| ST[state]
SC -->|hydraulics / biology| DOMAIN[domain models]
end
subgraph "generalFunctions"
GF[shared library]
end
REG --> W
GF -.->|logger, outputUtils, configManager,\nMeasurementContainer, validation, ...| NC
GF -.->|MeasurementContainer, state,\nconvert, predict, ...| SC
style W fill:#0f52a5,color:#fff
style NC fill:#0c99d9,color:#fff
style SC fill:#50a8d9,color:#fff
style GF fill:#86bbdd,color:#000
```
---
## 3. generalFunctions Module Map
The shared library (`nodes/generalFunctions/`) provides all cross-cutting concerns.
```mermaid
graph TB
GF[generalFunctions/index.js]
subgraph "Core Helpers (src/helper/)"
LOGGER[logger]
OUTPUT[outputUtils]
CHILD[childRegistrationUtils]
CFGUTIL[configUtils]
ASSERT[assertionUtils]
VALID[validationUtils]
end
subgraph "Validators (src/helper/validators/)"
TV[typeValidators]
CV[collectionValidators]
CURV[curveValidator]
end
subgraph "Domain Modules (src/)"
MC[MeasurementContainer]
CFGMGR[configManager]
MENUMGR[MenuManager]
STATE[state]
CONVERT[convert / Fysics]
PREDICT[predict / interpolation]
NRMSE[nrmse / errorMetrics]
COOLPROP[coolprop]
end
subgraph "Data (datasets/)"
CURVES[assetData/curves]
ASSETS[assetData/assetData.json]
UNITS[unitData.json]
end
subgraph "Constants (src/constants/)"
POS[POSITIONS / POSITION_VALUES]
end
GF --> LOGGER
GF --> OUTPUT
GF --> CHILD
GF --> CFGUTIL
GF --> ASSERT
GF --> VALID
VALID --> TV
VALID --> CV
VALID --> CURV
GF --> MC
GF --> CFGMGR
GF --> MENUMGR
GF --> STATE
GF --> CONVERT
GF --> PREDICT
GF --> NRMSE
GF --> COOLPROP
GF --> CURVES
GF --> POS
style GF fill:#0f52a5,color:#fff
style LOGGER fill:#86bbdd,color:#000
style OUTPUT fill:#86bbdd,color:#000
style VALID fill:#86bbdd,color:#000
style MC fill:#50a8d9,color:#fff
style CFGMGR fill:#50a8d9,color:#fff
style MENUMGR fill:#50a8d9,color:#fff
```
---
## 4. Data Flow (Message Lifecycle)
Sequence diagram showing a typical input message and the periodic tick output cycle.
```mermaid
sequenceDiagram
participant NR as Node-RED
participant W as wrapper.js
participant NC as nodeClass
participant SC as specificClass
participant OU as outputUtils
Note over W: Node startup
W->>NC: new nodeClass(config, RED, node, name)
NC->>NC: _loadConfig (configManager.buildConfig)
NC->>SC: new specificClass(config, stateConfig, options)
NC->>NR: send([null, null, {topic: registerChild}])
Note over NC: Every 1 second (tick loop)
NC->>SC: getOutput()
SC-->>NC: raw measurement data
NC->>OU: formatMsg(raw, config, 'process')
NC->>OU: formatMsg(raw, config, 'influxdb')
NC->>NR: send([processMsg, influxMsg])
Note over NR: Incoming control message
NR->>W: msg {topic: 'execMovement', payload: {...}}
W->>NC: onInput(msg)
NC->>SC: handleInput(source, action, setpoint)
SC->>SC: update state machine & measurements
```
---
## 5. Node Types
| Node | S88 Level | Purpose |
|------|-----------|---------|
| **measurement** | Control Module | Generic measurement point — reads, validates, and stores sensor values |
| **valve** | Control Module | Valve simulation with hydraulic model, position control, flow/pressure prediction |
| **rotatingMachine** | Control Module | Pumps, blowers, mixers — rotating equipment with speed control and efficiency curves |
| **diffuser** | Control Module | Aeration diffuser — models oxygen transfer and pressure drop |
| **settler** | Equipment | Sludge settler — models settling behavior and sludge blanket |
| **reactor** | Equipment | Hydraulic tank and biological process simulator (activated sludge, digestion) |
| **monster** | Equipment | MONitoring and STrEam Routing — complex measurement aggregation |
| **pumpingStation** | Unit | Coordinates multiple pumps as a pumping station |
| **valveGroupControl** | Unit | Manages multiple valves as a coordinated group — distributes flow, monitors pressure |
| **machineGroupControl** | Unit | Group control for rotating machines — load balancing and sequencing |
| **dashboardAPI** | Utility | Exposes data and unit conversion endpoints for external dashboards |
# EVOLV Architecture
## Node Hierarchy (S88)
EVOLV follows the ISA-88 (S88) batch control standard. Each node maps to an S88 level and uses a consistent color scheme in the Node-RED editor.
```mermaid
graph TD
classDef area fill:#0f52a5,color:#fff,stroke:#0a3d7a
classDef processCell fill:#0c99d9,color:#fff,stroke:#0977aa
classDef unit fill:#50a8d9,color:#fff,stroke:#3d89b3
classDef equipment fill:#86bbdd,color:#000,stroke:#6a9bb8
classDef controlModule fill:#a9daee,color:#000,stroke:#87b8cc
classDef standalone fill:#f0f0f0,color:#000,stroke:#999
%% S88 Levels
subgraph "S88: Area"
PS[pumpingStation]
end
subgraph "S88: Equipment"
MGC[machineGroupControl]
VGC[valveGroupControl]
end
subgraph "S88: Control Module"
RM[rotatingMachine]
V[valve]
M[measurement]
R[reactor]
S[settler]
end
subgraph "Standalone"
MON[monster]
DASH[dashboardAPI]
DIFF[diffuser - not implemented]
end
%% Parent-child registration relationships
PS -->|"accepts: measurement"| M
PS -->|"accepts: machine"| RM
PS -->|"accepts: machineGroup"| MGC
PS -->|"accepts: pumpingStation"| PS2[pumpingStation]
MGC -->|"accepts: machine"| RM
RM -->|"accepts: measurement"| M2[measurement]
RM -->|"accepts: reactor"| R
VGC -->|"accepts: valve"| V
VGC -->|"accepts: machine / rotatingmachine"| RM2[rotatingMachine]
VGC -->|"accepts: machinegroup / machinegroupcontrol"| MGC2[machineGroupControl]
VGC -->|"accepts: pumpingstation / valvegroupcontrol"| PS3["pumpingStation / valveGroupControl"]
R -->|"accepts: measurement"| M3[measurement]
R -->|"accepts: reactor"| R2[reactor]
S -->|"accepts: measurement"| M4[measurement]
S -->|"accepts: reactor"| R3[reactor]
S -->|"accepts: machine"| RM3[rotatingMachine]
%% Styling
class PS,PS2,PS3 area
class MGC,MGC2 equipment
class VGC equipment
class RM,RM2,RM3 controlModule
class V controlModule
class M,M2,M3,M4 controlModule
class R,R2,R3 controlModule
class S controlModule
class MON,DASH,DIFF standalone
```
### Registration Summary
```mermaid
graph LR
classDef parent fill:#0c99d9,color:#fff
classDef child fill:#a9daee,color:#000
PS[pumpingStation] -->|measurement| LEAF1((leaf))
PS -->|machine| RM1[rotatingMachine]
PS -->|machineGroup| MGC1[machineGroupControl]
PS -->|pumpingStation| PS1[pumpingStation]
MGC[machineGroupControl] -->|machine| RM2[rotatingMachine]
VGC[valveGroupControl] -->|valve| V1[valve]
VGC -->|source| SRC["machine, machinegroup,<br/>pumpingstation, valvegroupcontrol"]
RM[rotatingMachine] -->|measurement| LEAF2((leaf))
RM -->|reactor| R1[reactor]
R[reactor] -->|measurement| LEAF3((leaf))
R -->|reactor| R2[reactor]
S[settler] -->|measurement| LEAF4((leaf))
S -->|reactor| R3[reactor]
S -->|machine| RM3[rotatingMachine]
class PS,MGC,VGC,RM,R,S parent
class LEAF1,LEAF2,LEAF3,LEAF4,RM1,RM2,RM3,MGC1,PS1,V1,SRC,R1,R2,R3 child
```
## Node Types
| Node | S88 Level | softwareType | role | Accepts Children | Outputs |
|------|-----------|-------------|------|-----------------|---------|
| **pumpingStation** | Area | `pumpingstation` | StationController | measurement, machine (rotatingMachine), machineGroup, pumpingStation | [process, dbase, parent] |
| **machineGroupControl** | Equipment | `machinegroupcontrol` | GroupController | machine (rotatingMachine) | [process, dbase, parent] |
| **valveGroupControl** | Equipment | `valvegroupcontrol` | ValveGroupController | valve, machine, rotatingmachine, machinegroup, machinegroupcontrol, pumpingstation, valvegroupcontrol | [process, dbase, parent] |
| **rotatingMachine** | Control Module | `rotatingmachine` | RotationalDeviceController | measurement, reactor | [process, dbase, parent] |
| **valve** | Control Module | `valve` | controller | _(leaf node, no children)_ | [process, dbase, parent] |
| **measurement** | Control Module | `measurement` | Sensor | _(leaf node, no children)_ | [process, dbase, parent] |
| **reactor** | Control Module | `reactor` | Biological reactor | measurement, reactor (upstream chaining) | [process, dbase, parent] |
| **settler** | Control Module | `settler` | Secondary settler | measurement, reactor (upstream), machine (return pump) | [process, dbase, parent] |
| **monster** | Standalone | - | - | dual-parent, standalone | - |
| **dashboardAPI** | Standalone | - | - | accepts any child (Grafana integration) | - |
| **diffuser** | Standalone | - | - | _(not implemented)_ | - |
## Data Flow
### Measurement Data Flow (upstream to downstream)
```mermaid
sequenceDiagram
participant Sensor as measurement (sensor)
participant Machine as rotatingMachine
participant Group as machineGroupControl
participant Station as pumpingStation
Note over Sensor: Sensor reads value<br/>(pressure, flow, level, temp)
Sensor->>Sensor: measurements.type(t).variant("measured").position(p).value(v)
Sensor->>Sensor: emitter.emit("type.measured.position", eventData)
Sensor->>Machine: Event: "pressure.measured.upstream"
Machine->>Machine: Store in own MeasurementContainer
Machine->>Machine: getMeasuredPressure() -> calcFlow() -> calcPower()
Machine->>Machine: emitter.emit("flow.predicted.downstream", eventData)
Machine->>Group: Event: "flow.predicted.downstream"
Group->>Group: handlePressureChange()
Group->>Group: Aggregate flows across all machines
Group->>Group: Calculate group totals and efficiency
Machine->>Station: Event: "flow.predicted.downstream"
Station->>Station: Store predicted flow in/out
Station->>Station: _updateVolumePrediction()
Station->>Station: _calcNetFlow(), _calcTimeRemaining()
```
### Control Command Flow (downstream to upstream)
```mermaid
sequenceDiagram
participant Station as pumpingStation
participant Group as machineGroupControl
participant Machine as rotatingMachine
participant Machine2 as rotatingMachine (2)
Station->>Group: handleInput("parent", action, param)
Group->>Group: Determine scaling strategy
Group->>Group: Calculate setpoints per machine
Group->>Machine: handleInput("parent", "execMovement", setpoint)
Group->>Machine2: handleInput("parent", "execMovement", setpoint)
Machine->>Machine: setpoint() -> state.moveTo(pos)
Machine->>Machine: updatePosition() -> calcFlow(), calcPower()
Machine->>Machine: emitter.emit("flow.predicted.downstream")
Machine2->>Machine2: setpoint() -> state.moveTo(pos)
Machine2->>Machine2: updatePosition() -> calcFlow(), calcPower()
Machine2->>Machine2: emitter.emit("flow.predicted.downstream")
```
### Wastewater Treatment Process Flow
```mermaid
graph LR
classDef process fill:#50a8d9,color:#fff
classDef equipment fill:#86bbdd,color:#000
PS_IN[pumpingStation<br/>Influent] -->|flow| R1[reactor<br/>Anoxic]
R1 -->|effluent| R2[reactor<br/>Aerated]
R2 -->|effluent| SET[settler]
SET -->|effluent out| PS_OUT[pumpingStation<br/>Effluent]
SET -->|sludge return| RM_RET[rotatingMachine<br/>Return pump]
RM_RET -->|recirculation| R1
PS_IN --- MGC_IN[machineGroupControl]
MGC_IN --- RM_IN[rotatingMachine<br/>Influent pumps]
class PS_IN,PS_OUT process
class R1,R2,SET process
class MGC_IN,RM_IN,RM_RET equipment
```
### Event-Driven Communication Pattern
All parent-child communication uses Node.js `EventEmitter`:
1. **Registration**: Parent calls `childRegistrationUtils.registerChild(child, position)` which stores the child and calls the parent's `registerChild(child, softwareType)` method.
2. **Event binding**: The parent's `registerChild()` subscribes to the child's `measurements.emitter` events (e.g., `"flow.predicted.downstream"`).
3. **Data propagation**: When a child updates a measurement, it emits an event. The parent's listener stores the value in its own `MeasurementContainer` and runs its domain logic.
4. **Three outputs**: Every node sends data to three Node-RED outputs: `[process, dbase, parent]` -- process data for downstream nodes, InfluxDB for persistence, and parent aggregation data.
### Position Convention
Children register with a position relative to their parent:
- `upstream` -- before the parent in the flow direction
- `downstream` -- after the parent in the flow direction
- `atEquipment` -- physically located at/on the parent equipment

29
eslint.config.js Normal file
View File

@@ -0,0 +1,29 @@
const js = require('@eslint/js');
const globals = require('globals');
module.exports = [
js.configs.recommended,
{
languageOptions: {
ecmaVersion: 2022,
sourceType: 'commonjs',
globals: {
...globals.node,
...globals.jest,
RED: 'readonly',
},
},
rules: {
'no-unused-vars': ['warn', { argsIgnorePattern: '^_', varsIgnorePattern: '^_' }],
'no-console': 'off',
'no-prototype-builtins': 'warn',
'no-constant-condition': 'warn',
},
},
{
ignores: [
'node_modules/**',
'nodes/generalFunctions/src/coolprop-node/coolprop/**',
],
},
];

12
jest.config.js Normal file
View File

@@ -0,0 +1,12 @@
module.exports = {
testEnvironment: 'node',
verbose: true,
testMatch: [
'<rootDir>/nodes/generalFunctions/src/coolprop-node/test/**/*.test.js',
'<rootDir>/nodes/generalFunctions/test/**/*.test.js',
],
testPathIgnorePatterns: [
'/node_modules/',
],
testTimeout: 15000,
};

4250
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -25,8 +25,26 @@
},
"author": "Rene De Ren, Pim Moerman, Janneke Tack, Sjoerd Fijnje, Dieke Gabriels, pieter van der wilt",
"license": "SEE LICENSE",
"scripts": {
"preinstall": "node scripts/patch-deps.js",
"postinstall": "git checkout -- package.json 2>/dev/null || true",
"test": "jest --forceExit",
"test:node": "node --test nodes/valve/test/basic/*.test.js nodes/valve/test/edge/*.test.js nodes/valve/test/integration/*.test.js nodes/valveGroupControl/test/basic/*.test.js nodes/valveGroupControl/test/edge/*.test.js nodes/valveGroupControl/test/integration/*.test.js",
"test:legacy": "node nodes/machineGroupControl/src/groupcontrol.test.js && node nodes/generalFunctions/src/nrmse/errorMetric.test.js",
"test:all": "npm test && npm run test:node && npm run test:legacy",
"lint": "eslint nodes/",
"lint:fix": "eslint nodes/ --fix",
"ci": "npm run lint && npm run test:all",
"test:e2e": "bash test/e2e/run-e2e.sh"
},
"dependencies": {
"generalFunctions": "git+https://gitea.centraal.wbd-rd.nl/RnD/generalFunctions.git",
"generalFunctions": "file:nodes/generalFunctions",
"mathjs": "^13.2.0"
},
"devDependencies": {
"@eslint/js": "^8.57.0",
"eslint": "^8.57.0",
"globals": "^15.0.0",
"jest": "^29.7.0"
}
}

20
scripts/patch-deps.js Normal file
View File

@@ -0,0 +1,20 @@
/**
* Preinstall script: rewrites the generalFunctions dependency
* from git+https to a local file path when the submodule exists.
* This avoids needing Gitea credentials during npm install.
*/
const fs = require('fs');
const path = require('path');
const pkgPath = path.join(__dirname, '..', 'package.json');
const localGF = path.join(__dirname, '..', 'nodes', 'generalFunctions');
if (fs.existsSync(localGF) && fs.existsSync(path.join(localGF, 'index.js'))) {
const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf8'));
if (pkg.dependencies && pkg.dependencies.generalFunctions &&
pkg.dependencies.generalFunctions.startsWith('git+')) {
pkg.dependencies.generalFunctions = 'file:./nodes/generalFunctions';
fs.writeFileSync(pkgPath, JSON.stringify(pkg, null, 2) + '\n');
console.log('[patch-deps] Rewrote generalFunctions to local path');
}
}

71
test/e2e/flows.json Normal file
View File

@@ -0,0 +1,71 @@
[
{
"id": "e2e-flow-tab",
"type": "tab",
"label": "E2E Test Flow",
"disabled": false,
"info": "End-to-end test flow that verifies EVOLV nodes are loaded and messages can flow through the pipeline."
},
{
"id": "inject-trigger",
"type": "inject",
"z": "e2e-flow-tab",
"name": "Trigger after 2s",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": true,
"onceDelay": "2",
"topic": "e2e-test",
"payload": "",
"payloadType": "date",
"x": 150,
"y": 100,
"wires": [
["simulate-measurement"]
]
},
{
"id": "simulate-measurement",
"type": "function",
"z": "e2e-flow-tab",
"name": "Simulate Measurement",
"func": "msg.payload = {\n measurement: 'e2e_test',\n tags: { source: 'evolv-e2e', node: 'measurement-sim' },\n fields: { value: Math.random() * 100, status: 1 },\n timestamp: Date.now()\n};\nmsg.topic = 'e2e/measurement';\nnode.status({ fill: 'green', shape: 'dot', text: 'sent' });\nreturn msg;",
"outputs": 1,
"timeout": "",
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 370,
"y": 100,
"wires": [
["debug-output"]
]
},
{
"id": "debug-output",
"type": "debug",
"z": "e2e-flow-tab",
"name": "E2E Debug Output",
"active": true,
"tosidebar": true,
"console": true,
"tostatus": true,
"complete": "true",
"targetType": "full",
"statusVal": "",
"statusType": "auto",
"x": 580,
"y": 100,
"wires": []
}
]

142
test/e2e/run-e2e.sh Executable file
View File

@@ -0,0 +1,142 @@
#!/usr/bin/env bash
#
# End-to-end test runner for EVOLV Node-RED stack.
# Starts Node-RED + InfluxDB + Grafana via Docker Compose,
# verifies that EVOLV nodes are registered in the palette,
# and tears down the stack on exit.
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
COMPOSE_FILE="$PROJECT_ROOT/docker-compose.e2e.yml"
NODERED_URL="http://localhost:1880"
MAX_WAIT=120 # seconds to wait for Node-RED to become healthy
# EVOLV node types that must appear in the palette (from package.json node-red.nodes)
EXPECTED_NODES=(
"dashboardapi"
"machineGroupControl"
"measurement"
"monster"
"pumpingstation"
"reactor"
"rotatingMachine"
"settler"
"valve"
"valveGroupControl"
)
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
# Determine docker compose command (handle permission via sg docker if needed)
DOCKER_COMPOSE="docker compose"
if ! docker info >/dev/null 2>&1; then
if sg docker -c "docker info" >/dev/null 2>&1; then
DOCKER_COMPOSE="sg docker -c 'docker compose'"
log_info "Using sg docker for Docker access"
else
log_error "Docker is not accessible. Please ensure Docker is running and you have permissions."
exit 1
fi
fi
cleanup() {
log_info "Tearing down E2E stack..."
eval $DOCKER_COMPOSE -f "$COMPOSE_FILE" down --volumes --remove-orphans 2>/dev/null || true
}
# Always clean up on exit
trap cleanup EXIT
# --- Step 1: Build and start the stack ---
log_info "Building and starting E2E stack..."
eval $DOCKER_COMPOSE -f "$COMPOSE_FILE" up -d --build
# --- Step 2: Wait for Node-RED to be healthy ---
log_info "Waiting for Node-RED to become healthy (max ${MAX_WAIT}s)..."
elapsed=0
while [ $elapsed -lt $MAX_WAIT ]; do
if curl -sf "$NODERED_URL/" >/dev/null 2>&1; then
log_info "Node-RED is up after ${elapsed}s"
break
fi
sleep 2
elapsed=$((elapsed + 2))
done
if [ $elapsed -ge $MAX_WAIT ]; then
log_error "Node-RED did not become healthy within ${MAX_WAIT}s"
log_error "Container logs:"
eval $DOCKER_COMPOSE -f "$COMPOSE_FILE" logs nodered
exit 1
fi
# Give Node-RED a few extra seconds to finish loading all nodes
sleep 5
# --- Step 3: Verify EVOLV nodes are registered in the palette ---
log_info "Querying Node-RED for registered nodes..."
NODES_RESPONSE=$(curl -sf "$NODERED_URL/nodes" 2>&1) || {
log_error "Failed to query Node-RED /nodes endpoint"
exit 1
}
FAILURES=0
for node_type in "${EXPECTED_NODES[@]}"; do
if echo "$NODES_RESPONSE" | grep -qi "$node_type"; then
log_info " [PASS] Node type '$node_type' found in palette"
else
log_error " [FAIL] Node type '$node_type' NOT found in palette"
FAILURES=$((FAILURES + 1))
fi
done
# --- Step 4: Verify flows are deployed ---
log_info "Checking deployed flows..."
FLOWS_RESPONSE=$(curl -sf "$NODERED_URL/flows" 2>&1) || {
log_error "Failed to query Node-RED /flows endpoint"
exit 1
}
if echo "$FLOWS_RESPONSE" | grep -q "e2e-flow-tab"; then
log_info " [PASS] E2E test flow is deployed"
else
log_warn " [WARN] E2E test flow not found in deployed flows (may need manual deploy)"
fi
# --- Step 5: Verify InfluxDB is reachable ---
log_info "Checking InfluxDB health..."
INFLUX_HEALTH=$(curl -sf "http://localhost:8086/health" 2>&1) || {
log_error "Failed to reach InfluxDB health endpoint"
FAILURES=$((FAILURES + 1))
}
if echo "$INFLUX_HEALTH" | grep -q '"status":"pass"'; then
log_info " [PASS] InfluxDB is healthy"
else
log_error " [FAIL] InfluxDB health check failed"
FAILURES=$((FAILURES + 1))
fi
# --- Step 6: Summary ---
echo ""
if [ $FAILURES -eq 0 ]; then
log_info "========================================="
log_info " E2E tests PASSED - all checks green"
log_info "========================================="
exit 0
else
log_error "========================================="
log_error " E2E tests FAILED - $FAILURES check(s) failed"
log_error "========================================="
exit 1
fi