Compare commits
28 Commits
dev-rene
...
7ded2a4415
| Author | SHA1 | Date | |
|---|---|---|---|
|
7ded2a4415 | ||
|
|
6d19038784 | ||
|
|
fd9d1679cb | ||
|
|
4336002b77 | ||
|
|
f57343f5e3 | ||
|
|
65ceb696ab | ||
|
|
91a298960c | ||
|
|
35221fc5dd | ||
|
|
93a5b6a90e | ||
|
|
1d98670706 | ||
|
|
a432eea7fe | ||
|
|
9cb3657bae | ||
|
|
bd9432eebb | ||
|
|
c9bacb64c8 | ||
|
|
e580c93c84 | ||
|
|
b02306c42f | ||
|
|
2c76430394 | ||
|
|
49ebd833db | ||
|
|
905a061590 | ||
|
|
80de324b32 | ||
|
|
c8d5ea0fce | ||
|
|
b871b23c24 | ||
|
|
91b681a74d | ||
|
|
76d2008e52 | ||
|
|
3c304f14e5 | ||
|
|
24c443840b | ||
|
|
c4c8629c01 | ||
| 609c72cedc |
@@ -0,0 +1,43 @@
|
|||||||
|
## Context
|
||||||
|
|
||||||
|
The single demo bioreactor did not reflect the intended EVOLV biological treatment concept. The owner requested:
|
||||||
|
|
||||||
|
- four reactor zones in series
|
||||||
|
- staged aeration based on effluent NH4
|
||||||
|
- local visualization per zone for NH4, NO3, O2, and other relevant state variables
|
||||||
|
- improved PFR numerical stability by increasing reactor resolution
|
||||||
|
|
||||||
|
The localhost deployment also needed to remain usable for E2E debugging with Node-RED, InfluxDB, and Grafana.
|
||||||
|
|
||||||
|
## Options Considered
|
||||||
|
|
||||||
|
1. Keep one large PFR and add more internal profile visualization only.
|
||||||
|
2. Split the biology into four explicit reactor zones in the flow and control aeration at zone level.
|
||||||
|
3. Replace the PFR demo with a simpler CSTR train for faster visual response.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Choose option 2.
|
||||||
|
|
||||||
|
The demo flow now uses four explicit PFR zones in series with:
|
||||||
|
|
||||||
|
- equal-zone sizing (`4 x 500 m3`, total `2000 m3`)
|
||||||
|
- explicit `Fluent` forwarding between zones
|
||||||
|
- common clocking for all zones
|
||||||
|
- external `OTR` control instead of fixed `kla`
|
||||||
|
- staged NH4-based aeration escalation with 30-minute hold logic
|
||||||
|
- per-zone telemetry to InfluxDB and Node-RED dashboard charts
|
||||||
|
|
||||||
|
For runtime stability on localhost, the demo uses a higher spatial resolution with moderate compute load rather than the earlier single-reactor setup.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The flow is easier to reason about operationally because each aeration zone is explicit.
|
||||||
|
- Zone-level telemetry is available for dashboarding and debugging.
|
||||||
|
- PFR outlet response remains residence-time dependent, so zone outlet composition will not change instantly after startup or inflow changes.
|
||||||
|
- Grafana datasource query round-trip remains valid, but dashboard auto-generation still needs separate follow-up if strict dashboard creation is required in E2E checks.
|
||||||
|
|
||||||
|
## Rollback / Migration Notes
|
||||||
|
|
||||||
|
- Rolling back to the earlier demo means restoring the single `demo_reactor` topology in `docker/demo-flow.json`.
|
||||||
|
- Existing E2E checks and dashboards should prefer the explicit zone measurements (`reactor_demo_reactor_z1` ... `reactor_demo_reactor_z4`) going forward.
|
||||||
41
.gitea/workflows/ci.yml
Normal file
41
.gitea/workflows/ci.yml
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main, develop, dev-Rene]
|
||||||
|
pull_request:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
lint-and-test:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
container:
|
||||||
|
image: node:20-slim
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Install git
|
||||||
|
run: apt-get update -qq && apt-get install -y -qq git
|
||||||
|
|
||||||
|
- name: Checkout with submodules
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
submodules: recursive
|
||||||
|
|
||||||
|
- name: Rewrite generalFunctions to local path
|
||||||
|
run: |
|
||||||
|
sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: npm install --ignore-scripts
|
||||||
|
|
||||||
|
- name: Lint
|
||||||
|
run: npm run lint
|
||||||
|
|
||||||
|
- name: Test (Jest)
|
||||||
|
run: npm test
|
||||||
|
|
||||||
|
- name: Test (node:test)
|
||||||
|
run: npm run test:node
|
||||||
|
|
||||||
|
- name: Test (legacy)
|
||||||
|
run: npm run test:legacy
|
||||||
74
.gitmodules
vendored
74
.gitmodules
vendored
@@ -1,37 +1,37 @@
|
|||||||
|
|
||||||
[submodule "nodes/machineGroupControl"]
|
[submodule "nodes/machineGroupControl"]
|
||||||
path = nodes/machineGroupControl
|
path = nodes/machineGroupControl
|
||||||
url = https://gitea.wbd-rd.nl/RnD/machineGroupControl.git
|
url = https://gitea.wbd-rd.nl/RnD/machineGroupControl.git
|
||||||
[submodule "nodes/generalFunctions"]
|
[submodule "nodes/generalFunctions"]
|
||||||
path = nodes/generalFunctions
|
path = nodes/generalFunctions
|
||||||
url = https://gitea.wbd-rd.nl/RnD/generalFunctions.git
|
url = https://gitea.wbd-rd.nl/RnD/generalFunctions.git
|
||||||
[submodule "nodes/valveGroupControl"]
|
[submodule "nodes/valveGroupControl"]
|
||||||
path = nodes/valveGroupControl
|
path = nodes/valveGroupControl
|
||||||
url = https://gitea.wbd-rd.nl/RnD/valveGroupControl.git
|
url = https://gitea.wbd-rd.nl/RnD/valveGroupControl.git
|
||||||
[submodule "nodes/valve"]
|
[submodule "nodes/valve"]
|
||||||
path = nodes/valve
|
path = nodes/valve
|
||||||
url = https://gitea.wbd-rd.nl/RnD/valve.git
|
url = https://gitea.wbd-rd.nl/RnD/valve.git
|
||||||
[submodule "nodes/rotatingMachine"]
|
[submodule "nodes/rotatingMachine"]
|
||||||
path = nodes/rotatingMachine
|
path = nodes/rotatingMachine
|
||||||
url = https://gitea.wbd-rd.nl/RnD/rotatingMachine.git
|
url = https://gitea.wbd-rd.nl/RnD/rotatingMachine.git
|
||||||
[submodule "nodes/monster"]
|
[submodule "nodes/monster"]
|
||||||
path = nodes/monster
|
path = nodes/monster
|
||||||
url = https://gitea.wbd-rd.nl/RnD/monster.git
|
url = https://gitea.wbd-rd.nl/RnD/monster.git
|
||||||
[submodule "nodes/measurement"]
|
[submodule "nodes/measurement"]
|
||||||
path = nodes/measurement
|
path = nodes/measurement
|
||||||
url = https://gitea.wbd-rd.nl/RnD/measurement.git
|
url = https://gitea.wbd-rd.nl/RnD/measurement.git
|
||||||
[submodule "nodes/diffuser"]
|
[submodule "nodes/diffuser"]
|
||||||
path = nodes/diffuser
|
path = nodes/diffuser
|
||||||
url = https://gitea.wbd-rd.nl/RnD/diffuser.git
|
url = https://gitea.wbd-rd.nl/RnD/diffuser.git
|
||||||
[submodule "nodes/dashboardAPI"]
|
[submodule "nodes/dashboardAPI"]
|
||||||
path = nodes/dashboardAPI
|
path = nodes/dashboardAPI
|
||||||
url = https://gitea.wbd-rd.nl/RnD/dashboardAPI.git
|
url = https://gitea.wbd-rd.nl/RnD/dashboardAPI.git
|
||||||
[submodule "nodes/reactor"]
|
[submodule "nodes/reactor"]
|
||||||
path = nodes/reactor
|
path = nodes/reactor
|
||||||
url = https://gitea.wbd-rd.nl/RnD/reactor.git
|
url = https://gitea.wbd-rd.nl/RnD/reactor.git
|
||||||
[submodule "nodes/pumpingStation"]
|
[submodule "nodes/pumpingStation"]
|
||||||
path = nodes/pumpingStation
|
path = nodes/pumpingStation
|
||||||
url = https://gitea.wbd-rd.nl/RnD/pumpingStation
|
url = https://gitea.wbd-rd.nl/RnD/pumpingStation
|
||||||
[submodule "nodes/settler"]
|
[submodule "nodes/settler"]
|
||||||
path = nodes/settler
|
path = nodes/settler
|
||||||
url = https://gitea.wbd-rd.nl/RnD/settler.git
|
url = https://gitea.wbd-rd.nl/RnD/settler.git
|
||||||
|
|||||||
32
CLAUDE.md
Normal file
32
CLAUDE.md
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
# EVOLV - Claude Code Project Guide
|
||||||
|
|
||||||
|
## What This Is
|
||||||
|
Node-RED custom nodes package for wastewater treatment plant automation. Developed by Waterschap Brabantse Delta R&D team. Follows ISA-88 (S88) batch control standard.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
Each node follows a three-layer pattern:
|
||||||
|
1. **Node-RED wrapper** (`<name>.js`) - registers the node type, sets up HTTP endpoints
|
||||||
|
2. **Node adapter** (`src/nodeClass.js`) - bridges Node-RED API with domain logic, handles config loading, tick loops, events
|
||||||
|
3. **Domain logic** (`src/specificClass.js`) - pure business logic, no Node-RED dependencies
|
||||||
|
|
||||||
|
## Key Shared Library: `nodes/generalFunctions/`
|
||||||
|
- `logger` - structured logging (use this, NOT console.log)
|
||||||
|
- `MeasurementContainer` - chainable measurement storage (type/variant/position)
|
||||||
|
- `configManager` - loads JSON configs from `src/configs/`
|
||||||
|
- `MenuManager` - dynamic UI dropdowns
|
||||||
|
- `outputUtils` - formats messages for InfluxDB and process outputs
|
||||||
|
- `childRegistrationUtils` - parent-child node relationships
|
||||||
|
- `coolprop` - thermodynamic property calculations
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
- Nodes register under category `'EVOLV'` in Node-RED
|
||||||
|
- S88 color scheme: Area=#0f52a5, ProcessCell=#0c99d9, Unit=#50a8d9, Equipment=#86bbdd, ControlModule=#a9daee
|
||||||
|
- Config JSON files in `generalFunctions/src/configs/` define defaults, types, enums per node
|
||||||
|
- Tick loop runs at 1000ms intervals for time-based updates
|
||||||
|
- Three outputs per node: [process, dbase, parent]
|
||||||
|
|
||||||
|
## Development Notes
|
||||||
|
- No build step required - pure Node.js
|
||||||
|
- Install: `npm install` in root
|
||||||
|
- Submodule URLs were rewritten from `gitea.centraal.wbd-rd.nl` to `gitea.wbd-rd.nl` for external access
|
||||||
|
- Dependencies: mathjs, generalFunctions (git submodule)
|
||||||
29
Dockerfile.e2e
Normal file
29
Dockerfile.e2e
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
FROM nodered/node-red:latest
|
||||||
|
|
||||||
|
# Switch to root for setup
|
||||||
|
USER root
|
||||||
|
|
||||||
|
# Copy EVOLV directly into where Node-RED looks for custom nodes
|
||||||
|
COPY package.json /data/node_modules/EVOLV/package.json
|
||||||
|
COPY nodes/ /data/node_modules/EVOLV/nodes/
|
||||||
|
|
||||||
|
# Rewrite generalFunctions dependency to local file path (no-op if already local)
|
||||||
|
RUN sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' \
|
||||||
|
/data/node_modules/EVOLV/package.json
|
||||||
|
|
||||||
|
# Fix ownership for node-red user
|
||||||
|
RUN chown -R node-red:root /data
|
||||||
|
|
||||||
|
USER node-red
|
||||||
|
|
||||||
|
# Install EVOLV's own dependencies inside the EVOLV package directory
|
||||||
|
WORKDIR /data/node_modules/EVOLV
|
||||||
|
RUN npm install --ignore-scripts --production
|
||||||
|
|
||||||
|
# Copy test flows into Node-RED data directory
|
||||||
|
COPY --chown=node-red:root test/e2e/flows.json /data/flows.json
|
||||||
|
|
||||||
|
# Reset workdir to Node-RED default
|
||||||
|
WORKDIR /usr/src/node-red
|
||||||
|
|
||||||
|
EXPOSE 1880
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
# Functional Issues Backlog (Deprecated Location)
|
|
||||||
|
|
||||||
This backlog has moved to:
|
|
||||||
- `.agents/improvements/IMPROVEMENTS_BACKLOG.md`
|
|
||||||
|
|
||||||
Use `.agents/improvements/TOP10_PRODUCTION_PRIORITIES_YYYY-MM-DD.md` for ranked review lists.
|
|
||||||
50
Makefile
Normal file
50
Makefile
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
.PHONY: install lint lint-fix test test-jest test-node test-legacy ci docker-ci docker-test docker-lint e2e e2e-up e2e-down
|
||||||
|
|
||||||
|
install:
|
||||||
|
@sed -i 's|"generalFunctions": "git+https://[^"]*"|"generalFunctions": "file:./nodes/generalFunctions"|' package.json
|
||||||
|
npm install
|
||||||
|
@git checkout -- package.json 2>/dev/null || true
|
||||||
|
|
||||||
|
lint:
|
||||||
|
npx eslint nodes/
|
||||||
|
|
||||||
|
lint-fix:
|
||||||
|
npx eslint nodes/ --fix
|
||||||
|
|
||||||
|
test-jest:
|
||||||
|
npx jest --forceExit
|
||||||
|
|
||||||
|
test-node:
|
||||||
|
node --test \
|
||||||
|
nodes/valve/test/basic/*.test.js \
|
||||||
|
nodes/valve/test/edge/*.test.js \
|
||||||
|
nodes/valve/test/integration/*.test.js \
|
||||||
|
nodes/valveGroupControl/test/basic/*.test.js \
|
||||||
|
nodes/valveGroupControl/test/edge/*.test.js \
|
||||||
|
nodes/valveGroupControl/test/integration/*.test.js
|
||||||
|
|
||||||
|
test-legacy:
|
||||||
|
node nodes/machineGroupControl/src/groupcontrol.test.js
|
||||||
|
node nodes/generalFunctions/src/nrmse/errorMetric.test.js
|
||||||
|
|
||||||
|
test: test-jest test-node test-legacy
|
||||||
|
|
||||||
|
ci: lint test
|
||||||
|
|
||||||
|
docker-ci:
|
||||||
|
docker compose run --rm ci
|
||||||
|
|
||||||
|
docker-test:
|
||||||
|
docker compose run --rm test
|
||||||
|
|
||||||
|
docker-lint:
|
||||||
|
docker compose run --rm lint
|
||||||
|
|
||||||
|
e2e:
|
||||||
|
bash test/e2e/run-e2e.sh
|
||||||
|
|
||||||
|
e2e-up:
|
||||||
|
docker compose -f docker-compose.e2e.yml up -d --build
|
||||||
|
|
||||||
|
e2e-down:
|
||||||
|
docker compose -f docker-compose.e2e.yml down
|
||||||
@@ -40,7 +40,7 @@ Alle bouwblokken van het R&D-team zijn gebundeld in de **EVOLV-repository**, waa
|
|||||||
### Eerste keer klonen:
|
### Eerste keer klonen:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone --recurse-submodules https://gitea.centraal.wbd-rd.nl/RnD/EVOLV.git
|
git clone --recurse-submodules https://gitea.wbd-rd.nl/RnD/EVOLV.git
|
||||||
cd EVOLV
|
cd EVOLV
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -77,7 +77,7 @@ git commit -m "Update submodule <bouwblok-naam>"
|
|||||||
1. Clone de gewenste repository:
|
1. Clone de gewenste repository:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://gitea.centraal.wbd-rd.nl/<repo-naam>.git
|
git clone https://gitea.wbd-rd.nl/<repo-naam>.git
|
||||||
```
|
```
|
||||||
|
|
||||||
2. Kopieer het bouwblok naar je Node-RED map:
|
2. Kopieer het bouwblok naar je Node-RED map:
|
||||||
|
|||||||
49
docker-compose.e2e.yml
Normal file
49
docker-compose.e2e.yml
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
services:
|
||||||
|
influxdb:
|
||||||
|
image: influxdb:2.7
|
||||||
|
environment:
|
||||||
|
- DOCKER_INFLUXDB_INIT_MODE=setup
|
||||||
|
- DOCKER_INFLUXDB_INIT_USERNAME=admin
|
||||||
|
- DOCKER_INFLUXDB_INIT_PASSWORD=adminpassword
|
||||||
|
- DOCKER_INFLUXDB_INIT_ORG=evolv
|
||||||
|
- DOCKER_INFLUXDB_INIT_BUCKET=evolv
|
||||||
|
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=evolv-e2e-token
|
||||||
|
ports:
|
||||||
|
- "8086:8086"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "influx", "ping"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
nodered:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile.e2e
|
||||||
|
ports:
|
||||||
|
- "1880:1880"
|
||||||
|
depends_on:
|
||||||
|
influxdb:
|
||||||
|
condition: service_healthy
|
||||||
|
environment:
|
||||||
|
- INFLUXDB_URL=http://influxdb:8086
|
||||||
|
- INFLUXDB_TOKEN=evolv-e2e-token
|
||||||
|
- INFLUXDB_ORG=evolv
|
||||||
|
- INFLUXDB_BUCKET=evolv
|
||||||
|
volumes:
|
||||||
|
- ./test/e2e/flows.json:/data/flows.json
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:1880/"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
grafana:
|
||||||
|
image: grafana/grafana:latest
|
||||||
|
ports:
|
||||||
|
- "3000:3000"
|
||||||
|
environment:
|
||||||
|
- GF_SECURITY_ADMIN_PASSWORD=admin
|
||||||
|
- GF_AUTH_ANONYMOUS_ENABLED=true
|
||||||
|
depends_on:
|
||||||
|
- influxdb
|
||||||
File diff suppressed because it is too large
Load Diff
29
eslint.config.js
Normal file
29
eslint.config.js
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
const js = require('@eslint/js');
|
||||||
|
const globals = require('globals');
|
||||||
|
|
||||||
|
module.exports = [
|
||||||
|
js.configs.recommended,
|
||||||
|
{
|
||||||
|
languageOptions: {
|
||||||
|
ecmaVersion: 2022,
|
||||||
|
sourceType: 'commonjs',
|
||||||
|
globals: {
|
||||||
|
...globals.node,
|
||||||
|
...globals.jest,
|
||||||
|
RED: 'readonly',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
rules: {
|
||||||
|
'no-unused-vars': ['warn', { argsIgnorePattern: '^_', varsIgnorePattern: '^_' }],
|
||||||
|
'no-console': 'off',
|
||||||
|
'no-prototype-builtins': 'warn',
|
||||||
|
'no-constant-condition': 'warn',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
ignores: [
|
||||||
|
'node_modules/**',
|
||||||
|
'nodes/generalFunctions/src/coolprop-node/coolprop/**',
|
||||||
|
],
|
||||||
|
},
|
||||||
|
];
|
||||||
19
jest.config.js
Normal file
19
jest.config.js
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
module.exports = {
|
||||||
|
testEnvironment: 'node',
|
||||||
|
verbose: true,
|
||||||
|
testMatch: [
|
||||||
|
'<rootDir>/nodes/generalFunctions/src/coolprop-node/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/generalFunctions/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/dashboardAPI/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/diffuser/test/specificClass.test.js',
|
||||||
|
'<rootDir>/nodes/monster/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/pumpingStation/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/reactor/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/settler/test/**/*.test.js',
|
||||||
|
'<rootDir>/nodes/measurement/test/**/*.test.js',
|
||||||
|
],
|
||||||
|
testPathIgnorePatterns: [
|
||||||
|
'/node_modules/',
|
||||||
|
],
|
||||||
|
testTimeout: 15000,
|
||||||
|
};
|
||||||
Submodule nodes/diffuser updated: c4dda5955f...3ccac81acf
Submodule nodes/generalFunctions updated: 27a6d3c709...024db5533a
Submodule nodes/machineGroupControl updated: b337bf9eb7...d55f401ab3
Submodule nodes/measurement updated: 43b5269f0b...0918be7705
Submodule nodes/monster updated: 32ebfd7154...5942a59cce
Submodule nodes/pumpingStation updated: 7efd3b0a07...7d05d37678
Submodule nodes/reactor updated: 2c69a5a0c1...556dc39049
Submodule nodes/rotatingMachine updated: 6b2a8239f2...07af7cef40
Submodule nodes/settler updated: 7f2d326612...518262ac98
7299
package-lock.json
generated
7299
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
118
package.json
118
package.json
@@ -1,50 +1,68 @@
|
|||||||
{
|
{
|
||||||
"name": "EVOLV",
|
"name": "EVOLV",
|
||||||
"version": "1.0.29",
|
"version": "1.0.29",
|
||||||
"description": "Modular Node-RED package containing all control and automation nodes developed under the EVOLV project.",
|
"description": "Modular Node-RED package containing all control and automation nodes developed under the EVOLV project.",
|
||||||
"keywords": [
|
"keywords": [
|
||||||
"node-red",
|
"node-red",
|
||||||
"EVOLV",
|
"EVOLV",
|
||||||
"automation",
|
"automation",
|
||||||
"control",
|
"control",
|
||||||
"wastewater"
|
"wastewater"
|
||||||
],
|
],
|
||||||
"node-red": {
|
"node-red": {
|
||||||
"nodes": {
|
"nodes": {
|
||||||
"dashboardapi": "nodes/dashboardAPI/dashboardapi.js",
|
"dashboardapi": "nodes/dashboardAPI/dashboardapi.js",
|
||||||
"machineGroupControl": "nodes/machineGroupControl/mgc.js",
|
"diffuser": "nodes/diffuser/diffuser.js",
|
||||||
"measurement": "nodes/measurement/measurement.js",
|
"machineGroupControl": "nodes/machineGroupControl/mgc.js",
|
||||||
"monster": "nodes/monster/monster.js",
|
"measurement": "nodes/measurement/measurement.js",
|
||||||
"reactor": "nodes/reactor/reactor.js",
|
"monster": "nodes/monster/monster.js",
|
||||||
"rotatingMachine": "nodes/rotatingMachine/rotatingMachine.js",
|
"pumpingstation": "nodes/pumpingStation/pumpingStation.js",
|
||||||
"valve": "nodes/valve/valve.js",
|
"reactor": "nodes/reactor/reactor.js",
|
||||||
"valveGroupControl": "nodes/valveGroupControl/vgc.js",
|
"rotatingMachine": "nodes/rotatingMachine/rotatingMachine.js",
|
||||||
"pumpingstation": "nodes/pumpingStation/pumpingStation.js",
|
"settler": "nodes/settler/settler.js",
|
||||||
"settler": "nodes/settler/settler.js"
|
"valve": "nodes/valve/valve.js",
|
||||||
}
|
"valveGroupControl": "nodes/valveGroupControl/vgc.js"
|
||||||
},
|
}
|
||||||
"scripts": {
|
},
|
||||||
"docker:build": "docker compose build",
|
"scripts": {
|
||||||
"docker:up": "docker compose up -d",
|
"preinstall": "node scripts/patch-deps.js",
|
||||||
"docker:down": "docker compose down",
|
"postinstall": "git checkout -- package.json 2>/dev/null || true",
|
||||||
"docker:logs": "docker compose logs -f nodered",
|
"docker:build": "docker compose build",
|
||||||
"docker:shell": "docker compose exec nodered sh",
|
"docker:up": "docker compose up -d",
|
||||||
"docker:test": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh",
|
"docker:down": "docker compose down",
|
||||||
"docker:test:basic": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh basic",
|
"docker:logs": "docker compose logs -f nodered",
|
||||||
"docker:test:integration": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh integration",
|
"docker:shell": "docker compose exec nodered sh",
|
||||||
"docker:test:edge": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh edge",
|
"docker:test": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh",
|
||||||
"docker:test:gf": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh gf",
|
"docker:test:basic": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh basic",
|
||||||
"docker:validate": "docker compose exec nodered sh /data/evolv/scripts/validate-nodes.sh",
|
"docker:test:integration": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh integration",
|
||||||
"docker:deploy": "docker compose exec nodered sh /data/evolv/scripts/deploy-flow.sh",
|
"docker:test:edge": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh edge",
|
||||||
"docker:reset": "docker compose down -v && docker compose up -d --build"
|
"docker:test:gf": "docker compose exec nodered sh /data/evolv/scripts/test-all.sh gf",
|
||||||
},
|
"docker:validate": "docker compose exec nodered sh /data/evolv/scripts/validate-nodes.sh",
|
||||||
"author": "Rene De Ren, Pim Moerman, Janneke Tack, Sjoerd Fijnje, Dieke Gabriels, pieter van der wilt",
|
"docker:deploy": "docker compose exec nodered sh /data/evolv/scripts/deploy-flow.sh",
|
||||||
"license": "SEE LICENSE",
|
"docker:reset": "docker compose down -v && docker compose up -d --build",
|
||||||
"dependencies": {
|
"test": "jest --forceExit",
|
||||||
"@flowfuse/node-red-dashboard": "^1.30.2",
|
"test:node": "node --test nodes/valve/test/basic/*.test.js nodes/valve/test/edge/*.test.js nodes/valve/test/integration/*.test.js nodes/valveGroupControl/test/basic/*.test.js nodes/valveGroupControl/test/edge/*.test.js nodes/valveGroupControl/test/integration/*.test.js",
|
||||||
"@tensorflow/tfjs": "^4.22.0",
|
"test:legacy": "node nodes/machineGroupControl/src/groupcontrol.test.js && node nodes/generalFunctions/src/nrmse/errorMetric.test.js",
|
||||||
"@tensorflow/tfjs-node": "^4.22.0",
|
"test:all": "npm test && npm run test:node && npm run test:legacy",
|
||||||
"generalFunctions": "file:nodes/generalFunctions",
|
"test:e2e:reactor": "node scripts/e2e-reactor-roundtrip.js",
|
||||||
"mathjs": "^13.2.0"
|
"lint": "eslint nodes/",
|
||||||
}
|
"lint:fix": "eslint nodes/ --fix",
|
||||||
}
|
"ci": "npm run lint && npm run test:all",
|
||||||
|
"test:e2e": "bash test/e2e/run-e2e.sh"
|
||||||
|
},
|
||||||
|
"author": "Rene De Ren, Pim Moerman, Janneke Tack, Sjoerd Fijnje, Dieke Gabriels, pieter van der wilt",
|
||||||
|
"license": "SEE LICENSE",
|
||||||
|
"dependencies": {
|
||||||
|
"@flowfuse/node-red-dashboard": "^1.30.2",
|
||||||
|
"@tensorflow/tfjs": "^4.22.0",
|
||||||
|
"@tensorflow/tfjs-node": "^4.22.0",
|
||||||
|
"generalFunctions": "file:nodes/generalFunctions",
|
||||||
|
"mathjs": "^13.2.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@eslint/js": "^8.57.0",
|
||||||
|
"eslint": "^8.57.0",
|
||||||
|
"globals": "^15.0.0",
|
||||||
|
"jest": "^29.7.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
269
scripts/e2e-reactor-roundtrip.js
Normal file
269
scripts/e2e-reactor-roundtrip.js
Normal file
@@ -0,0 +1,269 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* E2E reactor round-trip test:
|
||||||
|
* Node-RED -> InfluxDB -> Grafana proxy query
|
||||||
|
*/
|
||||||
|
|
||||||
|
const fs = require('node:fs');
|
||||||
|
const path = require('node:path');
|
||||||
|
|
||||||
|
const NR_URL = process.env.NR_URL || 'http://localhost:1880';
|
||||||
|
const INFLUX_URL = process.env.INFLUX_URL || 'http://localhost:8086';
|
||||||
|
const GRAFANA_URL = process.env.GRAFANA_URL || 'http://localhost:3000';
|
||||||
|
const GRAFANA_USER = process.env.GRAFANA_USER || 'admin';
|
||||||
|
const GRAFANA_PASSWORD = process.env.GRAFANA_PASSWORD || 'evolv';
|
||||||
|
const INFLUX_ORG = process.env.INFLUX_ORG || 'evolv';
|
||||||
|
const INFLUX_BUCKET = process.env.INFLUX_BUCKET || 'telemetry';
|
||||||
|
const INFLUX_TOKEN = process.env.INFLUX_TOKEN || 'evolv-dev-token';
|
||||||
|
const GRAFANA_DS_UID = process.env.GRAFANA_DS_UID || 'cdzg44tv250jkd';
|
||||||
|
const FLOW_FILE = path.join(__dirname, '..', 'docker', 'demo-flow.json');
|
||||||
|
const REQUIRE_GRAFANA_DASHBOARDS = process.env.REQUIRE_GRAFANA_DASHBOARDS === '1';
|
||||||
|
const REACTOR_MEASUREMENTS = [
|
||||||
|
'reactor_demo_reactor_z1',
|
||||||
|
'reactor_demo_reactor_z2',
|
||||||
|
'reactor_demo_reactor_z3',
|
||||||
|
'reactor_demo_reactor_z4',
|
||||||
|
];
|
||||||
|
const REACTOR_MEASUREMENT = REACTOR_MEASUREMENTS[3];
|
||||||
|
const QUERY_TIMEOUT_MS = 90000;
|
||||||
|
const POLL_INTERVAL_MS = 3000;
|
||||||
|
const REQUIRED_DASHBOARD_TITLES = ['Bioreactor Z1', 'Bioreactor Z2', 'Bioreactor Z3', 'Bioreactor Z4', 'Settler S1'];
|
||||||
|
|
||||||
|
async function wait(ms) {
|
||||||
|
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchJson(url, options = {}) {
|
||||||
|
const response = await fetch(url, options);
|
||||||
|
const text = await response.text();
|
||||||
|
let body = null;
|
||||||
|
if (text) {
|
||||||
|
try {
|
||||||
|
body = JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
body = text;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { response, body, text };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function assertReachable() {
|
||||||
|
const checks = [
|
||||||
|
[`${NR_URL}/settings`, 'Node-RED'],
|
||||||
|
[`${INFLUX_URL}/health`, 'InfluxDB'],
|
||||||
|
[`${GRAFANA_URL}/api/health`, 'Grafana'],
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const [url, label] of checks) {
|
||||||
|
const { response, text } = await fetchJson(url, {
|
||||||
|
headers: label === 'Grafana'
|
||||||
|
? { Authorization: `Basic ${Buffer.from(`${GRAFANA_USER}:${GRAFANA_PASSWORD}`).toString('base64')}` }
|
||||||
|
: undefined,
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`${label} not reachable at ${url} (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
console.log(`PASS: ${label} reachable`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deployDemoFlow() {
|
||||||
|
const flow = JSON.parse(fs.readFileSync(FLOW_FILE, 'utf8'));
|
||||||
|
const { response, text } = await fetchJson(`${NR_URL}/flows`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
'Node-RED-Deployment-Type': 'full',
|
||||||
|
},
|
||||||
|
body: JSON.stringify(flow),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!(response.status === 200 || response.status === 204)) {
|
||||||
|
throw new Error(`Flow deploy failed (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
console.log(`PASS: Demo flow deployed (${response.status})`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function queryInfluxCsv(query) {
|
||||||
|
const response = await fetch(`${INFLUX_URL}/api/v2/query?org=${encodeURIComponent(INFLUX_ORG)}`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
Authorization: `Token ${INFLUX_TOKEN}`,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
Accept: 'application/csv',
|
||||||
|
},
|
||||||
|
body: JSON.stringify({ query }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const text = await response.text();
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Influx query failed (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
return text;
|
||||||
|
}
|
||||||
|
|
||||||
|
function countCsvDataRows(csvText) {
|
||||||
|
return csvText
|
||||||
|
.split('\n')
|
||||||
|
.map((line) => line.trim())
|
||||||
|
.filter((line) => line && !line.startsWith('#') && line.includes(','))
|
||||||
|
.length;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForReactorTelemetry() {
|
||||||
|
const deadline = Date.now() + QUERY_TIMEOUT_MS;
|
||||||
|
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
const counts = {};
|
||||||
|
for (const measurement of REACTOR_MEASUREMENTS) {
|
||||||
|
const query = `
|
||||||
|
from(bucket: "${INFLUX_BUCKET}")
|
||||||
|
|> range(start: -15m)
|
||||||
|
|> filter(fn: (r) => r._measurement == "${measurement}")
|
||||||
|
|> limit(n: 20)
|
||||||
|
`.trim();
|
||||||
|
counts[measurement] = countCsvDataRows(await queryInfluxCsv(query));
|
||||||
|
}
|
||||||
|
|
||||||
|
const missing = Object.entries(counts)
|
||||||
|
.filter(([, rows]) => rows === 0)
|
||||||
|
.map(([measurement]) => measurement);
|
||||||
|
|
||||||
|
if (missing.length === 0) {
|
||||||
|
const summary = Object.entries(counts)
|
||||||
|
.map(([measurement, rows]) => `${measurement}=${rows}`)
|
||||||
|
.join(', ');
|
||||||
|
console.log(`PASS: Reactor telemetry reached InfluxDB (${summary})`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
console.log(`WAIT: reactor telemetry not yet present in InfluxDB for ${missing.join(', ')}`);
|
||||||
|
await wait(POLL_INTERVAL_MS);
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(`Timed out waiting for reactor telemetry measurements ${REACTOR_MEASUREMENTS.join(', ')}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function assertGrafanaDatasource() {
|
||||||
|
const auth = `Basic ${Buffer.from(`${GRAFANA_USER}:${GRAFANA_PASSWORD}`).toString('base64')}`;
|
||||||
|
const { response, body, text } = await fetchJson(`${GRAFANA_URL}/api/datasources/uid/${GRAFANA_DS_UID}`, {
|
||||||
|
headers: { Authorization: auth },
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Grafana datasource lookup failed (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (body?.uid !== GRAFANA_DS_UID) {
|
||||||
|
throw new Error(`Grafana datasource UID mismatch: expected ${GRAFANA_DS_UID}, got ${body?.uid}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`PASS: Grafana datasource ${GRAFANA_DS_UID} is present`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function queryGrafanaDatasource() {
|
||||||
|
const auth = `Basic ${Buffer.from(`${GRAFANA_USER}:${GRAFANA_PASSWORD}`).toString('base64')}`;
|
||||||
|
const response = await fetch(`${GRAFANA_URL}/api/ds/query`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
Authorization: auth,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
},
|
||||||
|
body: JSON.stringify({
|
||||||
|
from: 'now-15m',
|
||||||
|
to: 'now',
|
||||||
|
queries: [
|
||||||
|
{
|
||||||
|
refId: 'A',
|
||||||
|
datasource: { uid: GRAFANA_DS_UID, type: 'influxdb' },
|
||||||
|
query: `
|
||||||
|
from(bucket: "${INFLUX_BUCKET}")
|
||||||
|
|> range(start: -15m)
|
||||||
|
|> filter(fn: (r) => r._measurement == "${REACTOR_MEASUREMENT}" and r._field == "S_O")
|
||||||
|
|> last()
|
||||||
|
`.trim(),
|
||||||
|
rawQuery: true,
|
||||||
|
intervalMs: 1000,
|
||||||
|
maxDataPoints: 100,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
const text = await response.text();
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Grafana datasource query failed (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const body = JSON.parse(text);
|
||||||
|
const frames = body?.results?.A?.frames || [];
|
||||||
|
if (frames.length === 0) {
|
||||||
|
throw new Error('Grafana datasource query returned no reactor frames');
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`PASS: Grafana can query reactor telemetry through datasource (${frames.length} frame(s))`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitForGrafanaDashboards(timeoutMs = QUERY_TIMEOUT_MS) {
|
||||||
|
const deadline = Date.now() + timeoutMs;
|
||||||
|
const auth = `Basic ${Buffer.from(`${GRAFANA_USER}:${GRAFANA_PASSWORD}`).toString('base64')}`;
|
||||||
|
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
const response = await fetch(`${GRAFANA_URL}/api/search?query=`, {
|
||||||
|
headers: { Authorization: auth },
|
||||||
|
});
|
||||||
|
const text = await response.text();
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Grafana dashboard search failed (${response.status}): ${text}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const results = JSON.parse(text);
|
||||||
|
const titles = new Set(results.map((item) => item.title));
|
||||||
|
const missing = REQUIRED_DASHBOARD_TITLES.filter((title) => !titles.has(title));
|
||||||
|
const pumpingStationCount = results.filter((item) => item.title === 'pumpingStation').length;
|
||||||
|
if (missing.length === 0 && pumpingStationCount >= 3) {
|
||||||
|
console.log(`PASS: Grafana dashboards created (${REQUIRED_DASHBOARD_TITLES.join(', ')} + ${pumpingStationCount} pumpingStation dashboards)`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const missingParts = [];
|
||||||
|
if (missing.length > 0) {
|
||||||
|
missingParts.push(`missing titled dashboards: ${missing.join(', ')}`);
|
||||||
|
}
|
||||||
|
if (pumpingStationCount < 3) {
|
||||||
|
missingParts.push(`pumpingStation dashboards=${pumpingStationCount}`);
|
||||||
|
}
|
||||||
|
console.log(`WAIT: Grafana dashboards not ready: ${missingParts.join(' | ')}`);
|
||||||
|
await wait(POLL_INTERVAL_MS);
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(`Timed out waiting for Grafana dashboards: ${REQUIRED_DASHBOARD_TITLES.join(', ')} and >=3 pumpingStation dashboards`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
console.log('=== EVOLV Reactor E2E Round Trip ===');
|
||||||
|
await assertReachable();
|
||||||
|
await deployDemoFlow();
|
||||||
|
console.log('WAIT: allowing Node-RED inject/tick loops to populate telemetry');
|
||||||
|
await wait(12000);
|
||||||
|
await waitForReactorTelemetry();
|
||||||
|
await assertGrafanaDatasource();
|
||||||
|
await queryGrafanaDatasource();
|
||||||
|
if (REQUIRE_GRAFANA_DASHBOARDS) {
|
||||||
|
await waitForGrafanaDashboards();
|
||||||
|
console.log('PASS: Node-RED -> InfluxDB -> Grafana round trip is working for reactor telemetry and dashboard generation');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await waitForGrafanaDashboards(15000);
|
||||||
|
console.log('PASS: Node-RED -> InfluxDB -> Grafana round trip is working for reactor telemetry and dashboard generation');
|
||||||
|
} catch (error) {
|
||||||
|
console.warn(`WARN: Grafana dashboard auto-generation is not ready yet: ${error.message}`);
|
||||||
|
console.log('PASS: Node-RED -> InfluxDB -> Grafana round trip is working for live reactor telemetry');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((error) => {
|
||||||
|
console.error(`FAIL: ${error.message}`);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
20
scripts/patch-deps.js
Normal file
20
scripts/patch-deps.js
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
/**
|
||||||
|
* Preinstall script: rewrites the generalFunctions dependency
|
||||||
|
* from git+https to a local file path when the submodule exists.
|
||||||
|
* This avoids needing Gitea credentials during npm install.
|
||||||
|
*/
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
|
||||||
|
const pkgPath = path.join(__dirname, '..', 'package.json');
|
||||||
|
const localGF = path.join(__dirname, '..', 'nodes', 'generalFunctions');
|
||||||
|
|
||||||
|
if (fs.existsSync(localGF) && fs.existsSync(path.join(localGF, 'index.js'))) {
|
||||||
|
const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf8'));
|
||||||
|
if (pkg.dependencies && pkg.dependencies.generalFunctions &&
|
||||||
|
pkg.dependencies.generalFunctions.startsWith('git+')) {
|
||||||
|
pkg.dependencies.generalFunctions = 'file:./nodes/generalFunctions';
|
||||||
|
fs.writeFileSync(pkgPath, JSON.stringify(pkg, null, 2) + '\n');
|
||||||
|
console.log('[patch-deps] Rewrote generalFunctions to local path');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
# Copy this file to `.env` on the target server and populate real values there.
|
|
||||||
# Keep the real `.env` out of version control.
|
|
||||||
|
|
||||||
INFLUXDB_ADMIN_USER=replace-me
|
|
||||||
INFLUXDB_ADMIN_PASSWORD=replace-me
|
|
||||||
INFLUXDB_BUCKET=lvl0
|
|
||||||
INFLUXDB_ORG=wbd
|
|
||||||
|
|
||||||
GF_SECURITY_ADMIN_USER=replace-me
|
|
||||||
GF_SECURITY_ADMIN_PASSWORD=replace-me
|
|
||||||
|
|
||||||
NPM_DB_MYSQL_HOST=db
|
|
||||||
NPM_DB_MYSQL_PORT=3306
|
|
||||||
NPM_DB_MYSQL_USER=npm
|
|
||||||
NPM_DB_MYSQL_PASSWORD=replace-me
|
|
||||||
NPM_DB_MYSQL_NAME=npm
|
|
||||||
|
|
||||||
MYSQL_ROOT_PASSWORD=replace-me
|
|
||||||
MYSQL_DATABASE=npm
|
|
||||||
MYSQL_USER=npm
|
|
||||||
MYSQL_PASSWORD=replace-me
|
|
||||||
|
|
||||||
RABBITMQ_DEFAULT_USER=replace-me
|
|
||||||
RABBITMQ_DEFAULT_PASS=replace-me
|
|
||||||
117
temp/cloud.yml
117
temp/cloud.yml
@@ -1,117 +0,0 @@
|
|||||||
services:
|
|
||||||
node-red:
|
|
||||||
image: nodered/node-red:latest
|
|
||||||
container_name: node-red
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "1880:1880"
|
|
||||||
volumes:
|
|
||||||
- node_red_data:/data
|
|
||||||
|
|
||||||
influxdb:
|
|
||||||
image: influxdb:2.7
|
|
||||||
container_name: influxdb
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "8086:8086"
|
|
||||||
environment:
|
|
||||||
- INFLUXDB_ADMIN_USER=${INFLUXDB_ADMIN_USER}
|
|
||||||
- INFLUXDB_ADMIN_PASSWORD=${INFLUXDB_ADMIN_PASSWORD}
|
|
||||||
- INFLUXDB_BUCKET=${INFLUXDB_BUCKET}
|
|
||||||
- INFLUXDB_ORG=${INFLUXDB_ORG}
|
|
||||||
volumes:
|
|
||||||
- influxdb_data:/var/lib/influxdb2
|
|
||||||
|
|
||||||
grafana:
|
|
||||||
image: grafana/grafana:latest
|
|
||||||
container_name: grafana
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "3000:3000"
|
|
||||||
environment:
|
|
||||||
- GF_SECURITY_ADMIN_USER=${GF_SECURITY_ADMIN_USER}
|
|
||||||
- GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD}
|
|
||||||
volumes:
|
|
||||||
- grafana_data:/var/lib/grafana
|
|
||||||
depends_on:
|
|
||||||
- influxdb
|
|
||||||
|
|
||||||
jenkins:
|
|
||||||
image: jenkins/jenkins:lts
|
|
||||||
container_name: jenkins
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "8080:8080" # Web
|
|
||||||
- "50000:50000" # Agents
|
|
||||||
volumes:
|
|
||||||
- jenkins_home:/var/jenkins_home
|
|
||||||
|
|
||||||
gitea:
|
|
||||||
image: gitea/gitea:latest
|
|
||||||
container_name: gitea
|
|
||||||
restart: always
|
|
||||||
environment:
|
|
||||||
- USER_UID=1000
|
|
||||||
- USER_GID=1000
|
|
||||||
ports:
|
|
||||||
- "3001:3000" # Webinterface (anders dan Grafana)
|
|
||||||
- "222:22" # SSH voor Git
|
|
||||||
volumes:
|
|
||||||
- gitea_data:/data
|
|
||||||
|
|
||||||
proxymanager:
|
|
||||||
image: jc21/nginx-proxy-manager:latest
|
|
||||||
container_name: proxymanager
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "80:80" # HTTP
|
|
||||||
- "443:443" # HTTPS
|
|
||||||
- "81:81" # Admin UI
|
|
||||||
environment:
|
|
||||||
DB_MYSQL_HOST: ${NPM_DB_MYSQL_HOST:-db}
|
|
||||||
DB_MYSQL_PORT: ${NPM_DB_MYSQL_PORT:-3306}
|
|
||||||
DB_MYSQL_USER: ${NPM_DB_MYSQL_USER}
|
|
||||||
DB_MYSQL_PASSWORD: ${NPM_DB_MYSQL_PASSWORD}
|
|
||||||
DB_MYSQL_NAME: ${NPM_DB_MYSQL_NAME}
|
|
||||||
volumes:
|
|
||||||
- proxymanager_data:/data
|
|
||||||
- proxymanager_letsencrypt:/etc/letsencrypt
|
|
||||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
||||||
depends_on:
|
|
||||||
- db
|
|
||||||
|
|
||||||
db:
|
|
||||||
image: jc21/mariadb-aria:latest
|
|
||||||
container_name: proxymanager_db
|
|
||||||
restart: always
|
|
||||||
environment:
|
|
||||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
|
|
||||||
MYSQL_DATABASE: ${MYSQL_DATABASE}
|
|
||||||
MYSQL_USER: ${MYSQL_USER}
|
|
||||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
|
|
||||||
volumes:
|
|
||||||
- proxymanager_db_data:/var/lib/mysql
|
|
||||||
|
|
||||||
rabbitmq:
|
|
||||||
image: rabbitmq:3-management
|
|
||||||
container_name: rabbitmq
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "5672:5672" # AMQP protocol voor apps
|
|
||||||
- "15672:15672" # Management webinterface
|
|
||||||
environment:
|
|
||||||
- RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
|
|
||||||
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
|
|
||||||
volumes:
|
|
||||||
- rabbitmq_data:/var/lib/rabbitmq
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
rabbitmq_data:
|
|
||||||
node_red_data:
|
|
||||||
influxdb_data:
|
|
||||||
grafana_data:
|
|
||||||
jenkins_home:
|
|
||||||
gitea_data:
|
|
||||||
proxymanager_data:
|
|
||||||
proxymanager_letsencrypt:
|
|
||||||
proxymanager_db_data:
|
|
||||||
440
test/e2e/flows.json
Normal file
440
test/e2e/flows.json
Normal file
@@ -0,0 +1,440 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"id": "e2e-flow-tab",
|
||||||
|
"type": "tab",
|
||||||
|
"label": "E2E Test Flow",
|
||||||
|
"disabled": false,
|
||||||
|
"info": "End-to-end test flow that verifies EVOLV nodes load, accept input, and produce output."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-trigger",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Trigger once on start",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "3",
|
||||||
|
"topic": "e2e-test",
|
||||||
|
"payload": "",
|
||||||
|
"payloadType": "date",
|
||||||
|
"x": 160,
|
||||||
|
"y": 80,
|
||||||
|
"wires": [["build-measurement-msg"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "build-measurement-msg",
|
||||||
|
"type": "function",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Build measurement input",
|
||||||
|
"func": "// Simulate an analog sensor reading sent to the measurement node.\n// The measurement node expects a numeric payload on topic 'analogInput'.\nmsg.payload = 4.2 + Math.random() * 15.8; // 4-20 mA range\nmsg.topic = 'analogInput';\nnode.status({ fill: 'green', shape: 'dot', text: 'sent ' + msg.payload.toFixed(2) });\nreturn msg;",
|
||||||
|
"outputs": 1,
|
||||||
|
"timeout": "",
|
||||||
|
"noerr": 0,
|
||||||
|
"initialize": "",
|
||||||
|
"finalize": "",
|
||||||
|
"libs": [],
|
||||||
|
"x": 380,
|
||||||
|
"y": 80,
|
||||||
|
"wires": [["measurement-e2e-node"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "measurement-e2e-node",
|
||||||
|
"type": "measurement",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "E2E-Level-Sensor",
|
||||||
|
"scaling": true,
|
||||||
|
"i_min": 4,
|
||||||
|
"i_max": 20,
|
||||||
|
"i_offset": 0,
|
||||||
|
"o_min": 0,
|
||||||
|
"o_max": 5,
|
||||||
|
"simulator": false,
|
||||||
|
"smooth_method": "",
|
||||||
|
"count": "10",
|
||||||
|
"uuid": "",
|
||||||
|
"supplier": "e2e-test",
|
||||||
|
"category": "level",
|
||||||
|
"assetType": "sensor",
|
||||||
|
"model": "e2e-virtual",
|
||||||
|
"unit": "m",
|
||||||
|
"enableLog": false,
|
||||||
|
"logLevel": "error",
|
||||||
|
"positionVsParent": "upstream",
|
||||||
|
"positionIcon": "",
|
||||||
|
"hasDistance": false,
|
||||||
|
"distance": 0,
|
||||||
|
"distanceUnit": "m",
|
||||||
|
"distanceDescription": "",
|
||||||
|
"x": 600,
|
||||||
|
"y": 80,
|
||||||
|
"wires": [
|
||||||
|
["debug-process"],
|
||||||
|
["debug-dbase"],
|
||||||
|
["debug-parent"]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-process",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Process Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 830,
|
||||||
|
"y": 40,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-dbase",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Database Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 840,
|
||||||
|
"y": 80,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-parent",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Parent Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 830,
|
||||||
|
"y": 120,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-periodic",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Periodic (5s)",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "5",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "6",
|
||||||
|
"topic": "e2e-heartbeat",
|
||||||
|
"payload": "",
|
||||||
|
"payloadType": "date",
|
||||||
|
"x": 160,
|
||||||
|
"y": 200,
|
||||||
|
"wires": [["heartbeat-func"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "heartbeat-func",
|
||||||
|
"type": "function",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Heartbeat check",
|
||||||
|
"func": "// Verify the EVOLV measurement node is running by querying its presence\nmsg.payload = {\n check: 'heartbeat',\n timestamp: Date.now(),\n nodeCount: global.get('_e2e_msg_count') || 0\n};\n// Increment message counter\nlet count = global.get('_e2e_msg_count') || 0;\nglobal.set('_e2e_msg_count', count + 1);\nnode.status({ fill: 'blue', shape: 'ring', text: 'beat #' + (count+1) });\nreturn msg;",
|
||||||
|
"outputs": 1,
|
||||||
|
"timeout": "",
|
||||||
|
"noerr": 0,
|
||||||
|
"initialize": "",
|
||||||
|
"finalize": "",
|
||||||
|
"libs": [],
|
||||||
|
"x": 380,
|
||||||
|
"y": 200,
|
||||||
|
"wires": [["debug-heartbeat"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-heartbeat",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Heartbeat Debug",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": false,
|
||||||
|
"complete": "payload",
|
||||||
|
"targetType": "msg",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 600,
|
||||||
|
"y": 200,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-monster-prediction",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Monster prediction",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "4",
|
||||||
|
"topic": "model_prediction",
|
||||||
|
"payload": "120",
|
||||||
|
"payloadType": "num",
|
||||||
|
"x": 150,
|
||||||
|
"y": 320,
|
||||||
|
"wires": [["evolv-monster"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-monster-flow",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Monster flow",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "3",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "5",
|
||||||
|
"topic": "i_flow",
|
||||||
|
"payload": "3600",
|
||||||
|
"payloadType": "num",
|
||||||
|
"x": 140,
|
||||||
|
"y": 360,
|
||||||
|
"wires": [["evolv-monster"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-monster-start",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Monster start",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "6",
|
||||||
|
"topic": "start",
|
||||||
|
"payload": "",
|
||||||
|
"payloadType": "date",
|
||||||
|
"x": 140,
|
||||||
|
"y": 400,
|
||||||
|
"wires": [["evolv-monster"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "evolv-monster",
|
||||||
|
"type": "monster",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "E2E-Monster",
|
||||||
|
"samplingtime": 1,
|
||||||
|
"minvolume": 5,
|
||||||
|
"maxweight": 23,
|
||||||
|
"emptyWeightBucket": 3,
|
||||||
|
"aquon_sample_name": "112100",
|
||||||
|
"supplier": "e2e-test",
|
||||||
|
"subType": "samplingCabinet",
|
||||||
|
"model": "e2e-virtual",
|
||||||
|
"unit": "m3/h",
|
||||||
|
"enableLog": false,
|
||||||
|
"logLevel": "error",
|
||||||
|
"x": 390,
|
||||||
|
"y": 360,
|
||||||
|
"wires": [
|
||||||
|
["debug-monster-process"],
|
||||||
|
["debug-monster-dbase"],
|
||||||
|
[],
|
||||||
|
[]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-monster-process",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Monster Process Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 660,
|
||||||
|
"y": 340,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-monster-dbase",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Monster Database Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 670,
|
||||||
|
"y": 380,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-dashboardapi-register",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "DashboardAPI register child",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "12",
|
||||||
|
"payload": "",
|
||||||
|
"payloadType": "date",
|
||||||
|
"x": 160,
|
||||||
|
"y": 500,
|
||||||
|
"wires": [["build-dashboardapi-msg"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "build-dashboardapi-msg",
|
||||||
|
"type": "function",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Build dashboardapi input",
|
||||||
|
"func": "msg.topic = 'registerChild';\nmsg.payload = {\n config: {\n general: {\n name: 'E2E-Level-Sensor'\n },\n functionality: {\n softwareType: 'measurement'\n }\n }\n};\nreturn msg;",
|
||||||
|
"outputs": 1,
|
||||||
|
"timeout": "",
|
||||||
|
"noerr": 0,
|
||||||
|
"initialize": "",
|
||||||
|
"finalize": "",
|
||||||
|
"libs": [],
|
||||||
|
"x": 400,
|
||||||
|
"y": 500,
|
||||||
|
"wires": [["dashboardapi-e2e"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dashboardapi-e2e",
|
||||||
|
"type": "dashboardapi",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "E2E-DashboardAPI",
|
||||||
|
"logLevel": "error",
|
||||||
|
"enableLog": false,
|
||||||
|
"host": "grafana",
|
||||||
|
"port": "3000",
|
||||||
|
"bearerToken": "",
|
||||||
|
"x": 660,
|
||||||
|
"y": 500,
|
||||||
|
"wires": [["debug-dashboardapi-output"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-dashboardapi-output",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "DashboardAPI Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 920,
|
||||||
|
"y": 500,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "inject-diffuser-flow",
|
||||||
|
"type": "inject",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Diffuser airflow",
|
||||||
|
"props": [
|
||||||
|
{ "p": "payload" },
|
||||||
|
{ "p": "topic", "vt": "str" }
|
||||||
|
],
|
||||||
|
"repeat": "",
|
||||||
|
"crontab": "",
|
||||||
|
"once": true,
|
||||||
|
"onceDelay": "9",
|
||||||
|
"topic": "air_flow",
|
||||||
|
"payload": "24",
|
||||||
|
"payloadType": "num",
|
||||||
|
"x": 150,
|
||||||
|
"y": 620,
|
||||||
|
"wires": [["diffuser-e2e"]]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "diffuser-e2e",
|
||||||
|
"type": "diffuser",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "E2E-Diffuser",
|
||||||
|
"number": 1,
|
||||||
|
"i_elements": 4,
|
||||||
|
"i_diff_density": 2.4,
|
||||||
|
"i_m_water": 4.5,
|
||||||
|
"alfaf": 0.7,
|
||||||
|
"enableLog": false,
|
||||||
|
"logLevel": "error",
|
||||||
|
"x": 390,
|
||||||
|
"y": 620,
|
||||||
|
"wires": [["debug-diffuser-process"], ["debug-diffuser-dbase"], []]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-diffuser-process",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Diffuser Process Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 670,
|
||||||
|
"y": 600,
|
||||||
|
"wires": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "debug-diffuser-dbase",
|
||||||
|
"type": "debug",
|
||||||
|
"z": "e2e-flow-tab",
|
||||||
|
"name": "Diffuser Database Output",
|
||||||
|
"active": true,
|
||||||
|
"tosidebar": true,
|
||||||
|
"console": true,
|
||||||
|
"tostatus": true,
|
||||||
|
"complete": "true",
|
||||||
|
"targetType": "full",
|
||||||
|
"statusVal": "",
|
||||||
|
"statusType": "auto",
|
||||||
|
"x": 680,
|
||||||
|
"y": 640,
|
||||||
|
"wires": []
|
||||||
|
}
|
||||||
|
]
|
||||||
213
test/e2e/run-e2e.sh
Executable file
213
test/e2e/run-e2e.sh
Executable file
@@ -0,0 +1,213 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# End-to-end test runner for EVOLV Node-RED stack.
|
||||||
|
# Starts Node-RED + InfluxDB + Grafana via Docker Compose,
|
||||||
|
# verifies that EVOLV nodes are registered in the palette,
|
||||||
|
# and tears down the stack on exit.
|
||||||
|
#
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||||
|
COMPOSE_FILE="$PROJECT_ROOT/docker-compose.e2e.yml"
|
||||||
|
|
||||||
|
NODERED_URL="http://localhost:1880"
|
||||||
|
MAX_WAIT=120 # seconds to wait for Node-RED to become healthy
|
||||||
|
GRAFANA_URL="http://localhost:3000/api/health"
|
||||||
|
MAX_GRAFANA_WAIT=60
|
||||||
|
LOG_WAIT=20
|
||||||
|
|
||||||
|
# EVOLV node types that must appear in the palette (from package.json node-red.nodes)
|
||||||
|
EXPECTED_NODES=(
|
||||||
|
"dashboardapi"
|
||||||
|
"diffuser"
|
||||||
|
"machineGroupControl"
|
||||||
|
"measurement"
|
||||||
|
"monster"
|
||||||
|
"pumpingstation"
|
||||||
|
"reactor"
|
||||||
|
"rotatingMachine"
|
||||||
|
"settler"
|
||||||
|
"valve"
|
||||||
|
"valveGroupControl"
|
||||||
|
)
|
||||||
|
|
||||||
|
RED='\033[0;31m'
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||||
|
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||||
|
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
|
||||||
|
|
||||||
|
wait_for_log_pattern() {
|
||||||
|
local pattern="$1"
|
||||||
|
local description="$2"
|
||||||
|
local required="${3:-false}"
|
||||||
|
local elapsed=0
|
||||||
|
local logs=""
|
||||||
|
|
||||||
|
while [ $elapsed -lt $LOG_WAIT ]; do
|
||||||
|
logs=$(run_compose logs nodered 2>&1)
|
||||||
|
if echo "$logs" | grep -q "$pattern"; then
|
||||||
|
log_info " [PASS] $description"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 2
|
||||||
|
elapsed=$((elapsed + 2))
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$required" = true ]; then
|
||||||
|
log_error " [FAIL] $description not detected in logs"
|
||||||
|
FAILURES=$((FAILURES + 1))
|
||||||
|
else
|
||||||
|
log_warn " [WARN] $description not detected in logs"
|
||||||
|
fi
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Determine docker compose command (handle permission via sg docker if needed)
|
||||||
|
USE_SG_DOCKER=false
|
||||||
|
if ! docker info >/dev/null 2>&1; then
|
||||||
|
if sg docker -c "docker info" >/dev/null 2>&1; then
|
||||||
|
USE_SG_DOCKER=true
|
||||||
|
log_info "Using sg docker for Docker access"
|
||||||
|
else
|
||||||
|
log_error "Docker is not accessible. Please ensure Docker is running and you have permissions."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
run_compose() {
|
||||||
|
if [ "$USE_SG_DOCKER" = true ]; then
|
||||||
|
local cmd="docker compose -f $(printf '%q' "$COMPOSE_FILE")"
|
||||||
|
local arg
|
||||||
|
for arg in "$@"; do
|
||||||
|
cmd+=" $(printf '%q' "$arg")"
|
||||||
|
done
|
||||||
|
sg docker -c "$cmd"
|
||||||
|
else
|
||||||
|
docker compose -f "$COMPOSE_FILE" "$@"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
log_info "Tearing down E2E stack..."
|
||||||
|
run_compose down --volumes --remove-orphans 2>/dev/null || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# Always clean up on exit
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
# --- Step 1: Build and start the stack ---
|
||||||
|
log_info "Building and starting E2E stack..."
|
||||||
|
run_compose up -d --build
|
||||||
|
|
||||||
|
# --- Step 2: Wait for Node-RED to be healthy ---
|
||||||
|
log_info "Waiting for Node-RED to become healthy (max ${MAX_WAIT}s)..."
|
||||||
|
elapsed=0
|
||||||
|
while [ $elapsed -lt $MAX_WAIT ]; do
|
||||||
|
if curl -sf "$NODERED_URL/" >/dev/null 2>&1; then
|
||||||
|
log_info "Node-RED is up after ${elapsed}s"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 2
|
||||||
|
elapsed=$((elapsed + 2))
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ $elapsed -ge $MAX_WAIT ]; then
|
||||||
|
log_error "Node-RED did not become healthy within ${MAX_WAIT}s"
|
||||||
|
log_error "Container logs:"
|
||||||
|
run_compose logs nodered
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Give Node-RED a few extra seconds to finish loading all nodes and editor metadata
|
||||||
|
sleep 8
|
||||||
|
|
||||||
|
# --- Step 3: Verify EVOLV nodes are registered in the palette ---
|
||||||
|
log_info "Querying Node-RED for registered nodes..."
|
||||||
|
NODES_RESPONSE=$(curl -sf "$NODERED_URL/nodes" 2>&1) || {
|
||||||
|
log_error "Failed to query Node-RED /nodes endpoint"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
FAILURES=0
|
||||||
|
PALETTE_MISSES=0
|
||||||
|
for node_type in "${EXPECTED_NODES[@]}"; do
|
||||||
|
if echo "$NODES_RESPONSE" | grep -qi "$node_type"; then
|
||||||
|
log_info " [PASS] Node type '$node_type' found in palette"
|
||||||
|
else
|
||||||
|
log_warn " [WARN] Node type '$node_type' not found in /nodes response"
|
||||||
|
PALETTE_MISSES=$((PALETTE_MISSES + 1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# --- Step 4: Verify flows are deployed ---
|
||||||
|
log_info "Checking deployed flows..."
|
||||||
|
FLOWS_RESPONSE=$(curl -sf "$NODERED_URL/flows" 2>&1) || {
|
||||||
|
log_error "Failed to query Node-RED /flows endpoint"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
if echo "$FLOWS_RESPONSE" | grep -q "e2e-flow-tab"; then
|
||||||
|
log_info " [PASS] E2E test flow is deployed"
|
||||||
|
else
|
||||||
|
log_warn " [WARN] E2E test flow not found in deployed flows (may need manual deploy)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Step 5: Verify InfluxDB is reachable ---
|
||||||
|
log_info "Checking InfluxDB health..."
|
||||||
|
INFLUX_HEALTH=$(curl -sf "http://localhost:8086/health" 2>&1) || {
|
||||||
|
log_error "Failed to reach InfluxDB health endpoint"
|
||||||
|
FAILURES=$((FAILURES + 1))
|
||||||
|
}
|
||||||
|
|
||||||
|
if echo "$INFLUX_HEALTH" | grep -q '"status":"pass"'; then
|
||||||
|
log_info " [PASS] InfluxDB is healthy"
|
||||||
|
else
|
||||||
|
log_error " [FAIL] InfluxDB health check failed"
|
||||||
|
FAILURES=$((FAILURES + 1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Step 5b: Verify Grafana is reachable ---
|
||||||
|
log_info "Checking Grafana health..."
|
||||||
|
GRAFANA_HEALTH=""
|
||||||
|
elapsed=0
|
||||||
|
while [ $elapsed -lt $MAX_GRAFANA_WAIT ]; do
|
||||||
|
GRAFANA_HEALTH=$(curl -sf "$GRAFANA_URL" 2>&1) && break
|
||||||
|
sleep 2
|
||||||
|
elapsed=$((elapsed + 2))
|
||||||
|
done
|
||||||
|
|
||||||
|
if echo "$GRAFANA_HEALTH" | grep -Eq '"database"[[:space:]]*:[[:space:]]*"ok"'; then
|
||||||
|
log_info " [PASS] Grafana is healthy"
|
||||||
|
else
|
||||||
|
log_error " [FAIL] Grafana health check failed"
|
||||||
|
FAILURES=$((FAILURES + 1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Step 5c: Verify EVOLV measurement node produced output ---
|
||||||
|
log_info "Checking EVOLV measurement node output in container logs..."
|
||||||
|
wait_for_log_pattern "Database Output" "EVOLV measurement node produced database output" true || true
|
||||||
|
wait_for_log_pattern "Process Output" "EVOLV measurement node produced process output" true || true
|
||||||
|
wait_for_log_pattern "Monster Process Output" "EVOLV monster node produced process output" true || true
|
||||||
|
wait_for_log_pattern "Monster Database Output" "EVOLV monster node produced database output" true || true
|
||||||
|
wait_for_log_pattern "Diffuser Process Output" "EVOLV diffuser node produced process output" true || true
|
||||||
|
wait_for_log_pattern "Diffuser Database Output" "EVOLV diffuser node produced database output" true || true
|
||||||
|
wait_for_log_pattern "DashboardAPI Output" "EVOLV dashboardapi node produced create output" true || true
|
||||||
|
|
||||||
|
# --- Step 6: Summary ---
|
||||||
|
echo ""
|
||||||
|
if [ $FAILURES -eq 0 ]; then
|
||||||
|
log_info "========================================="
|
||||||
|
log_info " E2E tests PASSED - all checks green"
|
||||||
|
log_info "========================================="
|
||||||
|
exit 0
|
||||||
|
else
|
||||||
|
log_error "========================================="
|
||||||
|
log_error " E2E tests FAILED - $FAILURES check(s) failed"
|
||||||
|
log_error "========================================="
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
89
wiki/SCHEMA.md
Normal file
89
wiki/SCHEMA.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
# Project Wiki Schema
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
LLM-maintained knowledge base for this project. The LLM writes and maintains everything. You read it (ideally in Obsidian). Knowledge compounds across sessions instead of being lost in chat history.
|
||||||
|
|
||||||
|
## Directory Structure
|
||||||
|
```
|
||||||
|
wiki/
|
||||||
|
SCHEMA.md — this file (how to maintain the wiki)
|
||||||
|
index.md — catalog of all pages with one-line summaries
|
||||||
|
log.md — chronological record of updates
|
||||||
|
overview.md — project overview and current status
|
||||||
|
metrics.md — all numbers with provenance
|
||||||
|
knowledge-graph.yaml — structured data, machine-queryable
|
||||||
|
tools/ — search, lint, query scripts
|
||||||
|
concepts/ — core ideas and mechanisms
|
||||||
|
architecture/ — design decisions, system internals
|
||||||
|
findings/ — honest results (what worked AND what didn't)
|
||||||
|
sessions/ — per-session summaries
|
||||||
|
```
|
||||||
|
|
||||||
|
## Page Conventions
|
||||||
|
|
||||||
|
### Frontmatter
|
||||||
|
Every page starts with YAML frontmatter:
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
title: Page Title
|
||||||
|
created: YYYY-MM-DD
|
||||||
|
updated: YYYY-MM-DD
|
||||||
|
status: proven | disproven | evolving | speculative
|
||||||
|
tags: [tag1, tag2]
|
||||||
|
sources: [path/to/file.py, commit abc1234]
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
### Status values
|
||||||
|
- **proven**: tested and verified with evidence
|
||||||
|
- **disproven**: tested and honestly shown NOT to work (document WHY)
|
||||||
|
- **evolving**: partially working, boundary not fully mapped
|
||||||
|
- **speculative**: proposed but not yet tested
|
||||||
|
|
||||||
|
### Cross-references
|
||||||
|
Use `[[Page Name]]` Obsidian-style wikilinks.
|
||||||
|
|
||||||
|
### Contradictions
|
||||||
|
When new evidence contradicts a prior claim, DON'T delete the old claim. Add:
|
||||||
|
```
|
||||||
|
> [!warning] Superseded
|
||||||
|
> This was shown to be incorrect on YYYY-MM-DD. See [[New Finding]].
|
||||||
|
```
|
||||||
|
|
||||||
|
### Honesty rule
|
||||||
|
If something doesn't work, say so. If a result was a false positive, document how it was discovered. The wiki must be trustworthy.
|
||||||
|
|
||||||
|
## Operations
|
||||||
|
|
||||||
|
### Ingest (after a session or new source)
|
||||||
|
1. Read outputs, commits, findings
|
||||||
|
2. Update relevant pages
|
||||||
|
3. Create new pages for new concepts
|
||||||
|
4. Update `index.md`, `log.md`, `knowledge-graph.yaml`
|
||||||
|
5. Check for contradictions with existing pages
|
||||||
|
|
||||||
|
### Query
|
||||||
|
1. Use `python3 wiki/tools/query.py` for structured lookup
|
||||||
|
2. Use `wiki/tools/search.sh` for full-text
|
||||||
|
3. Read `index.md` to find relevant pages
|
||||||
|
4. File valuable answers back into the wiki
|
||||||
|
|
||||||
|
### Lint (periodically)
|
||||||
|
```bash
|
||||||
|
bash wiki/tools/lint.sh
|
||||||
|
```
|
||||||
|
Checks: orphan pages, broken wikilinks, missing frontmatter, index completeness.
|
||||||
|
|
||||||
|
## Data Layer
|
||||||
|
|
||||||
|
- `knowledge-graph.yaml` — structured YAML with every metric and data point
|
||||||
|
- `metrics.md` — human-readable dashboard
|
||||||
|
- When adding new results, update BOTH the wiki page AND the knowledge graph
|
||||||
|
- The knowledge graph is the single source of truth for numbers
|
||||||
|
|
||||||
|
## Source of Truth Hierarchy
|
||||||
|
1. **Test results** (actual outputs) — highest authority
|
||||||
|
2. **Code** (current state) — second authority
|
||||||
|
3. **Knowledge graph** (knowledge-graph.yaml) — structured metrics
|
||||||
|
4. **Wiki pages** — synthesis, may lag
|
||||||
|
5. **Chat/memory** — ephemeral, may be stale
|
||||||
56
wiki/architecture/3d-pump-curves.md
Normal file
56
wiki/architecture/3d-pump-curves.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
title: 3D Pump Curve Architecture
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [predict, curves, interpolation, rotatingMachine]
|
||||||
|
sources: [nodes/generalFunctions/src/predict/predict_class.js, nodes/rotatingMachine/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# 3D Pump Curve Prediction
|
||||||
|
|
||||||
|
## Data Structure
|
||||||
|
A family of 2D curves indexed by pressure (f-dimension):
|
||||||
|
- **X-axis**: control position (0-100%)
|
||||||
|
- **Y-axis**: flow (nq) or power (np) in canonical units
|
||||||
|
- **F-dimension**: pressure (Pa) — the 3rd dimension
|
||||||
|
|
||||||
|
Raw curves are in curve units (m3/h, kW, mbar). `_normalizeMachineCurve()` converts to canonical (m3/s, W, Pa).
|
||||||
|
|
||||||
|
## Interpolation
|
||||||
|
Monotonic cubic spline (Fritsch-Carlson) in both dimensions:
|
||||||
|
- **X-Y splines**: at each discrete pressure level
|
||||||
|
- **F-splines**: across pressure levels for intermediate pressure interpolation
|
||||||
|
|
||||||
|
## Prediction Flow
|
||||||
|
```
|
||||||
|
predict.y(x):
|
||||||
|
1. Clamp x to [currentFxyXMin, currentFxyXMax]
|
||||||
|
2. Normalize x to [normMin, normMax]
|
||||||
|
3. Evaluate spline at normalized x for current fDimension
|
||||||
|
4. Return y in canonical units (m3/s or W)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Unit Conversion Chain
|
||||||
|
```
|
||||||
|
Raw curve (m3/h, kW, mbar)
|
||||||
|
→ _normalizeMachineCurve → canonical (m3/s, W, Pa)
|
||||||
|
→ predict class → canonical output
|
||||||
|
→ MeasurementContainer.getCurrentValue(outputUnit) → output units
|
||||||
|
```
|
||||||
|
|
||||||
|
No double-conversion. Clean separation: specificClass handles units, predict handles normalization/interpolation.
|
||||||
|
|
||||||
|
## Three Predict Instances per Machine
|
||||||
|
- `predictFlow`: control % → flow (nq curve)
|
||||||
|
- `predictPower`: control % → power (np curve)
|
||||||
|
- `predictCtrl`: flow → control % (reversed nq curve)
|
||||||
|
|
||||||
|
## Boundary Behavior
|
||||||
|
- Below/above curve X range: flat extrapolation (clamped)
|
||||||
|
- Below/above f-dimension range: clamped to min/max pressure level
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
- `y(x)`: O(log n), effectively O(1) for 5-10 data points
|
||||||
|
- `buildAllFxyCurves`: sub-10ms for typical curves
|
||||||
|
- Full caching of normalized curves, splines, and calculated curves
|
||||||
@@ -1,3 +1,11 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Deployment Blueprint
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [deployment, docker, edge, site, central]
|
||||||
|
---
|
||||||
|
|
||||||
# EVOLV Deployment Blueprint
|
# EVOLV Deployment Blueprint
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
45
wiki/architecture/group-optimization.md
Normal file
45
wiki/architecture/group-optimization.md
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
---
|
||||||
|
title: Group Optimization Architecture
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, optimization, BEP-Gravitation]
|
||||||
|
sources: [nodes/machineGroupControl/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# machineGroupControl Optimization
|
||||||
|
|
||||||
|
## Algorithm: BEP-Gravitation + Marginal-Cost Refinement
|
||||||
|
|
||||||
|
### Step 1 — Pressure Equalization
|
||||||
|
Sets all non-operational pumps to the group's max downstream / min upstream pressure. Ensures fair curve evaluation across combinations.
|
||||||
|
|
||||||
|
### Step 2 — Combination Enumeration
|
||||||
|
Generates all 2^n pump subsets (n = number of machines). Filters by:
|
||||||
|
- Machine state (excludes off, cooling, stopping, emergency)
|
||||||
|
- Mode compatibility (`execsequence` allowed in auto)
|
||||||
|
- Flow bounds: `sumMinFlow ≤ Qd ≤ sumMaxFlow`
|
||||||
|
- Optional power cap
|
||||||
|
|
||||||
|
### Step 3 — BEP-Gravitation Distribution (per combination)
|
||||||
|
1. **BEP seed**: `estimatedBEP = minFlow + span * NCog` per pump
|
||||||
|
2. **Slope estimation**: samples dP/dQ at BEP ± delta (directional: slopeLeft, slopeRight)
|
||||||
|
3. **Slope redistribution**: iteratively shifts flow from steep to flat curves (weight = 1/slope)
|
||||||
|
4. **Marginal-cost refinement**: after slope redistribution, shifts flow from highest actual dP/dQ to lowest using real `inputFlowCalcPower` evaluations. Converges regardless of curve convexity. Max 50 iterations, typically 5-15.
|
||||||
|
|
||||||
|
### Step 4 — Best Selection
|
||||||
|
Pick combination with lowest total power. Tiebreak by deviation from BEP.
|
||||||
|
|
||||||
|
### Step 5 — Execution
|
||||||
|
Start/stop pumps as needed, send `flowmovement` commands in output units via `_canonicalToOutputFlow()`.
|
||||||
|
|
||||||
|
## Three Control Modes
|
||||||
|
|
||||||
|
| Mode | Distribution | Combination Selection |
|
||||||
|
|------|-------------|----------------------|
|
||||||
|
| optimalControl | BEP-Gravitation + refinement | exhaustive 2^n |
|
||||||
|
| priorityControl | equal split, priority-ordered | sequential add/remove |
|
||||||
|
| priorityPercentageControl | percentage-based, normalized | count-based |
|
||||||
|
|
||||||
|
## Key Design Decision
|
||||||
|
The `flowmovement` command sends flow in the **machine's output units** (m3/h), not canonical (m3/s). The `_canonicalToOutputFlow()` helper converts before sending. Without this conversion, every pump stays at minimum flow (the critical bug fixed on 2026-04-07).
|
||||||
426
wiki/architecture/node-architecture.md
Normal file
426
wiki/architecture/node-architecture.md
Normal file
@@ -0,0 +1,426 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Architecture
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [architecture, node-red, three-layer]
|
||||||
|
---
|
||||||
|
|
||||||
|
# EVOLV Architecture
|
||||||
|
|
||||||
|
## 1. System Overview
|
||||||
|
|
||||||
|
High-level view of how EVOLV fits into the wastewater treatment automation stack.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
NR[Node-RED Runtime] <-->|msg objects| EVOLV[EVOLV Nodes]
|
||||||
|
EVOLV -->|InfluxDB line protocol| INFLUX[(InfluxDB)]
|
||||||
|
INFLUX -->|queries| GRAFANA[Grafana Dashboards]
|
||||||
|
EVOLV -->|process output| NR
|
||||||
|
EVOLV -->|parent output| NR
|
||||||
|
|
||||||
|
style NR fill:#b22222,color:#fff
|
||||||
|
style EVOLV fill:#0f52a5,color:#fff
|
||||||
|
style INFLUX fill:#0c99d9,color:#fff
|
||||||
|
style GRAFANA fill:#50a8d9,color:#fff
|
||||||
|
```
|
||||||
|
|
||||||
|
Each EVOLV node produces three outputs:
|
||||||
|
| Port | Name | Purpose |
|
||||||
|
|------|------|---------|
|
||||||
|
| 0 | process | Process data forwarded to downstream nodes |
|
||||||
|
| 1 | dbase | InfluxDB-formatted measurement data |
|
||||||
|
| 2 | parent | Control messages to parent nodes (e.g. registerChild) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Node Architecture (Three-Layer Pattern)
|
||||||
|
|
||||||
|
Every node follows a consistent three-layer design that separates Node-RED wiring from domain logic.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TB
|
||||||
|
subgraph "Node-RED Runtime"
|
||||||
|
REG["RED.nodes.registerType()"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Layer 1 — Wrapper (valve.js)"
|
||||||
|
W[wrapper .js]
|
||||||
|
W -->|"new nodeClass(config, RED, this, name)"| NC
|
||||||
|
W -->|MenuManager| MENU[HTTP /name/menu.js]
|
||||||
|
W -->|configManager| CFG[HTTP /name/configData.js]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Layer 2 — Node Adapter (src/nodeClass.js)"
|
||||||
|
NC[nodeClass]
|
||||||
|
NC -->|_loadConfig| CFGM[configManager]
|
||||||
|
NC -->|_setupSpecificClass| SC
|
||||||
|
NC -->|_attachInputHandler| INPUT[onInput routing]
|
||||||
|
NC -->|_startTickLoop| TICK[1s tick loop]
|
||||||
|
NC -->|_tick → outputUtils| OUT[formatMsg]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Layer 3 — Domain Logic (src/specificClass.js)"
|
||||||
|
SC[specificClass]
|
||||||
|
SC -->|measurements| MC[MeasurementContainer]
|
||||||
|
SC -->|state machine| ST[state]
|
||||||
|
SC -->|hydraulics / biology| DOMAIN[domain models]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "generalFunctions"
|
||||||
|
GF[shared library]
|
||||||
|
end
|
||||||
|
|
||||||
|
REG --> W
|
||||||
|
GF -.->|logger, outputUtils, configManager,\nMeasurementContainer, validation, ...| NC
|
||||||
|
GF -.->|MeasurementContainer, state,\nconvert, predict, ...| SC
|
||||||
|
|
||||||
|
style W fill:#0f52a5,color:#fff
|
||||||
|
style NC fill:#0c99d9,color:#fff
|
||||||
|
style SC fill:#50a8d9,color:#fff
|
||||||
|
style GF fill:#86bbdd,color:#000
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. generalFunctions Module Map
|
||||||
|
|
||||||
|
The shared library (`nodes/generalFunctions/`) provides all cross-cutting concerns.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TB
|
||||||
|
GF[generalFunctions/index.js]
|
||||||
|
|
||||||
|
subgraph "Core Helpers (src/helper/)"
|
||||||
|
LOGGER[logger]
|
||||||
|
OUTPUT[outputUtils]
|
||||||
|
CHILD[childRegistrationUtils]
|
||||||
|
CFGUTIL[configUtils]
|
||||||
|
ASSERT[assertionUtils]
|
||||||
|
VALID[validationUtils]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Validators (src/helper/validators/)"
|
||||||
|
TV[typeValidators]
|
||||||
|
CV[collectionValidators]
|
||||||
|
CURV[curveValidator]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Domain Modules (src/)"
|
||||||
|
MC[MeasurementContainer]
|
||||||
|
CFGMGR[configManager]
|
||||||
|
MENUMGR[MenuManager]
|
||||||
|
STATE[state]
|
||||||
|
CONVERT[convert / Fysics]
|
||||||
|
PREDICT[predict / interpolation]
|
||||||
|
NRMSE[nrmse / errorMetrics]
|
||||||
|
COOLPROP[coolprop]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Data (datasets/)"
|
||||||
|
CURVES[assetData/curves]
|
||||||
|
ASSETS[assetData/assetData.json]
|
||||||
|
UNITS[unitData.json]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Constants (src/constants/)"
|
||||||
|
POS[POSITIONS / POSITION_VALUES]
|
||||||
|
end
|
||||||
|
|
||||||
|
GF --> LOGGER
|
||||||
|
GF --> OUTPUT
|
||||||
|
GF --> CHILD
|
||||||
|
GF --> CFGUTIL
|
||||||
|
GF --> ASSERT
|
||||||
|
GF --> VALID
|
||||||
|
VALID --> TV
|
||||||
|
VALID --> CV
|
||||||
|
VALID --> CURV
|
||||||
|
GF --> MC
|
||||||
|
GF --> CFGMGR
|
||||||
|
GF --> MENUMGR
|
||||||
|
GF --> STATE
|
||||||
|
GF --> CONVERT
|
||||||
|
GF --> PREDICT
|
||||||
|
GF --> NRMSE
|
||||||
|
GF --> COOLPROP
|
||||||
|
GF --> CURVES
|
||||||
|
GF --> POS
|
||||||
|
|
||||||
|
style GF fill:#0f52a5,color:#fff
|
||||||
|
style LOGGER fill:#86bbdd,color:#000
|
||||||
|
style OUTPUT fill:#86bbdd,color:#000
|
||||||
|
style VALID fill:#86bbdd,color:#000
|
||||||
|
style MC fill:#50a8d9,color:#fff
|
||||||
|
style CFGMGR fill:#50a8d9,color:#fff
|
||||||
|
style MENUMGR fill:#50a8d9,color:#fff
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Data Flow (Message Lifecycle)
|
||||||
|
|
||||||
|
Sequence diagram showing a typical input message and the periodic tick output cycle.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant NR as Node-RED
|
||||||
|
participant W as wrapper.js
|
||||||
|
participant NC as nodeClass
|
||||||
|
participant SC as specificClass
|
||||||
|
participant OU as outputUtils
|
||||||
|
|
||||||
|
Note over W: Node startup
|
||||||
|
W->>NC: new nodeClass(config, RED, node, name)
|
||||||
|
NC->>NC: _loadConfig (configManager.buildConfig)
|
||||||
|
NC->>SC: new specificClass(config, stateConfig, options)
|
||||||
|
NC->>NR: send([null, null, {topic: registerChild}])
|
||||||
|
|
||||||
|
Note over NC: Every 1 second (tick loop)
|
||||||
|
NC->>SC: getOutput()
|
||||||
|
SC-->>NC: raw measurement data
|
||||||
|
NC->>OU: formatMsg(raw, config, 'process')
|
||||||
|
NC->>OU: formatMsg(raw, config, 'influxdb')
|
||||||
|
NC->>NR: send([processMsg, influxMsg])
|
||||||
|
|
||||||
|
Note over NR: Incoming control message
|
||||||
|
NR->>W: msg {topic: 'execMovement', payload: {...}}
|
||||||
|
W->>NC: onInput(msg)
|
||||||
|
NC->>SC: handleInput(source, action, setpoint)
|
||||||
|
SC->>SC: update state machine & measurements
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Node Types
|
||||||
|
|
||||||
|
| Node | S88 Level | Purpose |
|
||||||
|
|------|-----------|---------|
|
||||||
|
| **measurement** | Control Module | Generic measurement point — reads, validates, and stores sensor values |
|
||||||
|
| **valve** | Control Module | Valve simulation with hydraulic model, position control, flow/pressure prediction |
|
||||||
|
| **rotatingMachine** | Control Module | Pumps, blowers, mixers — rotating equipment with speed control and efficiency curves |
|
||||||
|
| **diffuser** | Control Module | Aeration diffuser — models oxygen transfer and pressure drop |
|
||||||
|
| **settler** | Equipment | Sludge settler — models settling behavior and sludge blanket |
|
||||||
|
| **reactor** | Equipment | Hydraulic tank and biological process simulator (activated sludge, digestion) |
|
||||||
|
| **monster** | Equipment | MONitoring and STrEam Routing — complex measurement aggregation |
|
||||||
|
| **pumpingStation** | Unit | Coordinates multiple pumps as a pumping station |
|
||||||
|
| **valveGroupControl** | Unit | Manages multiple valves as a coordinated group — distributes flow, monitors pressure |
|
||||||
|
| **machineGroupControl** | Unit | Group control for rotating machines — load balancing and sequencing |
|
||||||
|
| **dashboardAPI** | Utility | Exposes data and unit conversion endpoints for external dashboards |
|
||||||
|
# EVOLV Architecture
|
||||||
|
|
||||||
|
## Node Hierarchy (S88)
|
||||||
|
|
||||||
|
EVOLV follows the ISA-88 (S88) batch control standard. Each node maps to an S88 level and uses a consistent color scheme in the Node-RED editor.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
classDef area fill:#0f52a5,color:#fff,stroke:#0a3d7a
|
||||||
|
classDef processCell fill:#0c99d9,color:#fff,stroke:#0977aa
|
||||||
|
classDef unit fill:#50a8d9,color:#fff,stroke:#3d89b3
|
||||||
|
classDef equipment fill:#86bbdd,color:#000,stroke:#6a9bb8
|
||||||
|
classDef controlModule fill:#a9daee,color:#000,stroke:#87b8cc
|
||||||
|
classDef standalone fill:#f0f0f0,color:#000,stroke:#999
|
||||||
|
|
||||||
|
%% S88 Levels
|
||||||
|
subgraph "S88: Area"
|
||||||
|
PS[pumpingStation]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "S88: Equipment"
|
||||||
|
MGC[machineGroupControl]
|
||||||
|
VGC[valveGroupControl]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "S88: Control Module"
|
||||||
|
RM[rotatingMachine]
|
||||||
|
V[valve]
|
||||||
|
M[measurement]
|
||||||
|
R[reactor]
|
||||||
|
S[settler]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph "Standalone"
|
||||||
|
MON[monster]
|
||||||
|
DASH[dashboardAPI]
|
||||||
|
DIFF[diffuser - not implemented]
|
||||||
|
end
|
||||||
|
|
||||||
|
%% Parent-child registration relationships
|
||||||
|
PS -->|"accepts: measurement"| M
|
||||||
|
PS -->|"accepts: machine"| RM
|
||||||
|
PS -->|"accepts: machineGroup"| MGC
|
||||||
|
PS -->|"accepts: pumpingStation"| PS2[pumpingStation]
|
||||||
|
|
||||||
|
MGC -->|"accepts: machine"| RM
|
||||||
|
|
||||||
|
RM -->|"accepts: measurement"| M2[measurement]
|
||||||
|
RM -->|"accepts: reactor"| R
|
||||||
|
|
||||||
|
VGC -->|"accepts: valve"| V
|
||||||
|
VGC -->|"accepts: machine / rotatingmachine"| RM2[rotatingMachine]
|
||||||
|
VGC -->|"accepts: machinegroup / machinegroupcontrol"| MGC2[machineGroupControl]
|
||||||
|
VGC -->|"accepts: pumpingstation / valvegroupcontrol"| PS3["pumpingStation / valveGroupControl"]
|
||||||
|
|
||||||
|
R -->|"accepts: measurement"| M3[measurement]
|
||||||
|
R -->|"accepts: reactor"| R2[reactor]
|
||||||
|
|
||||||
|
S -->|"accepts: measurement"| M4[measurement]
|
||||||
|
S -->|"accepts: reactor"| R3[reactor]
|
||||||
|
S -->|"accepts: machine"| RM3[rotatingMachine]
|
||||||
|
|
||||||
|
%% Styling
|
||||||
|
class PS,PS2,PS3 area
|
||||||
|
class MGC,MGC2 equipment
|
||||||
|
class VGC equipment
|
||||||
|
class RM,RM2,RM3 controlModule
|
||||||
|
class V controlModule
|
||||||
|
class M,M2,M3,M4 controlModule
|
||||||
|
class R,R2,R3 controlModule
|
||||||
|
class S controlModule
|
||||||
|
class MON,DASH,DIFF standalone
|
||||||
|
```
|
||||||
|
|
||||||
|
### Registration Summary
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
classDef parent fill:#0c99d9,color:#fff
|
||||||
|
classDef child fill:#a9daee,color:#000
|
||||||
|
|
||||||
|
PS[pumpingStation] -->|measurement| LEAF1((leaf))
|
||||||
|
PS -->|machine| RM1[rotatingMachine]
|
||||||
|
PS -->|machineGroup| MGC1[machineGroupControl]
|
||||||
|
PS -->|pumpingStation| PS1[pumpingStation]
|
||||||
|
|
||||||
|
MGC[machineGroupControl] -->|machine| RM2[rotatingMachine]
|
||||||
|
|
||||||
|
VGC[valveGroupControl] -->|valve| V1[valve]
|
||||||
|
VGC -->|source| SRC["machine, machinegroup,<br/>pumpingstation, valvegroupcontrol"]
|
||||||
|
|
||||||
|
RM[rotatingMachine] -->|measurement| LEAF2((leaf))
|
||||||
|
RM -->|reactor| R1[reactor]
|
||||||
|
|
||||||
|
R[reactor] -->|measurement| LEAF3((leaf))
|
||||||
|
R -->|reactor| R2[reactor]
|
||||||
|
|
||||||
|
S[settler] -->|measurement| LEAF4((leaf))
|
||||||
|
S -->|reactor| R3[reactor]
|
||||||
|
S -->|machine| RM3[rotatingMachine]
|
||||||
|
|
||||||
|
class PS,MGC,VGC,RM,R,S parent
|
||||||
|
class LEAF1,LEAF2,LEAF3,LEAF4,RM1,RM2,RM3,MGC1,PS1,V1,SRC,R1,R2,R3 child
|
||||||
|
```
|
||||||
|
|
||||||
|
## Node Types
|
||||||
|
|
||||||
|
| Node | S88 Level | softwareType | role | Accepts Children | Outputs |
|
||||||
|
|------|-----------|-------------|------|-----------------|---------|
|
||||||
|
| **pumpingStation** | Area | `pumpingstation` | StationController | measurement, machine (rotatingMachine), machineGroup, pumpingStation | [process, dbase, parent] |
|
||||||
|
| **machineGroupControl** | Equipment | `machinegroupcontrol` | GroupController | machine (rotatingMachine) | [process, dbase, parent] |
|
||||||
|
| **valveGroupControl** | Equipment | `valvegroupcontrol` | ValveGroupController | valve, machine, rotatingmachine, machinegroup, machinegroupcontrol, pumpingstation, valvegroupcontrol | [process, dbase, parent] |
|
||||||
|
| **rotatingMachine** | Control Module | `rotatingmachine` | RotationalDeviceController | measurement, reactor | [process, dbase, parent] |
|
||||||
|
| **valve** | Control Module | `valve` | controller | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||||
|
| **measurement** | Control Module | `measurement` | Sensor | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||||
|
| **reactor** | Control Module | `reactor` | Biological reactor | measurement, reactor (upstream chaining) | [process, dbase, parent] |
|
||||||
|
| **settler** | Control Module | `settler` | Secondary settler | measurement, reactor (upstream), machine (return pump) | [process, dbase, parent] |
|
||||||
|
| **monster** | Standalone | - | - | dual-parent, standalone | - |
|
||||||
|
| **dashboardAPI** | Standalone | - | - | accepts any child (Grafana integration) | - |
|
||||||
|
| **diffuser** | Standalone | - | - | _(not implemented)_ | - |
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
|
||||||
|
### Measurement Data Flow (upstream to downstream)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Sensor as measurement (sensor)
|
||||||
|
participant Machine as rotatingMachine
|
||||||
|
participant Group as machineGroupControl
|
||||||
|
participant Station as pumpingStation
|
||||||
|
|
||||||
|
Note over Sensor: Sensor reads value<br/>(pressure, flow, level, temp)
|
||||||
|
|
||||||
|
Sensor->>Sensor: measurements.type(t).variant("measured").position(p).value(v)
|
||||||
|
Sensor->>Sensor: emitter.emit("type.measured.position", eventData)
|
||||||
|
|
||||||
|
Sensor->>Machine: Event: "pressure.measured.upstream"
|
||||||
|
Machine->>Machine: Store in own MeasurementContainer
|
||||||
|
Machine->>Machine: getMeasuredPressure() -> calcFlow() -> calcPower()
|
||||||
|
Machine->>Machine: emitter.emit("flow.predicted.downstream", eventData)
|
||||||
|
|
||||||
|
Machine->>Group: Event: "flow.predicted.downstream"
|
||||||
|
Group->>Group: handlePressureChange()
|
||||||
|
Group->>Group: Aggregate flows across all machines
|
||||||
|
Group->>Group: Calculate group totals and efficiency
|
||||||
|
|
||||||
|
Machine->>Station: Event: "flow.predicted.downstream"
|
||||||
|
Station->>Station: Store predicted flow in/out
|
||||||
|
Station->>Station: _updateVolumePrediction()
|
||||||
|
Station->>Station: _calcNetFlow(), _calcTimeRemaining()
|
||||||
|
```
|
||||||
|
|
||||||
|
### Control Command Flow (downstream to upstream)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Station as pumpingStation
|
||||||
|
participant Group as machineGroupControl
|
||||||
|
participant Machine as rotatingMachine
|
||||||
|
participant Machine2 as rotatingMachine (2)
|
||||||
|
|
||||||
|
Station->>Group: handleInput("parent", action, param)
|
||||||
|
|
||||||
|
Group->>Group: Determine scaling strategy
|
||||||
|
Group->>Group: Calculate setpoints per machine
|
||||||
|
|
||||||
|
Group->>Machine: handleInput("parent", "execMovement", setpoint)
|
||||||
|
Group->>Machine2: handleInput("parent", "execMovement", setpoint)
|
||||||
|
|
||||||
|
Machine->>Machine: setpoint() -> state.moveTo(pos)
|
||||||
|
Machine->>Machine: updatePosition() -> calcFlow(), calcPower()
|
||||||
|
Machine->>Machine: emitter.emit("flow.predicted.downstream")
|
||||||
|
|
||||||
|
Machine2->>Machine2: setpoint() -> state.moveTo(pos)
|
||||||
|
Machine2->>Machine2: updatePosition() -> calcFlow(), calcPower()
|
||||||
|
Machine2->>Machine2: emitter.emit("flow.predicted.downstream")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Wastewater Treatment Process Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
classDef process fill:#50a8d9,color:#fff
|
||||||
|
classDef equipment fill:#86bbdd,color:#000
|
||||||
|
|
||||||
|
PS_IN[pumpingStation<br/>Influent] -->|flow| R1[reactor<br/>Anoxic]
|
||||||
|
R1 -->|effluent| R2[reactor<br/>Aerated]
|
||||||
|
R2 -->|effluent| SET[settler]
|
||||||
|
SET -->|effluent out| PS_OUT[pumpingStation<br/>Effluent]
|
||||||
|
SET -->|sludge return| RM_RET[rotatingMachine<br/>Return pump]
|
||||||
|
RM_RET -->|recirculation| R1
|
||||||
|
|
||||||
|
PS_IN --- MGC_IN[machineGroupControl]
|
||||||
|
MGC_IN --- RM_IN[rotatingMachine<br/>Influent pumps]
|
||||||
|
|
||||||
|
class PS_IN,PS_OUT process
|
||||||
|
class R1,R2,SET process
|
||||||
|
class MGC_IN,RM_IN,RM_RET equipment
|
||||||
|
```
|
||||||
|
|
||||||
|
### Event-Driven Communication Pattern
|
||||||
|
|
||||||
|
All parent-child communication uses Node.js `EventEmitter`:
|
||||||
|
|
||||||
|
1. **Registration**: Parent calls `childRegistrationUtils.registerChild(child, position)` which stores the child and calls the parent's `registerChild(child, softwareType)` method.
|
||||||
|
2. **Event binding**: The parent's `registerChild()` subscribes to the child's `measurements.emitter` events (e.g., `"flow.predicted.downstream"`).
|
||||||
|
3. **Data propagation**: When a child updates a measurement, it emits an event. The parent's listener stores the value in its own `MeasurementContainer` and runs its domain logic.
|
||||||
|
4. **Three outputs**: Every node sends data to three Node-RED outputs: `[process, dbase, parent]` -- process data for downstream nodes, InfluxDB for persistence, and parent aggregation data.
|
||||||
|
|
||||||
|
### Position Convention
|
||||||
|
|
||||||
|
Children register with a position relative to their parent:
|
||||||
|
- `upstream` -- before the parent in the flow direction
|
||||||
|
- `downstream` -- after the parent in the flow direction
|
||||||
|
- `atEquipment` -- physically located at/on the parent equipment
|
||||||
@@ -1,3 +1,11 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Platform Architecture
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [architecture, platform, edge-first]
|
||||||
|
---
|
||||||
|
|
||||||
# EVOLV Platform Architecture
|
# EVOLV Platform Architecture
|
||||||
|
|
||||||
## At A Glance
|
## At A Glance
|
||||||
@@ -1,3 +1,11 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Architecture Review
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [architecture, stack, review]
|
||||||
|
---
|
||||||
|
|
||||||
# EVOLV Architecture Review
|
# EVOLV Architecture Review
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
454
wiki/concepts/generalfunctions-api.md
Normal file
454
wiki/concepts/generalfunctions-api.md
Normal file
@@ -0,0 +1,454 @@
|
|||||||
|
---
|
||||||
|
title: generalFunctions API Reference
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [api, generalFunctions, reference]
|
||||||
|
---
|
||||||
|
|
||||||
|
# generalFunctions API Reference
|
||||||
|
|
||||||
|
Shared library (`nodes/generalFunctions/`) used across all EVOLV Node-RED nodes.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { logger, outputUtils, MeasurementContainer, ... } = require('generalFunctions');
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
1. [Logger](#logger)
|
||||||
|
2. [OutputUtils](#outpututils)
|
||||||
|
3. [ValidationUtils](#validationutils)
|
||||||
|
4. [MeasurementContainer](#measurementcontainer)
|
||||||
|
5. [ConfigManager](#configmanager)
|
||||||
|
6. [ChildRegistrationUtils](#childregistrationutils)
|
||||||
|
7. [MenuUtils](#menuutils)
|
||||||
|
8. [EndpointUtils](#endpointutils)
|
||||||
|
9. [Positions](#positions)
|
||||||
|
10. [AssetLoader / loadCurve](#assetloader--loadcurve)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Logger
|
||||||
|
|
||||||
|
Structured, level-filtered console logger.
|
||||||
|
|
||||||
|
**File:** `src/helper/logger.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new Logger(logging = true, logLevel = 'debug', nameModule = 'N/A')
|
||||||
|
```
|
||||||
|
|
||||||
|
| Param | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `logging` | `boolean` | `true` | Enable/disable all output |
|
||||||
|
| `logLevel` | `string` | `'debug'` | Minimum severity: `'debug'` \| `'info'` \| `'warn'` \| `'error'` |
|
||||||
|
| `nameModule` | `string` | `'N/A'` | Label prefixed to every message |
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `debug` | `(message: string): void` | Log at DEBUG level |
|
||||||
|
| `info` | `(message: string): void` | Log at INFO level |
|
||||||
|
| `warn` | `(message: string): void` | Log at WARN level |
|
||||||
|
| `error` | `(message: string): void` | Log at ERROR level |
|
||||||
|
| `setLogLevel` | `(level: string): void` | Change minimum level at runtime |
|
||||||
|
| `toggleLogging` | `(): void` | Flip logging on/off |
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const Logger = require('generalFunctions').logger;
|
||||||
|
const log = new Logger(true, 'info', 'MyNode');
|
||||||
|
log.info('Node started'); // [INFO] -> MyNode: Node started
|
||||||
|
log.debug('ignored'); // silent (below 'info')
|
||||||
|
log.setLogLevel('debug');
|
||||||
|
log.debug('now visible'); // [DEBUG] -> MyNode: now visible
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## OutputUtils
|
||||||
|
|
||||||
|
Tracks output state and formats messages for InfluxDB or process outputs. Only emits changed fields.
|
||||||
|
|
||||||
|
**File:** `src/helper/outputUtils.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new OutputUtils() // no parameters
|
||||||
|
```
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `formatMsg` | `(output, config, format)` | `object \| undefined` | Diff against last output; returns formatted msg or `undefined` if nothing changed |
|
||||||
|
| `checkForChanges` | `(output, format)` | `object` | Returns only the key/value pairs that changed since last call |
|
||||||
|
|
||||||
|
**`format`** must be `'influxdb'` or `'process'`.
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const out = new OutputUtils();
|
||||||
|
const msg = out.formatMsg(
|
||||||
|
{ temperature: 22.5, pressure: 1013 },
|
||||||
|
config,
|
||||||
|
'influxdb'
|
||||||
|
);
|
||||||
|
// msg = { topic: 'nodeName', payload: { measurement, fields, tags, timestamp } }
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ValidationUtils
|
||||||
|
|
||||||
|
Schema-driven config validation with type coercion, range clamping, and nested object support.
|
||||||
|
|
||||||
|
**File:** `src/helper/validationUtils.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new ValidationUtils(loggerEnabled = true, loggerLevel = 'warn')
|
||||||
|
```
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `validateSchema` | `(config, schema, name)` | `object` | Walk the schema, validate every field, return a clean config. Unknown keys are stripped. Missing keys get their schema default. |
|
||||||
|
| `constrain` | `(value, min, max)` | `number` | Clamp a numeric value to `[min, max]` |
|
||||||
|
| `removeUnwantedKeys` | `(obj)` | `object` | Strip `rules`/`description` metadata, collapse `default` values |
|
||||||
|
|
||||||
|
**Supported `rules.type` values:** `number`, `integer`, `boolean`, `string`, `enum`, `array`, `set`, `object`, `curve`, `machineCurve`.
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const ValidationUtils = require('generalFunctions').validation;
|
||||||
|
const v = new ValidationUtils(true, 'warn');
|
||||||
|
|
||||||
|
const schema = {
|
||||||
|
temperature: { default: 20, rules: { type: 'number', min: -40, max: 100 } },
|
||||||
|
unit: { default: 'C', rules: { type: 'enum', values: [{ value: 'C' }, { value: 'F' }] } }
|
||||||
|
};
|
||||||
|
|
||||||
|
const validated = v.validateSchema({ temperature: 999 }, schema, 'myNode');
|
||||||
|
// validated.temperature === 100 (clamped)
|
||||||
|
// validated.unit === 'C' (default applied)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## MeasurementContainer
|
||||||
|
|
||||||
|
Chainable measurement storage organised by **type / variant / position**. Supports auto unit conversion, windowed statistics, events, and positional difference calculations.
|
||||||
|
|
||||||
|
**File:** `src/measurements/MeasurementContainer.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new MeasurementContainer(options = {}, logger)
|
||||||
|
```
|
||||||
|
|
||||||
|
| Option | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `windowSize` | `number` | `10` | Rolling window for statistics |
|
||||||
|
| `defaultUnits` | `object` | `{ pressure:'mbar', flow:'m3/h', ... }` | Default unit per measurement type |
|
||||||
|
| `autoConvert` | `boolean` | `true` | Auto-convert values to target unit |
|
||||||
|
| `preferredUnits` | `object` | `{}` | Per-type unit overrides |
|
||||||
|
|
||||||
|
### Chainable Setters
|
||||||
|
|
||||||
|
All return `this` for chaining.
|
||||||
|
|
||||||
|
```js
|
||||||
|
container
|
||||||
|
.type('pressure')
|
||||||
|
.variant('static')
|
||||||
|
.position('upstream')
|
||||||
|
.distance(5)
|
||||||
|
.unit('bar')
|
||||||
|
.value(3.2, Date.now(), 'bar');
|
||||||
|
```
|
||||||
|
|
||||||
|
| Method | Signature | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `type` | `(typeName): this` | Set measurement type (e.g. `'pressure'`) |
|
||||||
|
| `variant` | `(variantName): this` | Set variant (e.g. `'static'`, `'differential'`) |
|
||||||
|
| `position` | `(positionValue): this` | Set position (e.g. `'upstream'`, `'downstream'`) |
|
||||||
|
| `distance` | `(distance): this` | Set physical distance from parent |
|
||||||
|
| `unit` | `(unitName): this` | Set unit on the underlying measurement |
|
||||||
|
| `value` | `(val, timestamp?, sourceUnit?): this` | Store a value; auto-converts if `sourceUnit` differs from target |
|
||||||
|
|
||||||
|
### Terminal / Query Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `get` | `()` | `Measurement \| null` | Get the raw measurement object |
|
||||||
|
| `getCurrentValue` | `(requestedUnit?)` | `number \| null` | Latest value, optionally converted |
|
||||||
|
| `getAverage` | `(requestedUnit?)` | `number \| null` | Windowed average |
|
||||||
|
| `getMin` | `()` | `number \| null` | Window minimum |
|
||||||
|
| `getMax` | `()` | `number \| null` | Window maximum |
|
||||||
|
| `getAllValues` | `()` | `array \| null` | All stored samples |
|
||||||
|
| `getLaggedValue` | `(lag?, requestedUnit?)` | `number \| null` | Value from `lag` samples ago |
|
||||||
|
| `getLaggedSample` | `(lag?, requestedUnit?)` | `object \| null` | Full sample `{ value, timestamp, unit }` from `lag` samples ago |
|
||||||
|
| `exists` | `({ type?, variant?, position?, requireValues? })` | `boolean` | Check if a measurement series exists |
|
||||||
|
| `difference` | `({ from?, to?, unit? })` | `object \| null` | Compute `{ value, avgDiff, unit }` between two positions |
|
||||||
|
|
||||||
|
### Introspection / Lifecycle
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `getTypes` | `()` | `string[]` | All registered measurement types |
|
||||||
|
| `getVariants` | `()` | `string[]` | Variants under current type |
|
||||||
|
| `getPositions` | `()` | `string[]` | Positions under current type+variant |
|
||||||
|
| `getAvailableUnits` | `(measurementType?)` | `string[]` | Units available for a type |
|
||||||
|
| `getBestUnit` | `(excludeUnits?)` | `object \| null` | Best human-readable unit for current value |
|
||||||
|
| `setPreferredUnit` | `(type, unit)` | `this` | Override default unit for a type |
|
||||||
|
| `setChildId` | `(id)` | `this` | Tag container with a child node ID |
|
||||||
|
| `setChildName` | `(name)` | `this` | Tag container with a child node name |
|
||||||
|
| `setParentRef` | `(parent)` | `this` | Store reference to parent node |
|
||||||
|
| `clear` | `()` | `void` | Reset all measurements and chain state |
|
||||||
|
|
||||||
|
### Events
|
||||||
|
|
||||||
|
The internal `emitter` fires `"type.variant.position"` on every `value()` call with:
|
||||||
|
|
||||||
|
```js
|
||||||
|
{ value, originalValue, unit, sourceUnit, timestamp, position, distance, variant, type, childId, childName, parentRef }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { MeasurementContainer } = require('generalFunctions');
|
||||||
|
const mc = new MeasurementContainer({ windowSize: 5 });
|
||||||
|
|
||||||
|
mc.type('pressure').variant('static').position('upstream').value(3.2);
|
||||||
|
mc.type('pressure').variant('static').position('downstream').value(2.8);
|
||||||
|
|
||||||
|
const diff = mc.type('pressure').variant('static').difference();
|
||||||
|
// diff = { value: -0.4, avgDiff: -0.4, unit: 'mbar', from: 'downstream', to: 'upstream' }
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ConfigManager
|
||||||
|
|
||||||
|
Loads JSON config files from disk and builds merged runtime configs.
|
||||||
|
|
||||||
|
**File:** `src/configs/index.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new ConfigManager(relPath = '.')
|
||||||
|
```
|
||||||
|
|
||||||
|
`relPath` is resolved relative to the configs directory.
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `getConfig` | `(configName)` | `object` | Load and parse `<configName>.json` |
|
||||||
|
| `getAvailableConfigs` | `()` | `string[]` | List config names (without `.json`) |
|
||||||
|
| `hasConfig` | `(configName)` | `boolean` | Check existence |
|
||||||
|
| `getBaseConfig` | `()` | `object` | Shortcut for `getConfig('baseConfig')` |
|
||||||
|
| `buildConfig` | `(nodeName, uiConfig, nodeId, domainConfig?)` | `object` | Merge base schema + UI overrides into a runtime config |
|
||||||
|
| `createEndpoint` | `(nodeName)` | `string` | Generate browser JS that injects config into `window.EVOLV.nodes` |
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { configManager } = require('generalFunctions');
|
||||||
|
const cfg = configManager.buildConfig('measurement', uiConfig, node.id, {
|
||||||
|
scaling: { enabled: true, inputMin: 0, inputMax: 100 }
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ChildRegistrationUtils
|
||||||
|
|
||||||
|
Manages parent-child node relationships: registration, lookup, and structure storage.
|
||||||
|
|
||||||
|
**File:** `src/helper/childRegistrationUtils.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new ChildRegistrationUtils(mainClass)
|
||||||
|
```
|
||||||
|
|
||||||
|
`mainClass` is the parent node instance (must expose `.logger` and optionally `.registerChild()`).
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `registerChild` | `(child, positionVsParent, distance?)` | `Promise<any>` | Register a child node under the parent. Sets up parent refs, measurement context, and stores by softwareType/category. |
|
||||||
|
| `getChildrenOfType` | `(softwareType, category?)` | `array` | Get children filtered by software type and optional category |
|
||||||
|
| `getChildById` | `(childId)` | `object \| null` | Lookup a single child by its ID |
|
||||||
|
| `getAllChildren` | `()` | `array` | All registered children |
|
||||||
|
| `logChildStructure` | `()` | `void` | Debug-print the full child tree |
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { childRegistrationUtils: CRU } = require('generalFunctions');
|
||||||
|
const cru = new CRU(parentNode);
|
||||||
|
await cru.registerChild(sensorNode, 'upstream');
|
||||||
|
cru.getChildrenOfType('measurement'); // [sensorNode]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## MenuUtils
|
||||||
|
|
||||||
|
Browser-side UI helper for Node-RED editor. Methods are mixed in from separate modules: toggles, data fetching, URL utils, dropdown population, and HTML generation.
|
||||||
|
|
||||||
|
**File:** `src/helper/menuUtils.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new MenuUtils() // no parameters; sets isCloud=false, configData=null
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Methods
|
||||||
|
|
||||||
|
**Toggles** -- control UI element visibility:
|
||||||
|
|
||||||
|
| Method | Signature | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `initBasicToggles` | `(elements)` | Bind log-level row visibility to log checkbox |
|
||||||
|
| `initMeasurementToggles` | `(elements)` | Bind scaling input rows to scaling checkbox |
|
||||||
|
| `initTensionToggles` | `(elements, node)` | Show/hide tension row based on interpolation method |
|
||||||
|
|
||||||
|
**Data Fetching:**
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `fetchData` | `(url, fallbackUrl)` | `Promise<array>` | Fetch JSON from primary URL; fall back on failure |
|
||||||
|
| `fetchProjectData` | `(url)` | `Promise<object>` | Fetch project-level data |
|
||||||
|
| `apiCall` | `(node)` | `Promise<object>` | POST to asset-register API |
|
||||||
|
|
||||||
|
**URL Construction:**
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `getSpecificConfigUrl` | `(nodeName, cloudAPI)` | `{ cloudConfigURL, localConfigURL }` | Build cloud + local config URLs |
|
||||||
|
| `constructUrl` | `(base, ...paths)` | `string` | Join URL segments safely |
|
||||||
|
| `constructCloudURL` | `(base, ...paths)` | `string` | Same as `constructUrl`, for cloud endpoints |
|
||||||
|
|
||||||
|
**Dropdown Population:**
|
||||||
|
|
||||||
|
| Method | Signature | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `fetchAndPopulateDropdowns` | `(configUrls, elements, node)` | Cascading supplier > subType > model > unit dropdowns |
|
||||||
|
| `populateDropdown` | `(htmlElement, options, node, property, callback?)` | Fill a `<select>` with options and wire change events |
|
||||||
|
| `populateLogLevelOptions` | `(logLevelSelect, configData, node)` | Populate log-level dropdown from config |
|
||||||
|
| `populateSmoothingMethods` | `(configUrls, elements, node)` | Populate smoothing method dropdown |
|
||||||
|
| `populateInterpolationMethods` | `(configUrls, elements, node)` | Populate interpolation method dropdown |
|
||||||
|
| `generateHtml` | `(htmlElement, options, savedValue)` | Write `<option>` HTML into an element |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## EndpointUtils
|
||||||
|
|
||||||
|
Server-side helper that serves `MenuUtils` as browser JavaScript via Node-RED HTTP endpoints.
|
||||||
|
|
||||||
|
**File:** `src/helper/endpointUtils.js`
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```js
|
||||||
|
new EndpointUtils({ MenuUtilsClass? })
|
||||||
|
```
|
||||||
|
|
||||||
|
| Param | Type | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `MenuUtilsClass` | `class` | `MenuUtils` | The MenuUtils constructor to introspect |
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
| Method | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `createMenuUtilsEndpoint` | `(RED, nodeName, customHelpers?)` | `void` | Register `GET /<nodeName>/resources/menuUtils.js` |
|
||||||
|
| `generateMenuUtilsCode` | `(nodeName, customHelpers?)` | `string` | Produce the browser JS string (introspects `MenuUtils.prototype`) |
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const EndpointUtils = require('generalFunctions/src/helper/endpointUtils');
|
||||||
|
const ep = new EndpointUtils();
|
||||||
|
ep.createMenuUtilsEndpoint(RED, 'valve');
|
||||||
|
// Browser can now load: GET /valve/resources/menuUtils.js
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Positions
|
||||||
|
|
||||||
|
Canonical constants for parent-child spatial relationships.
|
||||||
|
|
||||||
|
**File:** `src/constants/positions.js`
|
||||||
|
|
||||||
|
### Exports
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { POSITIONS, POSITION_VALUES, isValidPosition } = require('generalFunctions');
|
||||||
|
```
|
||||||
|
|
||||||
|
| Export | Type | Value |
|
||||||
|
|---|---|---|
|
||||||
|
| `POSITIONS` | `object` | `{ UPSTREAM: 'upstream', DOWNSTREAM: 'downstream', AT_EQUIPMENT: 'atEquipment', DELTA: 'delta' }` |
|
||||||
|
| `POSITION_VALUES` | `string[]` | `['upstream', 'downstream', 'atEquipment', 'delta']` |
|
||||||
|
| `isValidPosition` | `(pos: string): boolean` | Returns `true` if `pos` is one of the four values |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## AssetLoader / loadCurve
|
||||||
|
|
||||||
|
Loads JSON asset files (machine curves, etc.) from the datasets directory with LRU caching.
|
||||||
|
|
||||||
|
**File:** `datasets/assetData/curves/index.js`
|
||||||
|
|
||||||
|
### Singleton convenience functions
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { loadCurve } = require('generalFunctions');
|
||||||
|
```
|
||||||
|
|
||||||
|
| Function | Signature | Returns | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `loadCurve` | `(curveType: string)` | `object \| null` | Load `<curveType>.json` from the curves directory |
|
||||||
|
| `loadAsset` | `(datasetType, assetId)` | `object \| null` | Load any JSON asset by dataset folder and ID |
|
||||||
|
| `getAvailableAssets` | `(datasetType)` | `string[]` | List asset IDs in a dataset folder |
|
||||||
|
|
||||||
|
### AssetLoader class
|
||||||
|
|
||||||
|
```js
|
||||||
|
new AssetLoader(maxCacheSize = 100)
|
||||||
|
```
|
||||||
|
|
||||||
|
Same methods as above (`loadCurve`, `loadAsset`, `getAvailableAssets`), plus `clearCache()`.
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
const { loadCurve } = require('generalFunctions');
|
||||||
|
const curve = loadCurve('hidrostal-H05K-S03R');
|
||||||
|
// curve = { flow: [...], head: [...], ... } or null
|
||||||
|
```
|
||||||
38
wiki/findings/bep-gravitation-proof.md
Normal file
38
wiki/findings/bep-gravitation-proof.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
---
|
||||||
|
title: BEP-Gravitation Optimality Proof
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, optimization, BEP, brute-force]
|
||||||
|
sources: [nodes/machineGroupControl/test/integration/distribution-power-table.integration.test.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# BEP-Gravitation vs Brute-Force Global Optimum
|
||||||
|
|
||||||
|
## Claim
|
||||||
|
The machineGroupControl BEP-Gravitation algorithm (with marginal-cost refinement) produces near-optimal flow distribution across a pump group.
|
||||||
|
|
||||||
|
## Method
|
||||||
|
Brute-force exhaustive search: 1000 steps per pump, all 2^n combinations, 0.05% flow tolerance. Station: 2x H05K-S03R + 1x C5-D03R-SHN1 @ ΔP=2000 mbar.
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
| Demand | Brute force | machineGroupControl | Gap |
|
||||||
|
|--------|------------|--------------------|----|
|
||||||
|
| 10% (71 m3/h) | 17.65 kW | 17.63 kW | -0.10% (MGC wins) |
|
||||||
|
| 25% (136 m3/h) | 34.33 kW | 34.33 kW | +0.01% |
|
||||||
|
| 50% (243 m3/h) | 61.62 kW | 61.62 kW | -0.00% |
|
||||||
|
| 75% (351 m3/h) | 96.01 kW | 96.10 kW | +0.08% |
|
||||||
|
| 90% (415 m3/h) | 122.17 kW | 122.26 kW | +0.07% |
|
||||||
|
|
||||||
|
Maximum deviation: **0.1%** from proven global optimum.
|
||||||
|
|
||||||
|
## Why the Refinement Matters
|
||||||
|
|
||||||
|
Before the marginal-cost refinement loop, the gap at 50% demand was **2.12%**. The BEP-Gravitation slope estimate pushed 14.6 m3/h to C5 (costing 5.0 kW) when the optimum was 6.5 m3/h (0.59 kW). The refinement loop corrects this by shifting flow from highest actual dP/dQ to lowest until no improvement is possible.
|
||||||
|
|
||||||
|
## Stability
|
||||||
|
Sweep 5-95% in 2% steps: 1 switch (rising), 1 switch (falling), same transition point. No hysteresis. See [[Pump Switching Stability]].
|
||||||
|
|
||||||
|
## Computational Cost
|
||||||
|
0.027-0.153ms median per optimization call (3 pumps, 6 combinations). Uses 0.015% of the 1000ms tick budget.
|
||||||
34
wiki/findings/curve-non-convexity.md
Normal file
34
wiki/findings/curve-non-convexity.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
title: Pump Curve Non-Convexity
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [curves, interpolation, C5, non-convex]
|
||||||
|
sources: [nodes/generalFunctions/datasets/assetData/curves/hidrostal-C5-D03R-SHN1.json]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pump Curve Non-Convexity from Sparse Data
|
||||||
|
|
||||||
|
## Finding
|
||||||
|
The C5-D03R-SHN1 pump's power curve is non-convex after spline interpolation. The marginal cost (dP/dQ) shows a spike-then-valley pattern:
|
||||||
|
|
||||||
|
```
|
||||||
|
C5 dP/dQ across flow range @ ΔP=2000 mbar:
|
||||||
|
6.4 m3/h → 1,316,610 (high)
|
||||||
|
10.2 m3/h → 2,199,349 (spikes UP)
|
||||||
|
17.7 m3/h → 1,114,700 (dropping)
|
||||||
|
21.5 m3/h → 453,316 (valley — cheapest)
|
||||||
|
29.0 m3/h → 1,048,375 (rising again)
|
||||||
|
44.1 m3/h → 1,107,708 (high)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Root Cause
|
||||||
|
The C5 curve has only **5 raw data points** per pressure level. The monotonic cubic spline (Fritsch-Carlson) creates a smooth curve through all 5 points, but with such sparse data it introduces non-convex regions that don't match the physical convexity of a real pump.
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
- The equal-marginal-cost theorem (KKT conditions) does not apply — it requires convexity
|
||||||
|
- The BEP-Gravitation slope estimate at a single point can be misleading in non-convex regions
|
||||||
|
- The marginal-cost refinement loop fixes this by using actual power evaluations instead of slope assumptions
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
Add more data points (15-20 per pressure level) to the C5 curve. This would make the spline track the real convex physics more closely, eliminating the non-convex artifacts.
|
||||||
42
wiki/findings/ncog-behavior.md
Normal file
42
wiki/findings/ncog-behavior.md
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
---
|
||||||
|
title: NCog Behavior and Limitations
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [rotatingMachine, NCog, BEP, efficiency]
|
||||||
|
sources: [nodes/rotatingMachine/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# NCog — Normalized Center of Gravity
|
||||||
|
|
||||||
|
## What It Is
|
||||||
|
NCog is a 0-1 value indicating where on its flow range a pump operates most efficiently. Computed per tick from the current pressure slice of the 3D pump curve.
|
||||||
|
|
||||||
|
```
|
||||||
|
BEP_flow = minFlow + (maxFlow - minFlow) * NCog
|
||||||
|
```
|
||||||
|
|
||||||
|
## How It's Computed
|
||||||
|
1. Pressure sensors update → `getMeasuredPressure()` computes differential
|
||||||
|
2. `fDimension` locks the 2D slice at current system pressure
|
||||||
|
3. `calcCog()` computes Q/P (specific flow) across the curve
|
||||||
|
4. Peak Q/P index → `NCog = (flowAtPeak - flowMin) / (flowMax - flowMin)`
|
||||||
|
|
||||||
|
## When NCog is Meaningful
|
||||||
|
NCog requires **differential pressure** (upstream + downstream). With only one pressure sensor, fDimension is the raw sensor value (too high), producing a monotonic Q/P curve and NCog = 0.
|
||||||
|
|
||||||
|
| Condition | NCog for H05K | NCog for C5 |
|
||||||
|
|-----------|--------------|-------------|
|
||||||
|
| ΔP = 400 mbar | 0.333 | 0.355 |
|
||||||
|
| ΔP = 1000 mbar | 0.000 | 0.000 |
|
||||||
|
| ΔP = 1500 mbar | 0.135 | 0.000 |
|
||||||
|
| ΔP = 2000 mbar | 0.351 | 0.000 |
|
||||||
|
|
||||||
|
## Why NCog = 0 Happens
|
||||||
|
For variable-speed centrifugal pumps, Q/P is monotonically decreasing when the affinity laws dominate (P ∝ Q³). At certain pressure levels, the spline interpolation preserves this monotonicity and the peak is always at index 0 (minimum flow).
|
||||||
|
|
||||||
|
## How the machineGroupControl Uses NCog
|
||||||
|
The BEP-Gravitation algorithm seeds each pump at its BEP flow, then redistributes using slope-based weights + marginal-cost refinement. Even when NCog = 0, the slope redistribution produces near-optimal results because it uses actual power evaluations.
|
||||||
|
|
||||||
|
> [!warning] Disproven: NCog as proportional weight
|
||||||
|
> Using NCog directly as a flow-distribution weight (`flow = NCog/totalNCog * Qd`) is wrong. It starves pumps with NCog = 0 and overloads high-NCog pumps. See `calcBestCombination` in machineGroupControl.
|
||||||
88
wiki/findings/open-issues-2026-03.md
Normal file
88
wiki/findings/open-issues-2026-03.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
---
|
||||||
|
title: Open Issues — EVOLV Codebase
|
||||||
|
created: 2026-03-01
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [issues, backlog]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Open Issues — EVOLV Codebase
|
||||||
|
|
||||||
|
Issues identified during codebase scan (2026-03-12). Create these on Gitea when ready.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue 1: Restore diffuser node implementation
|
||||||
|
|
||||||
|
**Labels:** `enhancement`, `node`
|
||||||
|
**Priority:** Medium
|
||||||
|
|
||||||
|
The `nodes/diffuser/` directory contains only `.git`, `LICENSE`, and `README.md` — no implementation. There was a previous experimental version. Needs:
|
||||||
|
|
||||||
|
- Retrieve original diffuser logic from user/backup
|
||||||
|
- Rebuild to current three-layer architecture (wrapper `.js` + `src/nodeClass.js` + `src/specificClass.js`)
|
||||||
|
- Use `require('generalFunctions')` barrel imports
|
||||||
|
- Add config JSON in `generalFunctions/src/configs/diffuser.json`
|
||||||
|
- Register under category `'EVOLV'` with appropriate S88 color
|
||||||
|
- Add tests
|
||||||
|
|
||||||
|
**Blocked on:** User providing original diffuser logic/requirements.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue 2: Relocate prediction/ML modules to external service
|
||||||
|
|
||||||
|
**Labels:** `enhancement`, `architecture`
|
||||||
|
**Priority:** Medium
|
||||||
|
|
||||||
|
TensorFlow-based influent prediction code was removed from monster node (was broken/incomplete). The prediction functionality needs a new home:
|
||||||
|
|
||||||
|
- LSTM model for 24-hour flow prediction based on precipitation data
|
||||||
|
- Standardization constants: hours `(mean=11.504, std=6.922)`, precipitation `(mean=0.090, std=0.439)`, response `(mean=1188.01, std=1024.19)`
|
||||||
|
- Model was served from `http://127.0.0.1:1880/generalFunctions/datasets/lstmData/tfjs_model/`
|
||||||
|
- Consider: separate microservice, Python-based inference, or ONNX runtime
|
||||||
|
- Monster node should accept predictions via `model_prediction` message topic from external service
|
||||||
|
|
||||||
|
**Related files removed:** `monster_class.js` methods `get_model_prediction()`, `model_loader()`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue 3: Modernize monster node to three-layer architecture
|
||||||
|
|
||||||
|
**Labels:** `refactor`, `node`
|
||||||
|
**Priority:** Low
|
||||||
|
|
||||||
|
Monster node uses old-style structure (`dependencies/monster/` instead of `src/`). Should be refactored:
|
||||||
|
|
||||||
|
- Move `dependencies/monster/monster_class.js` → `src/specificClass.js`
|
||||||
|
- Create `src/nodeClass.js` adapter (extract from `monster.js`)
|
||||||
|
- Slim down `monster.js` to standard wrapper pattern
|
||||||
|
- Move `monsterConfig.json` → `generalFunctions/src/configs/monster.json`
|
||||||
|
- Remove `modelLoader.js` (TF dependency removed)
|
||||||
|
- Add unit tests
|
||||||
|
|
||||||
|
**Note:** monster_class.js is ~500 lines of domain logic. Keep sampling_program(), aggregation, AQUON integration intact.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue 4: Clean up inline test/demo code in specificClass files
|
||||||
|
|
||||||
|
**Labels:** `cleanup`
|
||||||
|
**Priority:** Low
|
||||||
|
|
||||||
|
Several specificClass files have test/demo code after `module.exports`:
|
||||||
|
|
||||||
|
- `pumpingStation/src/specificClass.js` (lines 478-697): Demo code guarded with `require.main === module` — acceptable but could move to `test/` or `examples/`
|
||||||
|
- `machineGroupControl/src/specificClass.js` (lines 969-1158): Block-commented test code with `makeMachines()` — dead code, could be removed or moved to test file
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue 5: DashboardAPI node improvements
|
||||||
|
|
||||||
|
**Labels:** `enhancement`, `security`
|
||||||
|
**Priority:** Low
|
||||||
|
|
||||||
|
- Bearer token now relies on `GRAFANA_TOKEN` env var (hardcoded token was removed for security)
|
||||||
|
- Ensure deployment docs mention setting `GRAFANA_TOKEN`
|
||||||
|
- `dashboardapi_class.js` still has `console.log` calls (lines 154, 178) — should use logger
|
||||||
|
- Node doesn't follow three-layer architecture (older style)
|
||||||
34
wiki/findings/pump-switching-stability.md
Normal file
34
wiki/findings/pump-switching-stability.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
title: Pump Switching Stability
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, stability, switching]
|
||||||
|
sources: [nodes/machineGroupControl/test/integration/ncog-distribution.integration.test.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pump Switching Stability
|
||||||
|
|
||||||
|
## Concern
|
||||||
|
Frequent pump on/off cycling causes mechanical wear, water hammer, and process disturbance.
|
||||||
|
|
||||||
|
## Test Method
|
||||||
|
Sweep demand from 5% to 95% in 2% steps, count combination changes. Repeat in reverse to check for hysteresis.
|
||||||
|
|
||||||
|
## Results — Mixed Station (2x H05K + 1x C5)
|
||||||
|
|
||||||
|
Rising 5→95%: **1 switch** at 27% (H05K-1+C5 → all 3)
|
||||||
|
Falling 95→5%: **1 switch** at 25% (all 3 → H05K-1+C5)
|
||||||
|
|
||||||
|
Same transition zone, no hysteresis.
|
||||||
|
|
||||||
|
## Results — Equal Station (3x H05K)
|
||||||
|
|
||||||
|
Rising 5→95%: **2 switches**
|
||||||
|
- 19%: 1 pump → 2 pumps
|
||||||
|
- 37%: 2 pumps → 3 pumps
|
||||||
|
|
||||||
|
Clean monotonic transitions, no flickering.
|
||||||
|
|
||||||
|
## Why It's Stable
|
||||||
|
The marginal-cost refinement only adjusts flow distribution WITHIN a combination — it never changes which pumps are selected. Combination selection is driven by total power comparison, which changes smoothly with demand.
|
||||||
48
wiki/index.md
Normal file
48
wiki/index.md
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
---
|
||||||
|
title: Wiki Index
|
||||||
|
updated: 2026-04-07
|
||||||
|
---
|
||||||
|
|
||||||
|
# EVOLV Project Wiki Index
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
- [Project Overview](overview.md) — what works, what doesn't, node inventory
|
||||||
|
- [Metrics Dashboard](metrics.md) — test counts, power comparison, performance
|
||||||
|
- [Knowledge Graph](knowledge-graph.yaml) — structured data, machine-queryable
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
- [Node Architecture](architecture/node-architecture.md) — three-layer pattern, ports, mermaid diagrams
|
||||||
|
- [3D Pump Curves](architecture/3d-pump-curves.md) — predict class, spline interpolation, unit chain
|
||||||
|
- [Group Optimization](architecture/group-optimization.md) — BEP-Gravitation, combination selection, marginal-cost refinement
|
||||||
|
- [Platform Overview](architecture/platform-overview.md) — edge/site/central layering, telemetry model
|
||||||
|
- [Deployment Blueprint](architecture/deployment-blueprint.md) — Docker topology, rollout order
|
||||||
|
- [Stack Review](architecture/stack-review.md) — full stack architecture assessment
|
||||||
|
|
||||||
|
## Core Concepts
|
||||||
|
- [generalFunctions API](concepts/generalfunctions-api.md) — logger, MeasurementContainer, configManager, etc.
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
- [BEP-Gravitation Proof](findings/bep-gravitation-proof.md) — within 0.1% of brute-force optimum (proven)
|
||||||
|
- [NCog Behavior](findings/ncog-behavior.md) — when NCog works, when it's zero, how it's used (evolving)
|
||||||
|
- [Curve Non-Convexity](findings/curve-non-convexity.md) — C5 sparse data artifacts (proven)
|
||||||
|
- [Pump Switching Stability](findings/pump-switching-stability.md) — 1-2 transitions, no hysteresis (proven)
|
||||||
|
- [Open Issues (2026-03)](findings/open-issues-2026-03.md) — diffuser, monster refactor, ML relocation, etc.
|
||||||
|
|
||||||
|
## Sessions
|
||||||
|
- [2026-04-07: Production Hardening](sessions/2026-04-07-production-hardening.md) — rotatingMachine + machineGroupControl
|
||||||
|
|
||||||
|
## Other Documentation (outside wiki)
|
||||||
|
- `CLAUDE.md` — Claude Code project guide (root)
|
||||||
|
- `AGENTS.md` — agent routing table, orchestrator policy (root, used by `.claude/agents/`)
|
||||||
|
- `.agents/` — skills, decisions, function-anchors, improvements
|
||||||
|
- `.claude/` — Claude Code agents and rules
|
||||||
|
- `manuals/node-red/` — FlowFuse dashboard and Node-RED reference docs
|
||||||
|
|
||||||
|
## Not Yet Documented
|
||||||
|
- Parent-child registration protocol (Port 2 handshake)
|
||||||
|
- Prediction health scoring algorithm (confidence 0-1)
|
||||||
|
- MeasurementContainer internals (chainable API, delta compression)
|
||||||
|
- PID controller implementation
|
||||||
|
- reactor / settler / monster / measurement / valve nodes
|
||||||
|
- pumpingStation node (uses rotatingMachine children)
|
||||||
|
- InfluxDB telemetry format (Port 1)
|
||||||
161
wiki/knowledge-graph.yaml
Normal file
161
wiki/knowledge-graph.yaml
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
# Knowledge Graph — structured data with provenance
|
||||||
|
# Every claim has: value, source (file/commit), date, status
|
||||||
|
|
||||||
|
# ── TESTS ──
|
||||||
|
tests:
|
||||||
|
rotatingMachine:
|
||||||
|
basic:
|
||||||
|
count: 10
|
||||||
|
passing: 10
|
||||||
|
file: nodes/rotatingMachine/test/basic/
|
||||||
|
date: 2026-04-07
|
||||||
|
integration:
|
||||||
|
count: 16
|
||||||
|
passing: 16
|
||||||
|
file: nodes/rotatingMachine/test/integration/
|
||||||
|
date: 2026-04-07
|
||||||
|
edge:
|
||||||
|
count: 17
|
||||||
|
passing: 17
|
||||||
|
file: nodes/rotatingMachine/test/edge/
|
||||||
|
date: 2026-04-07
|
||||||
|
machineGroupControl:
|
||||||
|
basic:
|
||||||
|
count: 1
|
||||||
|
passing: 1
|
||||||
|
file: nodes/machineGroupControl/test/basic/
|
||||||
|
date: 2026-04-07
|
||||||
|
integration:
|
||||||
|
count: 3
|
||||||
|
passing: 3
|
||||||
|
file: nodes/machineGroupControl/test/integration/
|
||||||
|
date: 2026-04-07
|
||||||
|
edge:
|
||||||
|
count: 1
|
||||||
|
passing: 1
|
||||||
|
file: nodes/machineGroupControl/test/edge/
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── METRICS ──
|
||||||
|
metrics:
|
||||||
|
optimization_gap_to_brute_force:
|
||||||
|
value: "0.1% max"
|
||||||
|
source: distribution-power-table.integration.test.js
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "3 pumps, 1000-step brute force, 0.05% flow tolerance"
|
||||||
|
optimization_time_median:
|
||||||
|
value: "0.027-0.153ms"
|
||||||
|
source: benchmark script
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "3 pumps, 6 combinations, BEP-Gravitation + refinement"
|
||||||
|
pump_switching_stability:
|
||||||
|
value: "1-2 transitions across 5-95% demand"
|
||||||
|
source: stability sweep
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "2% demand steps, both ascending and descending"
|
||||||
|
pump_curves:
|
||||||
|
H05K-S03R:
|
||||||
|
pressure_levels: 33
|
||||||
|
pressure_range: "700-3900 mbar"
|
||||||
|
flow_range: "28-227 m3/h (at 2000 mbar)"
|
||||||
|
data_points_per_level: 5
|
||||||
|
anomalies_fixed: 3
|
||||||
|
date: 2026-04-07
|
||||||
|
C5-D03R-SHN1:
|
||||||
|
pressure_levels: 26
|
||||||
|
pressure_range: "400-2900 mbar"
|
||||||
|
flow_range: "6-53 m3/h"
|
||||||
|
data_points_per_level: 5
|
||||||
|
non_convex: true
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── DISPROVEN CLAIMS ──
|
||||||
|
disproven:
|
||||||
|
ncog_proportional_weight:
|
||||||
|
claimed: "Distributing flow proportional to NCog weights is optimal"
|
||||||
|
claimed_date: 2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "Simple implementation in calcBestCombination"
|
||||||
|
evidence_against: "Starves small pumps (NCog=0 gets zero flow), overloads large pumps at high demand. BEP-target + scale is correct approach."
|
||||||
|
root_cause: "NCog is a position indicator (0-1 on flow range), not a distribution weight"
|
||||||
|
efficiency_rounding:
|
||||||
|
claimed: "Math.round(flow/power * 100) / 100 preserves BEP signal"
|
||||||
|
claimed_date: pre-2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "Removes floating point noise"
|
||||||
|
evidence_against: "In canonical units (m3/s and W), Q/P ratio is ~1e-6. Rounding to 2 decimals produces 0 for all points. NCog, cog, BEP all became 0."
|
||||||
|
root_cause: "Canonical units make the ratio very small — rounding destroys the signal"
|
||||||
|
equal_marginal_cost_optimal:
|
||||||
|
claimed: "Equal dP/dQ across pumps guarantees global power minimum"
|
||||||
|
claimed_date: 2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "KKT conditions for convex functions"
|
||||||
|
evidence_against: "C5 pump curve is non-convex (dP/dQ dips from 1.3M to 453K then rises). Sparse data (5 points) causes spline artifacts."
|
||||||
|
root_cause: "Convexity assumption fails with interpolated curves from sparse data"
|
||||||
|
|
||||||
|
# ── PERFORMANCE ──
|
||||||
|
performance:
|
||||||
|
mgc_optimization:
|
||||||
|
median_ms: 0.09
|
||||||
|
p99_ms: 0.5
|
||||||
|
tick_budget_pct: 0.015
|
||||||
|
source: benchmark script
|
||||||
|
date: 2026-04-07
|
||||||
|
predict_y_call:
|
||||||
|
complexity: "O(log n), ~O(1) for 5-10 data points"
|
||||||
|
source: predict_class.js
|
||||||
|
|
||||||
|
# ── ARCHITECTURE ──
|
||||||
|
architecture:
|
||||||
|
canonical_units:
|
||||||
|
pressure: Pa
|
||||||
|
flow: "m3/s"
|
||||||
|
power: W
|
||||||
|
temperature: K
|
||||||
|
output_units:
|
||||||
|
pressure: mbar
|
||||||
|
flow: "m3/h"
|
||||||
|
power: kW
|
||||||
|
temperature: C
|
||||||
|
node_count: 13
|
||||||
|
submodules: 12
|
||||||
|
|
||||||
|
# ── BUGS FIXED ──
|
||||||
|
bugs_fixed:
|
||||||
|
flowmovement_unit_mismatch:
|
||||||
|
severity: critical
|
||||||
|
description: "machineGroupControl sent flow in canonical (m3/s) but rotatingMachine flowmovement expected output units (m3/h). Every pump stayed at minimum."
|
||||||
|
fix: "_canonicalToOutputFlow() conversion before all flowmovement calls"
|
||||||
|
commit: d55f401
|
||||||
|
date: 2026-04-07
|
||||||
|
emergencystop_case:
|
||||||
|
severity: critical
|
||||||
|
description: "specificClass called executeSequence('emergencyStop') but config key was 'emergencystop'"
|
||||||
|
fix: "Lowercase to match config"
|
||||||
|
commit: 07af7ce
|
||||||
|
date: 2026-04-07
|
||||||
|
curve_data_anomalies:
|
||||||
|
severity: high
|
||||||
|
description: "3 flow values leaked into power column in hidrostal-H05K-S03R.json at pressures 1600, 3200, 3300 mbar"
|
||||||
|
fix: "Linearly interpolated correct values from adjacent levels"
|
||||||
|
commit: 024db55
|
||||||
|
date: 2026-04-07
|
||||||
|
efficiency_rounding:
|
||||||
|
severity: high
|
||||||
|
description: "Math.round(Q/P * 100) / 100 destroyed all NCog/BEP calculations"
|
||||||
|
fix: "Removed rounding, use raw ratio"
|
||||||
|
commit: 07af7ce
|
||||||
|
date: 2026-04-07
|
||||||
|
absolute_scaling_bug:
|
||||||
|
severity: high
|
||||||
|
description: "handleInput compared demandQout (always 0) instead of demandQ for max cap"
|
||||||
|
fix: "Reordered conditions, use demandQ throughout"
|
||||||
|
commit: d55f401
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── TIMELINE ──
|
||||||
|
timeline:
|
||||||
|
- {date: 2026-04-07, commit: 024db55, desc: "Fix 3 anomalous power values in hidrostal curve"}
|
||||||
|
- {date: 2026-04-07, commit: 07af7ce, desc: "rotatingMachine production hardening: safety + prediction + 43 tests"}
|
||||||
|
- {date: 2026-04-07, commit: d55f401, desc: "machineGroupControl: unit fix + refinement + stability tests"}
|
||||||
|
- {date: 2026-04-07, commit: fd9d167, desc: "Update EVOLV submodule refs"}
|
||||||
11
wiki/log.md
Normal file
11
wiki/log.md
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
---
|
||||||
|
title: Wiki Log
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wiki Log
|
||||||
|
|
||||||
|
## [2026-04-07] Wiki initialized | Full codebase scan + session findings
|
||||||
|
- Created overview, metrics, knowledge graph from production hardening session
|
||||||
|
- Architecture pages: 3D pump curves, group optimization
|
||||||
|
- Findings: BEP-Gravitation proof, NCog behavior, curve non-convexity, switching stability
|
||||||
|
- Session log: 2026-04-07 production hardening
|
||||||
56
wiki/metrics.md
Normal file
56
wiki/metrics.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
title: Metrics Dashboard
|
||||||
|
updated: 2026-04-07
|
||||||
|
---
|
||||||
|
|
||||||
|
# Metrics Dashboard
|
||||||
|
|
||||||
|
All numbers with provenance. Source of truth: `knowledge-graph.yaml`.
|
||||||
|
|
||||||
|
## Test Results
|
||||||
|
|
||||||
|
| Suite | Pass/Total | File | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| rotatingMachine basic | 10/10 | test/basic/*.test.js | 2026-04-07 |
|
||||||
|
| rotatingMachine integration | 16/16 | test/integration/*.test.js | 2026-04-07 |
|
||||||
|
| rotatingMachine edge | 17/17 | test/edge/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl basic | 1/1 | test/basic/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl integration | 3/3 | test/integration/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl edge | 1/1 | test/edge/*.test.js | 2026-04-07 |
|
||||||
|
|
||||||
|
## Performance — machineGroupControl Optimization
|
||||||
|
|
||||||
|
| Metric | Value | Source | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| BEP-Gravitation + refinement (3 pumps, 6 combos) | 0.027-0.153ms median | benchmark script | 2026-04-07 |
|
||||||
|
| Tick loop budget used | 0.015% of 1000ms | benchmark script | 2026-04-07 |
|
||||||
|
| Max gap from brute-force optimum (1000 steps) | 0.1% | [[BEP Gravitation Proof]] | 2026-04-07 |
|
||||||
|
| Pump switching stability (5-95% sweep) | 1-2 transitions, no hysteresis | stability sweep | 2026-04-07 |
|
||||||
|
|
||||||
|
## Performance — rotatingMachine Prediction
|
||||||
|
|
||||||
|
| Metric | Value | Source |
|
||||||
|
|---|---|---|
|
||||||
|
| predict.y(x) call | O(log n), effectively O(1) | predict_class.js |
|
||||||
|
| buildAllFxyCurves | sub-10ms for typical curves | predict_class.js |
|
||||||
|
| Curve cache | full caching of splines + calculated curves | predict_class.js |
|
||||||
|
|
||||||
|
## Power Comparison: machineGroupControl vs Baselines
|
||||||
|
|
||||||
|
Station: 2x H05K-S03R + 1x C5-D03R-SHN1 @ ΔP=2000 mbar
|
||||||
|
|
||||||
|
| Demand | Qd (m3/h) | machineGroupControl | Spillover | Equal-all | Gap to optimum |
|
||||||
|
|--------|-----------|--------------------|-----------|-----------|----|
|
||||||
|
| 10% | 71 | 17.6 kW | 22.0 kW (+25%) | 23.9 kW (+36%) | -0.10% |
|
||||||
|
| 25% | 136 | 34.6 kW | 36.3 kW (+5%) | 39.1 kW (+13%) | +0.01% |
|
||||||
|
| 50% | 243 | 62.9 kW | 73.8 kW (+17%) | 64.2 kW (+2%) | -0.00% |
|
||||||
|
| 75% | 351 | 96.8 kW | 102.9 kW (+6%) | 99.6 kW (+3%) | +0.08% |
|
||||||
|
| 90% | 415 | 122.8 kW | 123.0 kW (0%) | 123.0 kW (0%) | +0.07% |
|
||||||
|
|
||||||
|
## Disproven Claims
|
||||||
|
|
||||||
|
| Claim | Evidence For | Evidence Against | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| NCog as proportional weight works | Simple implementation | Starves small pumps, overloads large ones at high demand | 2026-04-07 |
|
||||||
|
| Q/P ratio always has mid-range peak | Expected from pump physics | Monotonically decreasing at high ΔP due to affinity laws (P ∝ Q³) | 2026-04-07 |
|
||||||
|
| Equal-marginal-cost solver is optimal | KKT theory for convex curves | C5 curve is non-convex due to sparse data points (5 per pressure) | 2026-04-07 |
|
||||||
70
wiki/overview.md
Normal file
70
wiki/overview.md
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Project Overview
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [overview, wastewater, node-red, isa-88]
|
||||||
|
---
|
||||||
|
|
||||||
|
# EVOLV — Edge-Layer Evolution for Optimized Virtualization
|
||||||
|
|
||||||
|
Industrial automation platform for wastewater treatment, built as custom Node-RED nodes by Waterschap Brabantse Delta R&D. Follows ISA-88 (S88) batch control standard.
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
Node.js, Node-RED, InfluxDB (time-series), TensorFlow.js (prediction), CoolProp (thermodynamics). No build step — pure Node.js.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Each node follows a 3-tier pattern:
|
||||||
|
1. **Entry file** — registers with Node-RED, admin HTTP endpoints
|
||||||
|
2. **nodeClass** — Node-RED adapter (tick loop, message routing, status)
|
||||||
|
3. **specificClass** — pure domain logic (physics, state machines, predictions)
|
||||||
|
|
||||||
|
3-port output convention: Port 0 = process data, Port 1 = InfluxDB telemetry, Port 2 = parent-child registration.
|
||||||
|
|
||||||
|
## What Works
|
||||||
|
|
||||||
|
| Capability | Status | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| rotatingMachine state machine | proven | 76 tests passing, all sequences verified |
|
||||||
|
| 3D pump curve prediction (flow/power from pressure+control) | proven | Monotonic cubic spline interpolation across 34 pressure levels |
|
||||||
|
| NCog / BEP tracking per pump | proven | Produces meaningful values with differential pressure |
|
||||||
|
| machineGroupControl BEP-Gravitation | proven | Within 0.1% of brute-force global optimum |
|
||||||
|
| Combination selection (2^n exhaustive) | proven | Stable: 1-2 switches across 5-95% demand sweep, no hysteresis |
|
||||||
|
| Prediction health scoring | proven | NRMSE drift, pressure source penalties, edge detection |
|
||||||
|
| Hydraulic efficiency (η = QΔP/P) | proven | CoolProp density, head calculation |
|
||||||
|
| Unit conversion chain | proven | No double-conversion, clean layer separation |
|
||||||
|
|
||||||
|
## What Doesn't Work (honestly)
|
||||||
|
|
||||||
|
| Issue | Status | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| C5 curve non-convexity | evolving | 5 raw data points cause spline artifacts, dP/dQ non-monotonic |
|
||||||
|
| NCog = 0 at high ΔP | evolving | At ΔP > 800 mbar for H05K, Q/P is monotonically decreasing |
|
||||||
|
| calcBestCombination (NCog-weight mode) | disproven | Uses NCog as proportional weight instead of BEP target |
|
||||||
|
|
||||||
|
## Current Scale
|
||||||
|
|
||||||
|
- 13 custom Node-RED nodes (12 submodules + generalFunctions)
|
||||||
|
- rotatingMachine: 76 tests, 1563 lines domain logic
|
||||||
|
- machineGroupControl: 90+ tests, 1400+ lines domain logic
|
||||||
|
- 3 real pump curves: H05K-S03R, C5-D03R-SHN1, ECDV
|
||||||
|
- Tick loop: 1000ms interval
|
||||||
|
|
||||||
|
## Node Inventory
|
||||||
|
|
||||||
|
| Node | Purpose | Test Status |
|
||||||
|
|------|---------|-------------|
|
||||||
|
| rotatingMachine | Pump/compressor control | 76 tests (full) |
|
||||||
|
| machineGroupControl | Multi-pump optimization | 90 tests (full) |
|
||||||
|
| pumpingStation | Multi-pump station | needs review |
|
||||||
|
| valve | Valve modeling | needs review |
|
||||||
|
| valveGroupControl | Valve group coordination | needs review |
|
||||||
|
| reactor | Biological reactor (ASM kinetics) | needs review |
|
||||||
|
| settler | Secondary clarifier | needs review |
|
||||||
|
| monster | Multi-parameter bio monitoring | needs review |
|
||||||
|
| measurement | Sensor signal conditioning | needs review |
|
||||||
|
| diffuser | Aeration system control | needs review |
|
||||||
|
| dashboardAPI | InfluxDB + FlowFuse charts | needs review |
|
||||||
|
| generalFunctions | Shared utilities | partial |
|
||||||
46
wiki/sessions/2026-04-07-production-hardening.md
Normal file
46
wiki/sessions/2026-04-07-production-hardening.md
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
---
|
||||||
|
title: "Session: Production Hardening rotatingMachine + machineGroupControl"
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [session, rotatingMachine, machineGroupControl, testing]
|
||||||
|
---
|
||||||
|
|
||||||
|
# 2026-04-07 — Production Hardening
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
Full code review and hardening of rotatingMachine and machineGroupControl nodes for production readiness.
|
||||||
|
|
||||||
|
## Key Discoveries
|
||||||
|
|
||||||
|
1. **Efficiency rounding destroyed NCog/BEP** — `Math.round(Q/P * 100) / 100` in canonical units (m3/s and W) produces ratios ~1e-6 that all round to 0. All NCog, cog, and BEP calculations were non-functional. Fixed by removing rounding.
|
||||||
|
|
||||||
|
2. **flowmovement unit mismatch** — machineGroupControl computed flow in canonical (m3/s) and sent it directly to rotatingMachine which expected output units (m3/h). Every pump stayed at minimum flow. Fixed with `_canonicalToOutputFlow()`.
|
||||||
|
|
||||||
|
3. **emergencyStop case mismatch** — `"emergencyStop"` vs config key `"emergencystop"`. Emergency stop never worked. Fixed to lowercase.
|
||||||
|
|
||||||
|
4. **Curve data anomalies** — 3 flow values leaked into power columns in hidrostal-H05K-S03R.json at pressures 1600, 3200, 3300 mbar. Fixed with interpolated values.
|
||||||
|
|
||||||
|
5. **C5 pump non-convexity** — 5 data points per pressure level produces non-convex spline. The marginal-cost refinement loop closes the gap to brute-force optimum from 2.1% to 0.1%.
|
||||||
|
|
||||||
|
## Changes Made
|
||||||
|
|
||||||
|
### rotatingMachine (3 files, 7 test files)
|
||||||
|
- Async input handler, null guards, listener cleanup, tick loop race fix
|
||||||
|
- showCoG() implementation, efficiency variant fix, curve anomaly detection
|
||||||
|
- 43 new tests (76 total)
|
||||||
|
|
||||||
|
### machineGroupControl (1 file, 2 test files)
|
||||||
|
- `_canonicalToOutputFlow()` on all flowmovement calls
|
||||||
|
- Absolute scaling bug, empty Qd block, empty-machines guards
|
||||||
|
- Marginal-cost refinement loop in BEP-Gravitation
|
||||||
|
- Missing flowmovement after startup in equalFlowControl
|
||||||
|
|
||||||
|
### generalFunctions (1 file)
|
||||||
|
- 3 curve data fixes in hidrostal-H05K-S03R.json
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
- 90 tests passing across both nodes
|
||||||
|
- machineGroupControl within 0.1% of brute-force global optimum (1000-step search)
|
||||||
|
- Pump switching stable: 1-2 transitions across full demand range, no hysteresis
|
||||||
|
- Optimization cost: 0.03-0.15ms per call (0.015% of tick budget)
|
||||||
46
wiki/tools/lint.sh
Normal file
46
wiki/tools/lint.sh
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Wiki health check — find issues
|
||||||
|
# Usage: ./wiki/tools/lint.sh
|
||||||
|
|
||||||
|
WIKI_DIR="$(dirname "$(dirname "$(readlink -f "$0")")")"
|
||||||
|
|
||||||
|
echo "=== Wiki Health Check ==="
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Page count --"
|
||||||
|
find "$WIKI_DIR" -name "*.md" -not -path "*/tools/*" | wc -l
|
||||||
|
echo " total pages"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Orphans (not linked from other pages) --"
|
||||||
|
for f in $(find "$WIKI_DIR" -name "*.md" -not -name "index.md" -not -name "log.md" -not -name "SCHEMA.md" -not -path "*/tools/*"); do
|
||||||
|
basename=$(basename "$f" .md)
|
||||||
|
refs=$(grep -rl --include="*.md" "$basename" "$WIKI_DIR" 2>/dev/null | grep -v "$f" | wc -l)
|
||||||
|
if [ "$refs" -eq 0 ]; then
|
||||||
|
echo " ORPHAN: $f"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Status distribution --"
|
||||||
|
for status in proven disproven evolving speculative; do
|
||||||
|
count=$(grep -rl "status: $status" "$WIKI_DIR" --include="*.md" 2>/dev/null | wc -l)
|
||||||
|
echo " $status: $count"
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Pages missing frontmatter --"
|
||||||
|
for f in $(find "$WIKI_DIR" -name "*.md" -not -name "SCHEMA.md" -not -path "*/tools/*"); do
|
||||||
|
if ! head -1 "$f" | grep -q "^---"; then
|
||||||
|
echo " NO FRONTMATTER: $f"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Index completeness --"
|
||||||
|
indexed=$(grep -c '\[.*\](.*\.md)' "$WIKI_DIR/index.md" 2>/dev/null)
|
||||||
|
total=$(find "$WIKI_DIR" -name "*.md" -not -name "index.md" -not -name "log.md" -not -name "SCHEMA.md" -not -path "*/tools/*" | wc -l)
|
||||||
|
echo " Indexed: $indexed / Total: $total"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "=== Done ==="
|
||||||
249
wiki/tools/query.py
Normal file
249
wiki/tools/query.py
Normal file
@@ -0,0 +1,249 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Wiki Knowledge Graph query tool.
|
||||||
|
|
||||||
|
Queryable interface over knowledge-graph.yaml + wiki pages.
|
||||||
|
Usable by both humans (CLI) and LLM agents (imported).
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python wiki/tools/query.py health # project health
|
||||||
|
python wiki/tools/query.py entity "search term" # everything about an entity
|
||||||
|
python wiki/tools/query.py metric "search term" # find metrics
|
||||||
|
python wiki/tools/query.py status "proven" # all pages with status
|
||||||
|
python wiki/tools/query.py test "test name" # test results
|
||||||
|
python wiki/tools/query.py search "keyword" # full-text search
|
||||||
|
python wiki/tools/query.py related "page-name" # pages linking to/from
|
||||||
|
python wiki/tools/query.py timeline # commit timeline
|
||||||
|
"""
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
WIKI_DIR = Path(__file__).parent.parent
|
||||||
|
GRAPH_PATH = WIKI_DIR / 'knowledge-graph.yaml'
|
||||||
|
|
||||||
|
|
||||||
|
def load_graph():
|
||||||
|
if not GRAPH_PATH.exists():
|
||||||
|
return {}
|
||||||
|
with open(GRAPH_PATH) as f:
|
||||||
|
return yaml.safe_load(f) or {}
|
||||||
|
|
||||||
|
|
||||||
|
def load_all_pages():
|
||||||
|
pages = {}
|
||||||
|
for md_path in WIKI_DIR.rglob('*.md'):
|
||||||
|
if 'tools' in str(md_path):
|
||||||
|
continue
|
||||||
|
rel = md_path.relative_to(WIKI_DIR)
|
||||||
|
content = md_path.read_text()
|
||||||
|
meta = {}
|
||||||
|
if content.startswith('---'):
|
||||||
|
parts = content.split('---', 2)
|
||||||
|
if len(parts) >= 3:
|
||||||
|
try:
|
||||||
|
meta = yaml.safe_load(parts[1]) or {}
|
||||||
|
except yaml.YAMLError:
|
||||||
|
pass
|
||||||
|
content = parts[2]
|
||||||
|
links = re.findall(r'\[\[([^\]]+)\]\]', content)
|
||||||
|
pages[str(rel)] = {
|
||||||
|
'path': str(rel), 'meta': meta, 'content': content,
|
||||||
|
'links': links, 'title': meta.get('title', str(rel)),
|
||||||
|
'status': meta.get('status', 'unknown'),
|
||||||
|
'tags': meta.get('tags', []),
|
||||||
|
}
|
||||||
|
return pages
|
||||||
|
|
||||||
|
|
||||||
|
def flatten_graph(graph, prefix=''):
|
||||||
|
items = []
|
||||||
|
if isinstance(graph, dict):
|
||||||
|
for k, v in graph.items():
|
||||||
|
path = f"{prefix}.{k}" if prefix else k
|
||||||
|
if isinstance(v, (dict, list)):
|
||||||
|
items.extend(flatten_graph(v, path))
|
||||||
|
else:
|
||||||
|
items.append((path, str(v)))
|
||||||
|
elif isinstance(graph, list):
|
||||||
|
for i, v in enumerate(graph):
|
||||||
|
path = f"{prefix}[{i}]"
|
||||||
|
if isinstance(v, (dict, list)):
|
||||||
|
items.extend(flatten_graph(v, path))
|
||||||
|
else:
|
||||||
|
items.append((path, str(v)))
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_health():
|
||||||
|
graph = load_graph()
|
||||||
|
pages = load_all_pages()
|
||||||
|
statuses = {}
|
||||||
|
for p in pages.values():
|
||||||
|
s = p['status']
|
||||||
|
statuses[s] = statuses.get(s, 0) + 1
|
||||||
|
|
||||||
|
tests = graph.get('tests', {})
|
||||||
|
total_pass = sum(t.get('passing', 0) for t in tests.values() if isinstance(t, dict))
|
||||||
|
total_count = sum(t.get('count', t.get('total', 0)) for t in tests.values() if isinstance(t, dict))
|
||||||
|
disproven = len(graph.get('disproven', {}))
|
||||||
|
timeline = len(graph.get('timeline', []))
|
||||||
|
|
||||||
|
# Count broken links
|
||||||
|
all_titles = set()
|
||||||
|
for p in pages.values():
|
||||||
|
all_titles.add(p['title'].lower())
|
||||||
|
all_titles.add(p['path'].lower().replace('.md', '').split('/')[-1])
|
||||||
|
broken = sum(1 for p in pages.values() for link in p['links']
|
||||||
|
if not any(link.lower().replace('-', ' ') in t or t in link.lower().replace('-', ' ')
|
||||||
|
for t in all_titles))
|
||||||
|
|
||||||
|
print(f"Wiki Health:\n")
|
||||||
|
print(f" Pages: {len(pages)}")
|
||||||
|
print(f" Statuses: {statuses}")
|
||||||
|
if total_count:
|
||||||
|
print(f" Tests: {total_pass}/{total_count} passing")
|
||||||
|
print(f" Disproven: {disproven} claims tracked")
|
||||||
|
print(f" Timeline: {timeline} commits")
|
||||||
|
print(f" Broken links: {broken}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_entity(query):
|
||||||
|
graph = load_graph()
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Entity: '{query}'\n")
|
||||||
|
|
||||||
|
flat = flatten_graph(graph)
|
||||||
|
hits = [(p, v) for p, v in flat if q in p.lower() or q in v.lower()]
|
||||||
|
if hits:
|
||||||
|
print(" -- Knowledge Graph --")
|
||||||
|
for path, value in hits[:20]:
|
||||||
|
print(f" {path}: {value}")
|
||||||
|
|
||||||
|
print("\n -- Wiki Pages --")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
if q in page['content'].lower() or q in page['title'].lower():
|
||||||
|
lines = [l.strip() for l in page['content'].split('\n')
|
||||||
|
if q in l.lower() and l.strip()]
|
||||||
|
print(f" {rel} ({page['status']})")
|
||||||
|
for line in lines[:3]:
|
||||||
|
print(f" {line[:100]}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_metric(query):
|
||||||
|
flat = flatten_graph(load_graph())
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Metrics matching '{query}':\n")
|
||||||
|
found = 0
|
||||||
|
for path, value in flat:
|
||||||
|
if q in path.lower() or q in value.lower():
|
||||||
|
print(f" {path}: {value}")
|
||||||
|
found += 1
|
||||||
|
if not found:
|
||||||
|
print(" (no matches)")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_status(status):
|
||||||
|
pages = load_all_pages()
|
||||||
|
graph = load_graph()
|
||||||
|
print(f"Status: '{status}'\n")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
if page['status'] == status:
|
||||||
|
print(f" {page['title']} ({rel})")
|
||||||
|
if page['tags']:
|
||||||
|
print(f" tags: {page['tags']}")
|
||||||
|
if status == 'disproven' and 'disproven' in graph:
|
||||||
|
print("\n -- Disproven Claims --")
|
||||||
|
for name, claim in graph['disproven'].items():
|
||||||
|
print(f" {name}:")
|
||||||
|
for k, v in claim.items():
|
||||||
|
print(f" {k}: {v}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_test(query):
|
||||||
|
tests = load_graph().get('tests', {})
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Test results for '{query}':\n")
|
||||||
|
for name, suite in tests.items():
|
||||||
|
if q in name.lower() or q in str(suite).lower():
|
||||||
|
print(f" -- {name} --")
|
||||||
|
if isinstance(suite, dict):
|
||||||
|
for k, v in suite.items():
|
||||||
|
if isinstance(v, dict):
|
||||||
|
print(f" {k}: {v.get('passing', '?')}/{v.get('total', '?')}")
|
||||||
|
elif k in ('count', 'passing', 'accuracy', 'file', 'date'):
|
||||||
|
print(f" {k}: {v}")
|
||||||
|
elif k == 'results' and isinstance(v, list):
|
||||||
|
for r in v:
|
||||||
|
mark = '✓' if r.get('result') == 'pass' else '✗'
|
||||||
|
print(f" {mark} {r.get('test', '?')}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_search(query):
|
||||||
|
flat = flatten_graph(load_graph())
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Search: '{query}'\n")
|
||||||
|
|
||||||
|
graph_hits = [(p, v) for p, v in flat if q in v.lower()]
|
||||||
|
if graph_hits:
|
||||||
|
print(f" -- Knowledge Graph ({len(graph_hits)} hits) --")
|
||||||
|
for p, v in graph_hits[:10]:
|
||||||
|
print(f" {p}: {v[:80]}")
|
||||||
|
|
||||||
|
page_hits = sorted(
|
||||||
|
[(page['content'].lower().count(q), rel, page['title'])
|
||||||
|
for rel, page in pages.items() if q in page['content'].lower()],
|
||||||
|
reverse=True)
|
||||||
|
if page_hits:
|
||||||
|
print(f"\n -- Wiki Pages ({len(page_hits)} pages) --")
|
||||||
|
for count, rel, title in page_hits:
|
||||||
|
print(f" {count:3d}x {title} ({rel})")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_related(page_name):
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = page_name.lower().replace('-', ' ').replace('_', ' ')
|
||||||
|
print(f"Related to: '{page_name}'\n")
|
||||||
|
|
||||||
|
print(" -- Links TO --")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
for link in page['links']:
|
||||||
|
if q in link.lower().replace('-', ' '):
|
||||||
|
print(f" <- {page['title']} ({rel})")
|
||||||
|
break
|
||||||
|
|
||||||
|
print("\n -- Links FROM --")
|
||||||
|
for rel, page in pages.items():
|
||||||
|
if q in page['title'].lower().replace('-', ' '):
|
||||||
|
for link in page['links']:
|
||||||
|
print(f" -> [[{link}]]")
|
||||||
|
break
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_timeline():
|
||||||
|
for entry in load_graph().get('timeline', []):
|
||||||
|
print(f" [{entry.get('date')}] {entry.get('commit', '?')}: {entry.get('desc', '?')}")
|
||||||
|
|
||||||
|
|
||||||
|
COMMANDS = {
|
||||||
|
'health': cmd_health, 'entity': cmd_entity, 'metric': cmd_metric,
|
||||||
|
'status': cmd_status, 'test': cmd_test, 'search': cmd_search,
|
||||||
|
'related': cmd_related, 'timeline': cmd_timeline,
|
||||||
|
}
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) < 2 or sys.argv[1] not in COMMANDS:
|
||||||
|
print(f"Usage: query.py <{'|'.join(COMMANDS)}> [args]")
|
||||||
|
sys.exit(1)
|
||||||
|
cmd = sys.argv[1]
|
||||||
|
args = sys.argv[2:]
|
||||||
|
if cmd in ('timeline', 'health'):
|
||||||
|
COMMANDS[cmd]()
|
||||||
|
elif args:
|
||||||
|
COMMANDS[cmd](' '.join(args))
|
||||||
|
else:
|
||||||
|
print(f"Usage: query.py {cmd} <query>")
|
||||||
18
wiki/tools/search.sh
Normal file
18
wiki/tools/search.sh
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Search the wiki — usable by both humans and LLM agents
|
||||||
|
# Usage: ./wiki/tools/search.sh "query" [--files-only]
|
||||||
|
|
||||||
|
WIKI_DIR="$(dirname "$(dirname "$(readlink -f "$0")")")"
|
||||||
|
QUERY="$1"
|
||||||
|
MODE="${2:---content}"
|
||||||
|
|
||||||
|
if [ -z "$QUERY" ]; then
|
||||||
|
echo "Usage: $0 <query> [--files-only]"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$MODE" = "--files-only" ]; then
|
||||||
|
grep -rl --include="*.md" --include="*.yaml" "$QUERY" "$WIKI_DIR" 2>/dev/null | sort
|
||||||
|
else
|
||||||
|
grep -rn --include="*.md" --include="*.yaml" --color=auto -i "$QUERY" "$WIKI_DIR" 2>/dev/null
|
||||||
|
fi
|
||||||
Reference in New Issue
Block a user