docs: consolidate scattered documentation into wiki
Some checks failed
CI / lint-and-test (push) Has been cancelled
Some checks failed
CI / lint-and-test (push) Has been cancelled
Move architecture/, docs/ content into wiki/ for a single source of truth: - architecture/deployment-blueprint.md → wiki/architecture/ - architecture/stack-architecture-review.md → wiki/architecture/ - architecture/wiki-platform-overview.md → wiki/architecture/ - docs/ARCHITECTURE.md → wiki/architecture/node-architecture.md - docs/API_REFERENCE.md → wiki/concepts/generalfunctions-api.md - docs/ISSUES.md → wiki/findings/open-issues-2026-03.md Remove stale files: - FUNCTIONAL_ISSUES_BACKLOG.md (was just a redirect pointer) - temp/ (stale cloud env examples) Fix README.md gitea URL (centraal.wbd-rd.nl → wbd-rd.nl). Update wiki index with all consolidated pages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
znetsixe
278
wiki/architecture/deployment-blueprint.md
Normal file
278
wiki/architecture/deployment-blueprint.md
Normal file
@@ -0,0 +1,278 @@
|
||||
---
|
||||
title: EVOLV Deployment Blueprint
|
||||
created: 2026-03-01
|
||||
updated: 2026-04-07
|
||||
status: evolving
|
||||
tags: [deployment, docker, edge, site, central]
|
||||
---
|
||||
|
||||
# EVOLV Deployment Blueprint
|
||||
|
||||
## Purpose
|
||||
|
||||
This document turns the current EVOLV architecture into a concrete deployment model.
|
||||
|
||||
It focuses on:
|
||||
|
||||
- target infrastructure layout
|
||||
- container/service topology
|
||||
- environment and secret boundaries
|
||||
- rollout order from edge to site to central
|
||||
|
||||
It is the local source document behind the wiki deployment pages.
|
||||
|
||||
## 1. Deployment Principles
|
||||
|
||||
- edge-first operation: plant logic must continue when central is unavailable
|
||||
- site mediation: site services protect field systems and absorb plant-specific complexity
|
||||
- central governance: external APIs, analytics, IAM, CI/CD, and shared dashboards terminate centrally
|
||||
- layered telemetry: InfluxDB exists where operationally justified at edge, site, and central
|
||||
- configuration authority: `tagcodering` should become the source of truth for configuration
|
||||
- secrets hygiene: tracked manifests contain variables only; secrets live in server-side env or secret stores
|
||||
|
||||
## 2. Layered Deployment Model
|
||||
|
||||
### 2.1 Edge node
|
||||
|
||||
Purpose:
|
||||
|
||||
- interface with PLCs and field assets
|
||||
- execute local Node-RED logic
|
||||
- retain local telemetry for resilience and digital-twin use cases
|
||||
|
||||
Recommended services:
|
||||
|
||||
- `evolv-edge-nodered`
|
||||
- `evolv-edge-influxdb`
|
||||
- optional `evolv-edge-grafana`
|
||||
- optional `evolv-edge-broker`
|
||||
|
||||
Should not host:
|
||||
|
||||
- public API ingress
|
||||
- central IAM
|
||||
- source control or CI/CD
|
||||
|
||||
### 2.2 Site node
|
||||
|
||||
Purpose:
|
||||
|
||||
- aggregate one or more edge nodes
|
||||
- host plant-local dashboards and engineering visibility
|
||||
- mediate traffic between edge and central
|
||||
|
||||
Recommended services:
|
||||
|
||||
- `evolv-site-nodered` or `coresync-site`
|
||||
- `evolv-site-influxdb`
|
||||
- `evolv-site-grafana`
|
||||
- optional `evolv-site-broker`
|
||||
|
||||
### 2.3 Central platform
|
||||
|
||||
Purpose:
|
||||
|
||||
- fleet-wide analytics
|
||||
- API and integration ingress
|
||||
- engineering lifecycle and releases
|
||||
- identity and governance
|
||||
|
||||
Recommended services:
|
||||
|
||||
- reverse proxy / ingress
|
||||
- API gateway
|
||||
- IAM
|
||||
- central InfluxDB
|
||||
- central Grafana
|
||||
- Gitea
|
||||
- CI/CD runner/controller
|
||||
- optional broker for asynchronous site/central workflows
|
||||
- configuration services over `tagcodering`
|
||||
|
||||
## 3. Target Container Topology
|
||||
|
||||
### 3.1 Edge host
|
||||
|
||||
Minimum viable edge stack:
|
||||
|
||||
```text
|
||||
edge-host-01
|
||||
- Node-RED
|
||||
- InfluxDB
|
||||
- optional Grafana
|
||||
```
|
||||
|
||||
Preferred production edge stack:
|
||||
|
||||
```text
|
||||
edge-host-01
|
||||
- Node-RED
|
||||
- InfluxDB
|
||||
- local health/export service
|
||||
- optional local broker
|
||||
- optional local dashboard service
|
||||
```
|
||||
|
||||
### 3.2 Site host
|
||||
|
||||
Minimum viable site stack:
|
||||
|
||||
```text
|
||||
site-host-01
|
||||
- Site Node-RED / CoreSync
|
||||
- Site InfluxDB
|
||||
- Site Grafana
|
||||
```
|
||||
|
||||
Preferred production site stack:
|
||||
|
||||
```text
|
||||
site-host-01
|
||||
- Site Node-RED / CoreSync
|
||||
- Site InfluxDB
|
||||
- Site Grafana
|
||||
- API relay / sync service
|
||||
- optional site broker
|
||||
```
|
||||
|
||||
### 3.3 Central host group
|
||||
|
||||
Central should not be one giant undifferentiated host forever. It should trend toward at least these responsibility groups:
|
||||
|
||||
```text
|
||||
central-ingress
|
||||
- reverse proxy
|
||||
- API gateway
|
||||
- IAM
|
||||
|
||||
central-observability
|
||||
- central InfluxDB
|
||||
- Grafana
|
||||
|
||||
central-engineering
|
||||
- Gitea
|
||||
- CI/CD
|
||||
- deployment orchestration
|
||||
|
||||
central-config
|
||||
- tagcodering-backed config services
|
||||
```
|
||||
|
||||
For early rollout these may be colocated, but the responsibility split should remain clear.
|
||||
|
||||
## 4. Compose Strategy
|
||||
|
||||
The current repository shows:
|
||||
|
||||
- `docker-compose.yml` as a development stack
|
||||
- `temp/cloud.yml` as a broad central-stack example
|
||||
|
||||
For production, EVOLV should not rely on one flat compose file for every layer.
|
||||
|
||||
Recommended split:
|
||||
|
||||
- `compose.edge.yml`
|
||||
- `compose.site.yml`
|
||||
- `compose.central.yml`
|
||||
- optional overlay files for site-specific differences
|
||||
|
||||
Benefits:
|
||||
|
||||
- clearer ownership per layer
|
||||
- smaller blast radius during updates
|
||||
- easier secret and env separation
|
||||
- easier rollout per site
|
||||
|
||||
## 5. Environment And Secrets Strategy
|
||||
|
||||
### 5.1 Current baseline
|
||||
|
||||
`temp/cloud.yml` now uses environment variables instead of inline credentials. That is the minimum acceptable baseline.
|
||||
|
||||
### 5.2 Recommended production rule
|
||||
|
||||
- tracked compose files contain `${VARIABLE}` placeholders only
|
||||
- real secrets live in server-local `.env` files or a managed secret store
|
||||
- no shared default production passwords in git
|
||||
- separate env files per layer and per environment
|
||||
|
||||
Suggested structure:
|
||||
|
||||
```text
|
||||
/opt/evolv/
|
||||
compose.edge.yml
|
||||
compose.site.yml
|
||||
compose.central.yml
|
||||
env/
|
||||
edge.env
|
||||
site.env
|
||||
central.env
|
||||
```
|
||||
|
||||
## 6. Recommended Network Flow
|
||||
|
||||
### 6.1 Northbound
|
||||
|
||||
- edge publishes or syncs upward to site
|
||||
- site aggregates and forwards selected data to central
|
||||
- central exposes APIs and dashboards to approved consumers
|
||||
|
||||
### 6.2 Southbound
|
||||
|
||||
- central issues advice, approved config, or mediated requests
|
||||
- site validates and relays to edge where appropriate
|
||||
- edge remains the execution point near PLCs
|
||||
|
||||
### 6.3 Forbidden direct path
|
||||
|
||||
- enterprise or internet clients should not directly query PLC-connected edge runtimes
|
||||
|
||||
## 7. Rollout Order
|
||||
|
||||
### Phase 1: Edge baseline
|
||||
|
||||
- deploy edge Node-RED
|
||||
- deploy local InfluxDB
|
||||
- validate PLC connectivity
|
||||
- validate local telemetry and resilience
|
||||
|
||||
### Phase 2: Site mediation
|
||||
|
||||
- deploy site Node-RED / CoreSync
|
||||
- connect one or more edge nodes
|
||||
- validate site-local dashboards and outage behavior
|
||||
|
||||
### Phase 3: Central services
|
||||
|
||||
- deploy ingress, IAM, API, Grafana, central InfluxDB
|
||||
- deploy Gitea and CI/CD services
|
||||
- validate controlled northbound access
|
||||
|
||||
### Phase 4: Configuration backbone
|
||||
|
||||
- connect runtime layers to `tagcodering`
|
||||
- reduce config duplication in flows
|
||||
- formalize config promotion and rollback
|
||||
|
||||
### Phase 5: Smart telemetry policy
|
||||
|
||||
- classify signals
|
||||
- define reconstruction rules
|
||||
- define authoritative layer per horizon
|
||||
- validate analytics and auditability
|
||||
|
||||
## 8. Immediate Technical Recommendations
|
||||
|
||||
- treat `docker/settings.js` as development-only and create hardened production settings separately
|
||||
- split deployment manifests by layer
|
||||
- define env files per layer and environment
|
||||
- formalize healthchecks and backup procedures for every persistent service
|
||||
- define whether broker usage is required at edge, site, central, or only selectively
|
||||
|
||||
## 9. Next Technical Work Items
|
||||
|
||||
1. create draft `compose.edge.yml`, `compose.site.yml`, and `compose.central.yml`
|
||||
2. define server directory layout and env-file conventions
|
||||
3. define production Node-RED settings profile
|
||||
4. define site-to-central sync path
|
||||
5. define deployment and rollback runbook
|
||||
426
wiki/architecture/node-architecture.md
Normal file
426
wiki/architecture/node-architecture.md
Normal file
@@ -0,0 +1,426 @@
|
||||
---
|
||||
title: EVOLV Architecture
|
||||
created: 2026-03-01
|
||||
updated: 2026-04-07
|
||||
status: evolving
|
||||
tags: [architecture, node-red, three-layer]
|
||||
---
|
||||
|
||||
# EVOLV Architecture
|
||||
|
||||
## 1. System Overview
|
||||
|
||||
High-level view of how EVOLV fits into the wastewater treatment automation stack.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
NR[Node-RED Runtime] <-->|msg objects| EVOLV[EVOLV Nodes]
|
||||
EVOLV -->|InfluxDB line protocol| INFLUX[(InfluxDB)]
|
||||
INFLUX -->|queries| GRAFANA[Grafana Dashboards]
|
||||
EVOLV -->|process output| NR
|
||||
EVOLV -->|parent output| NR
|
||||
|
||||
style NR fill:#b22222,color:#fff
|
||||
style EVOLV fill:#0f52a5,color:#fff
|
||||
style INFLUX fill:#0c99d9,color:#fff
|
||||
style GRAFANA fill:#50a8d9,color:#fff
|
||||
```
|
||||
|
||||
Each EVOLV node produces three outputs:
|
||||
| Port | Name | Purpose |
|
||||
|------|------|---------|
|
||||
| 0 | process | Process data forwarded to downstream nodes |
|
||||
| 1 | dbase | InfluxDB-formatted measurement data |
|
||||
| 2 | parent | Control messages to parent nodes (e.g. registerChild) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Node Architecture (Three-Layer Pattern)
|
||||
|
||||
Every node follows a consistent three-layer design that separates Node-RED wiring from domain logic.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Node-RED Runtime"
|
||||
REG["RED.nodes.registerType()"]
|
||||
end
|
||||
|
||||
subgraph "Layer 1 — Wrapper (valve.js)"
|
||||
W[wrapper .js]
|
||||
W -->|"new nodeClass(config, RED, this, name)"| NC
|
||||
W -->|MenuManager| MENU[HTTP /name/menu.js]
|
||||
W -->|configManager| CFG[HTTP /name/configData.js]
|
||||
end
|
||||
|
||||
subgraph "Layer 2 — Node Adapter (src/nodeClass.js)"
|
||||
NC[nodeClass]
|
||||
NC -->|_loadConfig| CFGM[configManager]
|
||||
NC -->|_setupSpecificClass| SC
|
||||
NC -->|_attachInputHandler| INPUT[onInput routing]
|
||||
NC -->|_startTickLoop| TICK[1s tick loop]
|
||||
NC -->|_tick → outputUtils| OUT[formatMsg]
|
||||
end
|
||||
|
||||
subgraph "Layer 3 — Domain Logic (src/specificClass.js)"
|
||||
SC[specificClass]
|
||||
SC -->|measurements| MC[MeasurementContainer]
|
||||
SC -->|state machine| ST[state]
|
||||
SC -->|hydraulics / biology| DOMAIN[domain models]
|
||||
end
|
||||
|
||||
subgraph "generalFunctions"
|
||||
GF[shared library]
|
||||
end
|
||||
|
||||
REG --> W
|
||||
GF -.->|logger, outputUtils, configManager,\nMeasurementContainer, validation, ...| NC
|
||||
GF -.->|MeasurementContainer, state,\nconvert, predict, ...| SC
|
||||
|
||||
style W fill:#0f52a5,color:#fff
|
||||
style NC fill:#0c99d9,color:#fff
|
||||
style SC fill:#50a8d9,color:#fff
|
||||
style GF fill:#86bbdd,color:#000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. generalFunctions Module Map
|
||||
|
||||
The shared library (`nodes/generalFunctions/`) provides all cross-cutting concerns.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
GF[generalFunctions/index.js]
|
||||
|
||||
subgraph "Core Helpers (src/helper/)"
|
||||
LOGGER[logger]
|
||||
OUTPUT[outputUtils]
|
||||
CHILD[childRegistrationUtils]
|
||||
CFGUTIL[configUtils]
|
||||
ASSERT[assertionUtils]
|
||||
VALID[validationUtils]
|
||||
end
|
||||
|
||||
subgraph "Validators (src/helper/validators/)"
|
||||
TV[typeValidators]
|
||||
CV[collectionValidators]
|
||||
CURV[curveValidator]
|
||||
end
|
||||
|
||||
subgraph "Domain Modules (src/)"
|
||||
MC[MeasurementContainer]
|
||||
CFGMGR[configManager]
|
||||
MENUMGR[MenuManager]
|
||||
STATE[state]
|
||||
CONVERT[convert / Fysics]
|
||||
PREDICT[predict / interpolation]
|
||||
NRMSE[nrmse / errorMetrics]
|
||||
COOLPROP[coolprop]
|
||||
end
|
||||
|
||||
subgraph "Data (datasets/)"
|
||||
CURVES[assetData/curves]
|
||||
ASSETS[assetData/assetData.json]
|
||||
UNITS[unitData.json]
|
||||
end
|
||||
|
||||
subgraph "Constants (src/constants/)"
|
||||
POS[POSITIONS / POSITION_VALUES]
|
||||
end
|
||||
|
||||
GF --> LOGGER
|
||||
GF --> OUTPUT
|
||||
GF --> CHILD
|
||||
GF --> CFGUTIL
|
||||
GF --> ASSERT
|
||||
GF --> VALID
|
||||
VALID --> TV
|
||||
VALID --> CV
|
||||
VALID --> CURV
|
||||
GF --> MC
|
||||
GF --> CFGMGR
|
||||
GF --> MENUMGR
|
||||
GF --> STATE
|
||||
GF --> CONVERT
|
||||
GF --> PREDICT
|
||||
GF --> NRMSE
|
||||
GF --> COOLPROP
|
||||
GF --> CURVES
|
||||
GF --> POS
|
||||
|
||||
style GF fill:#0f52a5,color:#fff
|
||||
style LOGGER fill:#86bbdd,color:#000
|
||||
style OUTPUT fill:#86bbdd,color:#000
|
||||
style VALID fill:#86bbdd,color:#000
|
||||
style MC fill:#50a8d9,color:#fff
|
||||
style CFGMGR fill:#50a8d9,color:#fff
|
||||
style MENUMGR fill:#50a8d9,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Data Flow (Message Lifecycle)
|
||||
|
||||
Sequence diagram showing a typical input message and the periodic tick output cycle.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant NR as Node-RED
|
||||
participant W as wrapper.js
|
||||
participant NC as nodeClass
|
||||
participant SC as specificClass
|
||||
participant OU as outputUtils
|
||||
|
||||
Note over W: Node startup
|
||||
W->>NC: new nodeClass(config, RED, node, name)
|
||||
NC->>NC: _loadConfig (configManager.buildConfig)
|
||||
NC->>SC: new specificClass(config, stateConfig, options)
|
||||
NC->>NR: send([null, null, {topic: registerChild}])
|
||||
|
||||
Note over NC: Every 1 second (tick loop)
|
||||
NC->>SC: getOutput()
|
||||
SC-->>NC: raw measurement data
|
||||
NC->>OU: formatMsg(raw, config, 'process')
|
||||
NC->>OU: formatMsg(raw, config, 'influxdb')
|
||||
NC->>NR: send([processMsg, influxMsg])
|
||||
|
||||
Note over NR: Incoming control message
|
||||
NR->>W: msg {topic: 'execMovement', payload: {...}}
|
||||
W->>NC: onInput(msg)
|
||||
NC->>SC: handleInput(source, action, setpoint)
|
||||
SC->>SC: update state machine & measurements
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Node Types
|
||||
|
||||
| Node | S88 Level | Purpose |
|
||||
|------|-----------|---------|
|
||||
| **measurement** | Control Module | Generic measurement point — reads, validates, and stores sensor values |
|
||||
| **valve** | Control Module | Valve simulation with hydraulic model, position control, flow/pressure prediction |
|
||||
| **rotatingMachine** | Control Module | Pumps, blowers, mixers — rotating equipment with speed control and efficiency curves |
|
||||
| **diffuser** | Control Module | Aeration diffuser — models oxygen transfer and pressure drop |
|
||||
| **settler** | Equipment | Sludge settler — models settling behavior and sludge blanket |
|
||||
| **reactor** | Equipment | Hydraulic tank and biological process simulator (activated sludge, digestion) |
|
||||
| **monster** | Equipment | MONitoring and STrEam Routing — complex measurement aggregation |
|
||||
| **pumpingStation** | Unit | Coordinates multiple pumps as a pumping station |
|
||||
| **valveGroupControl** | Unit | Manages multiple valves as a coordinated group — distributes flow, monitors pressure |
|
||||
| **machineGroupControl** | Unit | Group control for rotating machines — load balancing and sequencing |
|
||||
| **dashboardAPI** | Utility | Exposes data and unit conversion endpoints for external dashboards |
|
||||
# EVOLV Architecture
|
||||
|
||||
## Node Hierarchy (S88)
|
||||
|
||||
EVOLV follows the ISA-88 (S88) batch control standard. Each node maps to an S88 level and uses a consistent color scheme in the Node-RED editor.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef area fill:#0f52a5,color:#fff,stroke:#0a3d7a
|
||||
classDef processCell fill:#0c99d9,color:#fff,stroke:#0977aa
|
||||
classDef unit fill:#50a8d9,color:#fff,stroke:#3d89b3
|
||||
classDef equipment fill:#86bbdd,color:#000,stroke:#6a9bb8
|
||||
classDef controlModule fill:#a9daee,color:#000,stroke:#87b8cc
|
||||
classDef standalone fill:#f0f0f0,color:#000,stroke:#999
|
||||
|
||||
%% S88 Levels
|
||||
subgraph "S88: Area"
|
||||
PS[pumpingStation]
|
||||
end
|
||||
|
||||
subgraph "S88: Equipment"
|
||||
MGC[machineGroupControl]
|
||||
VGC[valveGroupControl]
|
||||
end
|
||||
|
||||
subgraph "S88: Control Module"
|
||||
RM[rotatingMachine]
|
||||
V[valve]
|
||||
M[measurement]
|
||||
R[reactor]
|
||||
S[settler]
|
||||
end
|
||||
|
||||
subgraph "Standalone"
|
||||
MON[monster]
|
||||
DASH[dashboardAPI]
|
||||
DIFF[diffuser - not implemented]
|
||||
end
|
||||
|
||||
%% Parent-child registration relationships
|
||||
PS -->|"accepts: measurement"| M
|
||||
PS -->|"accepts: machine"| RM
|
||||
PS -->|"accepts: machineGroup"| MGC
|
||||
PS -->|"accepts: pumpingStation"| PS2[pumpingStation]
|
||||
|
||||
MGC -->|"accepts: machine"| RM
|
||||
|
||||
RM -->|"accepts: measurement"| M2[measurement]
|
||||
RM -->|"accepts: reactor"| R
|
||||
|
||||
VGC -->|"accepts: valve"| V
|
||||
VGC -->|"accepts: machine / rotatingmachine"| RM2[rotatingMachine]
|
||||
VGC -->|"accepts: machinegroup / machinegroupcontrol"| MGC2[machineGroupControl]
|
||||
VGC -->|"accepts: pumpingstation / valvegroupcontrol"| PS3["pumpingStation / valveGroupControl"]
|
||||
|
||||
R -->|"accepts: measurement"| M3[measurement]
|
||||
R -->|"accepts: reactor"| R2[reactor]
|
||||
|
||||
S -->|"accepts: measurement"| M4[measurement]
|
||||
S -->|"accepts: reactor"| R3[reactor]
|
||||
S -->|"accepts: machine"| RM3[rotatingMachine]
|
||||
|
||||
%% Styling
|
||||
class PS,PS2,PS3 area
|
||||
class MGC,MGC2 equipment
|
||||
class VGC equipment
|
||||
class RM,RM2,RM3 controlModule
|
||||
class V controlModule
|
||||
class M,M2,M3,M4 controlModule
|
||||
class R,R2,R3 controlModule
|
||||
class S controlModule
|
||||
class MON,DASH,DIFF standalone
|
||||
```
|
||||
|
||||
### Registration Summary
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
classDef parent fill:#0c99d9,color:#fff
|
||||
classDef child fill:#a9daee,color:#000
|
||||
|
||||
PS[pumpingStation] -->|measurement| LEAF1((leaf))
|
||||
PS -->|machine| RM1[rotatingMachine]
|
||||
PS -->|machineGroup| MGC1[machineGroupControl]
|
||||
PS -->|pumpingStation| PS1[pumpingStation]
|
||||
|
||||
MGC[machineGroupControl] -->|machine| RM2[rotatingMachine]
|
||||
|
||||
VGC[valveGroupControl] -->|valve| V1[valve]
|
||||
VGC -->|source| SRC["machine, machinegroup,<br/>pumpingstation, valvegroupcontrol"]
|
||||
|
||||
RM[rotatingMachine] -->|measurement| LEAF2((leaf))
|
||||
RM -->|reactor| R1[reactor]
|
||||
|
||||
R[reactor] -->|measurement| LEAF3((leaf))
|
||||
R -->|reactor| R2[reactor]
|
||||
|
||||
S[settler] -->|measurement| LEAF4((leaf))
|
||||
S -->|reactor| R3[reactor]
|
||||
S -->|machine| RM3[rotatingMachine]
|
||||
|
||||
class PS,MGC,VGC,RM,R,S parent
|
||||
class LEAF1,LEAF2,LEAF3,LEAF4,RM1,RM2,RM3,MGC1,PS1,V1,SRC,R1,R2,R3 child
|
||||
```
|
||||
|
||||
## Node Types
|
||||
|
||||
| Node | S88 Level | softwareType | role | Accepts Children | Outputs |
|
||||
|------|-----------|-------------|------|-----------------|---------|
|
||||
| **pumpingStation** | Area | `pumpingstation` | StationController | measurement, machine (rotatingMachine), machineGroup, pumpingStation | [process, dbase, parent] |
|
||||
| **machineGroupControl** | Equipment | `machinegroupcontrol` | GroupController | machine (rotatingMachine) | [process, dbase, parent] |
|
||||
| **valveGroupControl** | Equipment | `valvegroupcontrol` | ValveGroupController | valve, machine, rotatingmachine, machinegroup, machinegroupcontrol, pumpingstation, valvegroupcontrol | [process, dbase, parent] |
|
||||
| **rotatingMachine** | Control Module | `rotatingmachine` | RotationalDeviceController | measurement, reactor | [process, dbase, parent] |
|
||||
| **valve** | Control Module | `valve` | controller | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||
| **measurement** | Control Module | `measurement` | Sensor | _(leaf node, no children)_ | [process, dbase, parent] |
|
||||
| **reactor** | Control Module | `reactor` | Biological reactor | measurement, reactor (upstream chaining) | [process, dbase, parent] |
|
||||
| **settler** | Control Module | `settler` | Secondary settler | measurement, reactor (upstream), machine (return pump) | [process, dbase, parent] |
|
||||
| **monster** | Standalone | - | - | dual-parent, standalone | - |
|
||||
| **dashboardAPI** | Standalone | - | - | accepts any child (Grafana integration) | - |
|
||||
| **diffuser** | Standalone | - | - | _(not implemented)_ | - |
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Measurement Data Flow (upstream to downstream)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Sensor as measurement (sensor)
|
||||
participant Machine as rotatingMachine
|
||||
participant Group as machineGroupControl
|
||||
participant Station as pumpingStation
|
||||
|
||||
Note over Sensor: Sensor reads value<br/>(pressure, flow, level, temp)
|
||||
|
||||
Sensor->>Sensor: measurements.type(t).variant("measured").position(p).value(v)
|
||||
Sensor->>Sensor: emitter.emit("type.measured.position", eventData)
|
||||
|
||||
Sensor->>Machine: Event: "pressure.measured.upstream"
|
||||
Machine->>Machine: Store in own MeasurementContainer
|
||||
Machine->>Machine: getMeasuredPressure() -> calcFlow() -> calcPower()
|
||||
Machine->>Machine: emitter.emit("flow.predicted.downstream", eventData)
|
||||
|
||||
Machine->>Group: Event: "flow.predicted.downstream"
|
||||
Group->>Group: handlePressureChange()
|
||||
Group->>Group: Aggregate flows across all machines
|
||||
Group->>Group: Calculate group totals and efficiency
|
||||
|
||||
Machine->>Station: Event: "flow.predicted.downstream"
|
||||
Station->>Station: Store predicted flow in/out
|
||||
Station->>Station: _updateVolumePrediction()
|
||||
Station->>Station: _calcNetFlow(), _calcTimeRemaining()
|
||||
```
|
||||
|
||||
### Control Command Flow (downstream to upstream)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Station as pumpingStation
|
||||
participant Group as machineGroupControl
|
||||
participant Machine as rotatingMachine
|
||||
participant Machine2 as rotatingMachine (2)
|
||||
|
||||
Station->>Group: handleInput("parent", action, param)
|
||||
|
||||
Group->>Group: Determine scaling strategy
|
||||
Group->>Group: Calculate setpoints per machine
|
||||
|
||||
Group->>Machine: handleInput("parent", "execMovement", setpoint)
|
||||
Group->>Machine2: handleInput("parent", "execMovement", setpoint)
|
||||
|
||||
Machine->>Machine: setpoint() -> state.moveTo(pos)
|
||||
Machine->>Machine: updatePosition() -> calcFlow(), calcPower()
|
||||
Machine->>Machine: emitter.emit("flow.predicted.downstream")
|
||||
|
||||
Machine2->>Machine2: setpoint() -> state.moveTo(pos)
|
||||
Machine2->>Machine2: updatePosition() -> calcFlow(), calcPower()
|
||||
Machine2->>Machine2: emitter.emit("flow.predicted.downstream")
|
||||
```
|
||||
|
||||
### Wastewater Treatment Process Flow
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
classDef process fill:#50a8d9,color:#fff
|
||||
classDef equipment fill:#86bbdd,color:#000
|
||||
|
||||
PS_IN[pumpingStation<br/>Influent] -->|flow| R1[reactor<br/>Anoxic]
|
||||
R1 -->|effluent| R2[reactor<br/>Aerated]
|
||||
R2 -->|effluent| SET[settler]
|
||||
SET -->|effluent out| PS_OUT[pumpingStation<br/>Effluent]
|
||||
SET -->|sludge return| RM_RET[rotatingMachine<br/>Return pump]
|
||||
RM_RET -->|recirculation| R1
|
||||
|
||||
PS_IN --- MGC_IN[machineGroupControl]
|
||||
MGC_IN --- RM_IN[rotatingMachine<br/>Influent pumps]
|
||||
|
||||
class PS_IN,PS_OUT process
|
||||
class R1,R2,SET process
|
||||
class MGC_IN,RM_IN,RM_RET equipment
|
||||
```
|
||||
|
||||
### Event-Driven Communication Pattern
|
||||
|
||||
All parent-child communication uses Node.js `EventEmitter`:
|
||||
|
||||
1. **Registration**: Parent calls `childRegistrationUtils.registerChild(child, position)` which stores the child and calls the parent's `registerChild(child, softwareType)` method.
|
||||
2. **Event binding**: The parent's `registerChild()` subscribes to the child's `measurements.emitter` events (e.g., `"flow.predicted.downstream"`).
|
||||
3. **Data propagation**: When a child updates a measurement, it emits an event. The parent's listener stores the value in its own `MeasurementContainer` and runs its domain logic.
|
||||
4. **Three outputs**: Every node sends data to three Node-RED outputs: `[process, dbase, parent]` -- process data for downstream nodes, InfluxDB for persistence, and parent aggregation data.
|
||||
|
||||
### Position Convention
|
||||
|
||||
Children register with a position relative to their parent:
|
||||
- `upstream` -- before the parent in the flow direction
|
||||
- `downstream` -- after the parent in the flow direction
|
||||
- `atEquipment` -- physically located at/on the parent equipment
|
||||
158
wiki/architecture/platform-overview.md
Normal file
158
wiki/architecture/platform-overview.md
Normal file
@@ -0,0 +1,158 @@
|
||||
---
|
||||
title: EVOLV Platform Architecture
|
||||
created: 2026-03-01
|
||||
updated: 2026-04-07
|
||||
status: evolving
|
||||
tags: [architecture, platform, edge-first]
|
||||
---
|
||||
|
||||
# EVOLV Platform Architecture
|
||||
|
||||
## At A Glance
|
||||
|
||||
EVOLV is not only a Node-RED package. It is a layered automation platform:
|
||||
|
||||
- edge for plant-side execution
|
||||
- site for local aggregation and resilience
|
||||
- central for coordination, analytics, APIs, and governance
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph EDGE["Edge"]
|
||||
PLC["PLC / IO"]
|
||||
ENR["Node-RED"]
|
||||
EDB["Local InfluxDB"]
|
||||
EUI["Local Monitoring"]
|
||||
end
|
||||
|
||||
subgraph SITE["Site"]
|
||||
SNR["CoreSync / Site Node-RED"]
|
||||
SDB["Site InfluxDB"]
|
||||
SUI["Site Dashboards"]
|
||||
end
|
||||
|
||||
subgraph CENTRAL["Central"]
|
||||
API["API Gateway"]
|
||||
CFG["Tagcodering"]
|
||||
CDB["Central InfluxDB"]
|
||||
CGR["Grafana"]
|
||||
INTEL["Overview Intelligence"]
|
||||
GIT["Gitea + CI/CD"]
|
||||
end
|
||||
|
||||
PLC --> ENR
|
||||
ENR --> EDB
|
||||
ENR --> EUI
|
||||
ENR <--> SNR
|
||||
EDB <--> SDB
|
||||
SNR --> SUI
|
||||
SNR <--> API
|
||||
API <--> CFG
|
||||
API --> INTEL
|
||||
SDB <--> CDB
|
||||
CDB --> CGR
|
||||
GIT --> ENR
|
||||
GIT --> SNR
|
||||
```
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Edge-first operation
|
||||
|
||||
The edge layer must remain useful and safe when central systems are down.
|
||||
|
||||
That means:
|
||||
|
||||
- local logic remains operational
|
||||
- local telemetry remains queryable
|
||||
- local dashboards can keep working
|
||||
|
||||
### 2. Multi-level telemetry
|
||||
|
||||
InfluxDB is expected on multiple levels:
|
||||
|
||||
- local for resilience and digital-twin use
|
||||
- site for plant diagnostics
|
||||
- central for fleet analytics and advisory logic
|
||||
|
||||
### 3. Smart storage
|
||||
|
||||
Telemetry should not be stored only with naive deadband rules.
|
||||
|
||||
The target model is signal-aware:
|
||||
|
||||
- preserve critical change points
|
||||
- reduce low-information flat sections
|
||||
- allow downstream reconstruction where justified
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
SIG["Process Signal"] --> EVAL["Slope / Event Evaluation"]
|
||||
EVAL --> KEEP["Keep critical points"]
|
||||
EVAL --> REDUCE["Reduce reconstructable points"]
|
||||
KEEP --> L0["Local InfluxDB"]
|
||||
REDUCE --> L0
|
||||
L0 --> L1["Site InfluxDB"]
|
||||
L1 --> L2["Central InfluxDB"]
|
||||
```
|
||||
|
||||
### 4. Central is the safe entry point
|
||||
|
||||
External systems should enter through central APIs, not by directly calling field-edge systems.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
EXT["External Request"] --> API["Central API Gateway"]
|
||||
API --> AUTH["Auth / Policy"]
|
||||
AUTH --> SITE["Site Layer"]
|
||||
SITE --> EDGE["Edge Layer"]
|
||||
EDGE --> PLC["Field Assets"]
|
||||
|
||||
EXT -. blocked .-> EDGE
|
||||
EXT -. blocked .-> PLC
|
||||
```
|
||||
|
||||
### 5. Configuration belongs in `tagcodering`
|
||||
|
||||
The intended configuration source of truth is the database-backed `tagcodering` model:
|
||||
|
||||
- machine metadata
|
||||
- asset configuration
|
||||
- runtime-consumable configuration
|
||||
- future central/site configuration services
|
||||
|
||||
This already exists partially but still needs more work before it fully serves that role.
|
||||
|
||||
## Layer Roles
|
||||
|
||||
### Edge
|
||||
|
||||
- PLC connectivity
|
||||
- local logic
|
||||
- protocol translation
|
||||
- local telemetry buffering
|
||||
- local monitoring and digital-twin support
|
||||
|
||||
### Site
|
||||
|
||||
- aggregation of edge systems
|
||||
- local dashboards and diagnostics
|
||||
- mediation between OT and central
|
||||
- protected handoff for central requests
|
||||
|
||||
### Central
|
||||
|
||||
- enterprise/API gateway
|
||||
- fleet dashboards
|
||||
- analytics and intelligence
|
||||
- source control and CI/CD
|
||||
- configuration governance through `tagcodering`
|
||||
|
||||
## Why This Matters
|
||||
|
||||
This architecture gives EVOLV:
|
||||
|
||||
- better resilience
|
||||
- safer external integration
|
||||
- better data quality for analytics
|
||||
- a path from Node-RED package to platform
|
||||
632
wiki/architecture/stack-review.md
Normal file
632
wiki/architecture/stack-review.md
Normal file
@@ -0,0 +1,632 @@
|
||||
---
|
||||
title: EVOLV Architecture Review
|
||||
created: 2026-03-01
|
||||
updated: 2026-04-07
|
||||
status: evolving
|
||||
tags: [architecture, stack, review]
|
||||
---
|
||||
|
||||
# EVOLV Architecture Review
|
||||
|
||||
## Purpose
|
||||
|
||||
This document captures:
|
||||
|
||||
- the architecture implemented in this repository today
|
||||
- the broader edge/site/central architecture shown in the drawings under `temp/`
|
||||
- the key strengths and weaknesses of that direction
|
||||
- the currently preferred target stack based on owner decisions from this review
|
||||
|
||||
It is the local staging document for a later wiki update.
|
||||
|
||||
## Evidence Used
|
||||
|
||||
Implemented stack evidence:
|
||||
|
||||
- `docker-compose.yml`
|
||||
- `docker/settings.js`
|
||||
- `docker/grafana/provisioning/datasources/influxdb.yaml`
|
||||
- `package.json`
|
||||
- `nodes/*`
|
||||
|
||||
Target-state evidence:
|
||||
|
||||
- `temp/fullStack.pdf`
|
||||
- `temp/edge.pdf`
|
||||
- `temp/CoreSync.drawio.pdf`
|
||||
- `temp/cloud.yml`
|
||||
|
||||
Owner decisions from this review:
|
||||
|
||||
- local InfluxDB is required for operational resilience
|
||||
- central acts as the advisory/intelligence and API-entry layer, not as a direct field caller
|
||||
- intended configuration authority is the database-backed `tagcodering` model
|
||||
- architecture wiki pages should be visual, not text-only
|
||||
|
||||
## 1. What Exists Today
|
||||
|
||||
### 1.1 Product/runtime layer
|
||||
|
||||
The codebase is currently a modular Node-RED package for wastewater/process automation:
|
||||
|
||||
- EVOLV ships custom Node-RED nodes for plant assets and process logic
|
||||
- nodes emit both process/control messages and telemetry-oriented outputs
|
||||
- shared helper logic lives in `nodes/generalFunctions/`
|
||||
- Grafana-facing integration exists through `dashboardAPI` and Influx-oriented outputs
|
||||
|
||||
### 1.2 Implemented development stack
|
||||
|
||||
The concrete development stack in this repository is:
|
||||
|
||||
- Node-RED
|
||||
- InfluxDB 2.x
|
||||
- Grafana
|
||||
|
||||
That gives a clear local flow:
|
||||
|
||||
1. EVOLV logic runs in Node-RED.
|
||||
2. Telemetry is emitted in a time-series-oriented shape.
|
||||
3. InfluxDB stores the telemetry.
|
||||
4. Grafana renders operational dashboards.
|
||||
|
||||
### 1.3 Existing runtime pattern in the nodes
|
||||
|
||||
A recurring EVOLV pattern is:
|
||||
|
||||
- output 0: process/control message
|
||||
- output 1: Influx/telemetry message
|
||||
- output 2: registration/control plumbing where relevant
|
||||
|
||||
So even in its current implemented form, EVOLV is not only a Node-RED project. It is already a control-plus-observability platform, with Node-RED as orchestration/runtime and InfluxDB/Grafana as telemetry and visualization services.
|
||||
|
||||
## 2. What The Drawings Describe
|
||||
|
||||
Across `temp/fullStack.pdf` and `temp/CoreSync.drawio.pdf`, the intended platform is broader and layered.
|
||||
|
||||
### 2.1 Edge / OT layer
|
||||
|
||||
The drawings consistently place these capabilities at the edge:
|
||||
|
||||
- PLC / OPC UA connectivity
|
||||
- Node-RED container as protocol translator and logic runtime
|
||||
- local broker in some variants
|
||||
- local InfluxDB / Prometheus style storage in some variants
|
||||
- local Grafana/SCADA in some variants
|
||||
|
||||
This is the plant-side operational layer.
|
||||
|
||||
### 2.2 Site / local server layer
|
||||
|
||||
The CoreSync drawings also show a site aggregation layer:
|
||||
|
||||
- RWZI-local server
|
||||
- Node-RED / CoreSync services
|
||||
- site-local broker
|
||||
- site-local database
|
||||
- upward API-based synchronization
|
||||
|
||||
This layer decouples field assets from central services and absorbs plant-specific complexity.
|
||||
|
||||
### 2.3 Central / cloud layer
|
||||
|
||||
The broader stack drawings and `temp/cloud.yml` show a central platform layer with:
|
||||
|
||||
- Gitea
|
||||
- Jenkins
|
||||
- reverse proxy / ingress
|
||||
- Grafana
|
||||
- InfluxDB
|
||||
- Node-RED
|
||||
- RabbitMQ / messaging
|
||||
- VPN / tunnel concepts
|
||||
- Keycloak in the drawing
|
||||
- Portainer in the drawing
|
||||
|
||||
This is a platform-services layer, not just an application runtime.
|
||||
|
||||
## 3. Architecture Decisions From This Review
|
||||
|
||||
These decisions now shape the preferred EVOLV target architecture.
|
||||
|
||||
### 3.1 Local telemetry is mandatory for resilience
|
||||
|
||||
Local InfluxDB is not optional. It is required so that:
|
||||
|
||||
- operations continue when central SCADA or central services are down
|
||||
- local dashboards and advanced digital-twin workflows can still consume recent and relevant process history
|
||||
- local edge/site layers can make smarter decisions without depending on round-trips to central
|
||||
|
||||
### 3.2 Multi-level InfluxDB is part of the architecture
|
||||
|
||||
InfluxDB should exist on multiple levels where it adds operational value:
|
||||
|
||||
- edge/local for resilience and near-real-time replay
|
||||
- site for plant-level history, diagnostics, and resilience
|
||||
- central for fleet-wide analytics, benchmarking, and advisory intelligence
|
||||
|
||||
This is not just copy-paste storage at each level. The design intent is event-driven and selective.
|
||||
|
||||
### 3.3 Storage should be smart, not only deadband-driven
|
||||
|
||||
The target is not simple "store every point" or only a fixed deadband rule such as 1%.
|
||||
|
||||
The desired storage approach is:
|
||||
|
||||
- observe signal slope and change behavior
|
||||
- preserve points where state is changing materially
|
||||
- store fewer points where the signal can be reconstructed downstream with sufficient fidelity
|
||||
- carry enough metadata or conventions so reconstruction quality is auditable
|
||||
|
||||
This implies EVOLV should evolve toward smart storage and signal-aware retention rather than naive event dumping.
|
||||
|
||||
### 3.4 Central is the intelligence and API-entry layer
|
||||
|
||||
Central may advise and coordinate edge/site layers, but external API requests should not hit field-edge systems directly.
|
||||
|
||||
The intended pattern is:
|
||||
|
||||
- external and enterprise integrations terminate centrally
|
||||
- central evaluates, aggregates, authorizes, and advises
|
||||
- site/edge layers receive mediated requests, policies, or setpoints
|
||||
- field-edge remains protected behind an intermediate layer
|
||||
|
||||
This aligns with the stated security direction.
|
||||
|
||||
### 3.5 Configuration source of truth should be database-backed
|
||||
|
||||
The intended configuration authority is the database-backed `tagcodering` model, which already exists but is not yet complete enough to serve as the fully realized source of truth.
|
||||
|
||||
That means the architecture should assume:
|
||||
|
||||
- asset and machine metadata belong in `tagcodering`
|
||||
- Node-RED flows should consume configuration rather than silently becoming the only configuration store
|
||||
- more work is still needed before this behaves as the intended central configuration backbone
|
||||
|
||||
## 4. Visual Model
|
||||
|
||||
### 4.1 Platform topology
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph OT["OT / Field"]
|
||||
PLC["PLC / IO"]
|
||||
DEV["Sensors / Machines"]
|
||||
end
|
||||
|
||||
subgraph EDGE["Edge Layer"]
|
||||
ENR["Edge Node-RED"]
|
||||
EDB["Local InfluxDB"]
|
||||
EUI["Local Grafana / Local Monitoring"]
|
||||
EBR["Optional Local Broker"]
|
||||
end
|
||||
|
||||
subgraph SITE["Site Layer"]
|
||||
SNR["Site Node-RED / CoreSync"]
|
||||
SDB["Site InfluxDB"]
|
||||
SUI["Site Grafana / SCADA Support"]
|
||||
SBR["Site Broker"]
|
||||
end
|
||||
|
||||
subgraph CENTRAL["Central Layer"]
|
||||
API["API / Integration Gateway"]
|
||||
INTEL["Overview Intelligence / Advisory Logic"]
|
||||
CDB["Central InfluxDB"]
|
||||
CGR["Central Grafana"]
|
||||
CFG["Tagcodering Config Model"]
|
||||
GIT["Gitea"]
|
||||
CI["CI/CD"]
|
||||
IAM["IAM / Keycloak"]
|
||||
end
|
||||
|
||||
DEV --> PLC
|
||||
PLC --> ENR
|
||||
ENR --> EDB
|
||||
ENR --> EUI
|
||||
ENR --> EBR
|
||||
ENR <--> SNR
|
||||
EDB <--> SDB
|
||||
SNR --> SDB
|
||||
SNR --> SUI
|
||||
SNR --> SBR
|
||||
SNR <--> API
|
||||
API --> INTEL
|
||||
API <--> CFG
|
||||
SDB <--> CDB
|
||||
INTEL --> SNR
|
||||
CGR --> CDB
|
||||
CI --> GIT
|
||||
IAM --> API
|
||||
IAM --> CGR
|
||||
```
|
||||
|
||||
### 4.2 Command and access boundary
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
EXT["External APIs / Enterprise Requests"] --> API["Central API Gateway"]
|
||||
API --> AUTH["AuthN/AuthZ / Policy Checks"]
|
||||
AUTH --> INTEL["Central Advisory / Decision Support"]
|
||||
INTEL --> SITE["Site Integration Layer"]
|
||||
SITE --> EDGE["Edge Runtime"]
|
||||
EDGE --> PLC["PLC / Field Assets"]
|
||||
|
||||
EXT -. no direct access .-> EDGE
|
||||
EXT -. no direct access .-> PLC
|
||||
```
|
||||
|
||||
### 4.3 Smart telemetry flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
RAW["Raw Signal"] --> EDGELOGIC["Edge Signal Evaluation"]
|
||||
EDGELOGIC --> KEEP["Keep Critical Change Points"]
|
||||
EDGELOGIC --> SKIP["Skip Reconstructable Flat Points"]
|
||||
EDGELOGIC --> LOCAL["Local InfluxDB"]
|
||||
LOCAL --> SITE["Site InfluxDB"]
|
||||
SITE --> CENTRAL["Central InfluxDB"]
|
||||
KEEP --> LOCAL
|
||||
SKIP -. reconstruction assumptions / metadata .-> SITE
|
||||
CENTRAL --> DASH["Fleet Dashboards / Analytics"]
|
||||
```
|
||||
|
||||
## 5. Upsides Of This Direction
|
||||
|
||||
### 5.1 Strong separation between control and observability
|
||||
|
||||
Node-RED for runtime/orchestration and InfluxDB/Grafana for telemetry is still the right structural split:
|
||||
|
||||
- control stays close to the process
|
||||
- telemetry storage/querying stays in time-series-native tooling
|
||||
- dashboards do not need to overload Node-RED itself
|
||||
|
||||
### 5.2 Edge-first matches operational reality
|
||||
|
||||
For wastewater/process systems, edge-first remains correct:
|
||||
|
||||
- lower latency
|
||||
- better degraded-mode behavior
|
||||
- less dependence on WAN or central platform uptime
|
||||
- clearer OT trust boundary
|
||||
|
||||
### 5.3 Site mediation improves safety and security
|
||||
|
||||
Using central as the enterprise/API entry point and site as the mediator improves posture:
|
||||
|
||||
- field systems are less exposed
|
||||
- policy decisions can be centralized
|
||||
- external integrations do not probe the edge directly
|
||||
- site can continue operating even when upstream is degraded
|
||||
|
||||
### 5.4 Multi-level storage enables better analytics
|
||||
|
||||
Multiple Influx layers can support:
|
||||
|
||||
- local resilience
|
||||
- site diagnostics
|
||||
- fleet benchmarking
|
||||
- smarter retention and reconstruction strategies
|
||||
|
||||
That is substantially more capable than a single central historian model.
|
||||
|
||||
### 5.5 `tagcodering` is the right long-term direction
|
||||
|
||||
A database-backed configuration authority is stronger than embedding configuration only in flows because it supports:
|
||||
|
||||
- machine metadata management
|
||||
- controlled rollout of configuration changes
|
||||
- clearer versioning and provenance
|
||||
- future API-driven configuration services
|
||||
|
||||
## 6. Downsides And Risks
|
||||
|
||||
### 6.1 Smart storage raises algorithmic and governance complexity
|
||||
|
||||
Signal-aware storage and reconstruction is promising, but it creates architectural obligations:
|
||||
|
||||
- reconstruction rules must be explicit
|
||||
- acceptable reconstruction error must be defined per signal type
|
||||
- operators must know whether they see raw or reconstructed history
|
||||
- compliance-relevant data may need stricter retention than operational convenience data
|
||||
|
||||
Without those rules, smart storage can become opaque and hard to trust.
|
||||
|
||||
### 6.2 Multi-level databases can create ownership confusion
|
||||
|
||||
If edge, site, and central all store telemetry, you must define:
|
||||
|
||||
- which layer is authoritative for which time horizon
|
||||
- when backfill is allowed
|
||||
- when data is summarized vs copied
|
||||
- how duplicates or gaps are detected
|
||||
|
||||
Otherwise operations will argue over which trend is "the real one."
|
||||
|
||||
### 6.3 Central intelligence must remain advisory-first
|
||||
|
||||
Central guidance can become valuable, but direct closed-loop dependency on central would be risky.
|
||||
|
||||
The architecture should therefore preserve:
|
||||
|
||||
- local control authority at edge/site
|
||||
- bounded and explicit central advice
|
||||
- safe behavior if central recommendations stop arriving
|
||||
|
||||
### 6.4 `tagcodering` is not yet complete enough to lean on blindly
|
||||
|
||||
It is the right target, but its current partial state means there is still architecture debt:
|
||||
|
||||
- incomplete config workflows
|
||||
- likely mismatch between desired and implemented schema behavior
|
||||
- temporary duplication between flows, node config, and database-held metadata
|
||||
|
||||
This should be treated as a core platform workstream, not a side issue.
|
||||
|
||||
### 6.5 Broker responsibilities are still not crisp enough
|
||||
|
||||
The materials still reference MQTT/AMQP/RabbitMQ/brokers without one stable responsibility split. That needs to be resolved before large-scale deployment.
|
||||
|
||||
Questions still open:
|
||||
|
||||
- command bus or event bus?
|
||||
- site-only or cross-site?
|
||||
- telemetry transport or only synchronization/eventing?
|
||||
- durability expectations and replay behavior?
|
||||
|
||||
## 7. Security And Regulatory Positioning
|
||||
|
||||
### 7.1 Purdue-style layering is a good fit
|
||||
|
||||
EVOLV's preferred structure aligns well with a Purdue-style OT/IT layering approach:
|
||||
|
||||
- PLCs and field assets stay at the operational edge
|
||||
- edge runtimes stay close to the process
|
||||
- site systems mediate between OT and broader enterprise concerns
|
||||
- central services host APIs, identity, analytics, and engineering workflows
|
||||
|
||||
That is important because it supports segmented trust boundaries instead of direct enterprise-to-field reach-through.
|
||||
|
||||
### 7.2 NIS2 alignment
|
||||
|
||||
Directive (EU) 2022/2555 (NIS2) requires cybersecurity risk-management measures, incident handling, and stronger governance for covered entities.
|
||||
|
||||
This architecture supports that by:
|
||||
|
||||
- limiting direct exposure of field systems
|
||||
- separating operational layers
|
||||
- enabling central policy and oversight
|
||||
- preserving local operation during upstream failure
|
||||
|
||||
### 7.3 CER alignment
|
||||
|
||||
Directive (EU) 2022/2557 (Critical Entities Resilience Directive) focuses on resilience of essential services.
|
||||
|
||||
The edge-plus-site approach supports that direction because:
|
||||
|
||||
- local/site layers can continue during central disruption
|
||||
- essential service continuity does not depend on one central runtime
|
||||
- degraded-mode behavior can be explicitly designed per layer
|
||||
|
||||
### 7.4 Cyber Resilience Act alignment
|
||||
|
||||
Regulation (EU) 2024/2847 (Cyber Resilience Act) creates cybersecurity requirements for products with digital elements.
|
||||
|
||||
For EVOLV, that means the platform should keep strengthening:
|
||||
|
||||
- secure configuration handling
|
||||
- vulnerability and update management
|
||||
- release traceability
|
||||
- lifecycle ownership of components and dependencies
|
||||
|
||||
### 7.5 GDPR alignment where personal data is present
|
||||
|
||||
Regulation (EU) 2016/679 (GDPR) applies whenever EVOLV processes personal data.
|
||||
|
||||
The architecture helps by:
|
||||
|
||||
- centralizing ingress
|
||||
- reducing unnecessary propagation of data to field layers
|
||||
- making access, retention, and audit boundaries easier to define
|
||||
|
||||
### 7.6 What can and cannot be claimed
|
||||
|
||||
The defensible claim is that EVOLV can be deployed in a way that supports compliance with strict European cybersecurity and resilience expectations.
|
||||
|
||||
The non-defensible claim is that EVOLV is automatically compliant purely because of the architecture diagram.
|
||||
|
||||
Actual compliance still depends on implementation and operations, including:
|
||||
|
||||
- access control
|
||||
- patch and vulnerability management
|
||||
- incident response
|
||||
- logging and audit evidence
|
||||
- retention policy
|
||||
- data classification
|
||||
|
||||
## 8. Recommended Ideal Stack
|
||||
|
||||
The ideal EVOLV stack should be layered around operational boundaries, not around tools.
|
||||
|
||||
### 7.1 Layer A: Edge execution
|
||||
|
||||
Purpose:
|
||||
|
||||
- connect to PLCs and field assets
|
||||
- execute time-sensitive local logic
|
||||
- preserve operation during WAN/central loss
|
||||
- provide local telemetry access for resilience and digital-twin use cases
|
||||
|
||||
Recommended components:
|
||||
|
||||
- Node-RED runtime for EVOLV edge flows
|
||||
- OPC UA and protocol adapters
|
||||
- local InfluxDB
|
||||
- optional local Grafana for local engineering/monitoring
|
||||
- optional local broker only when multiple participants need decoupling
|
||||
|
||||
Principle:
|
||||
|
||||
- edge remains safe and useful when disconnected
|
||||
|
||||
### 7.2 Layer B: Site integration
|
||||
|
||||
Purpose:
|
||||
|
||||
- aggregate multiple edge systems at plant/site level
|
||||
- host plant-local dashboards and diagnostics
|
||||
- mediate between raw OT detail and central standardization
|
||||
- serve as the protected step between field systems and central requests
|
||||
|
||||
Recommended components:
|
||||
|
||||
- site Node-RED / CoreSync services
|
||||
- site InfluxDB
|
||||
- site Grafana / SCADA-supporting dashboards
|
||||
- site broker where asynchronous eventing is justified
|
||||
|
||||
Principle:
|
||||
|
||||
- site absorbs plant complexity and protects field assets
|
||||
|
||||
### 7.3 Layer C: Central platform
|
||||
|
||||
Purpose:
|
||||
|
||||
- fleet-wide analytics
|
||||
- shared dashboards
|
||||
- engineering lifecycle
|
||||
- enterprise/API entry point
|
||||
- overview intelligence and advisory logic
|
||||
|
||||
Recommended components:
|
||||
|
||||
- Gitea
|
||||
- CI/CD
|
||||
- central InfluxDB
|
||||
- central Grafana
|
||||
- API/integration gateway
|
||||
- IAM
|
||||
- VPN/private connectivity
|
||||
- `tagcodering`-backed configuration services
|
||||
|
||||
Principle:
|
||||
|
||||
- central coordinates, advises, and governs; it is not the direct field caller
|
||||
|
||||
### 7.4 Cross-cutting platform services
|
||||
|
||||
These should be explicit architecture elements:
|
||||
|
||||
- secrets management
|
||||
- certificate management
|
||||
- backup/restore
|
||||
- audit logging
|
||||
- monitoring/alerting of the platform itself
|
||||
- versioned configuration and schema management
|
||||
- rollout/rollback strategy
|
||||
|
||||
## 9. Recommended Opinionated Choices
|
||||
|
||||
### 8.1 Keep Node-RED as the orchestration layer, not the whole platform
|
||||
|
||||
Node-RED should own:
|
||||
|
||||
- process orchestration
|
||||
- protocol mediation
|
||||
- edge/site logic
|
||||
- KPI production
|
||||
|
||||
It should not become the sole owner of:
|
||||
|
||||
- identity
|
||||
- long-term configuration authority
|
||||
- secret management
|
||||
- compliance/audit authority
|
||||
|
||||
### 8.2 Use InfluxDB by function and horizon
|
||||
|
||||
Recommended split:
|
||||
|
||||
- edge: resilience, local replay, digital-twin input
|
||||
- site: plant diagnostics and local continuity
|
||||
- central: fleet analytics, advisory intelligence, benchmarking, and long-term cross-site views
|
||||
|
||||
### 8.3 Prefer smart telemetry retention over naive point dumping
|
||||
|
||||
Recommended rule:
|
||||
|
||||
- keep information-rich points
|
||||
- reduce information-poor flat spans
|
||||
- document reconstruction assumptions
|
||||
- define signal-class-specific fidelity expectations
|
||||
|
||||
This needs design discipline, but it is a real differentiator if executed well.
|
||||
|
||||
### 8.4 Put enterprise/API ingress at central, not at edge
|
||||
|
||||
This should become a hard architectural rule:
|
||||
|
||||
- external requests land centrally
|
||||
- central authenticates and authorizes
|
||||
- central or site mediates downward
|
||||
- edge never becomes the exposed public integration surface
|
||||
|
||||
### 8.5 Make `tagcodering` the target configuration backbone
|
||||
|
||||
The architecture should be designed so that `tagcodering` can mature into:
|
||||
|
||||
- machine and asset registry
|
||||
- configuration source of truth
|
||||
- site/central configuration exchange point
|
||||
- API-served configuration source for runtime layers
|
||||
|
||||
## 10. Suggested Phasing
|
||||
|
||||
### Phase 1: Stabilize contracts
|
||||
|
||||
- define topic and payload contracts
|
||||
- define telemetry classes and reconstruction policy
|
||||
- define asset, machine, and site identity model
|
||||
- define `tagcodering` scope and schema ownership
|
||||
|
||||
### Phase 2: Harden local/site resilience
|
||||
|
||||
- formalize edge and site runtime patterns
|
||||
- define local telemetry retention and replay behavior
|
||||
- define central-loss behavior
|
||||
- define dashboard behavior during isolation
|
||||
|
||||
### Phase 3: Harden central platform
|
||||
|
||||
- IAM
|
||||
- API gateway
|
||||
- central observability
|
||||
- CI/CD
|
||||
- backup and disaster recovery
|
||||
- config services over `tagcodering`
|
||||
|
||||
### Phase 4: Introduce selective synchronization and intelligence
|
||||
|
||||
- event-driven telemetry propagation rules
|
||||
- smart-storage promotion/backfill policies
|
||||
- advisory services from central
|
||||
- auditability of downward recommendations and configuration changes
|
||||
|
||||
## 11. Immediate Open Questions Before Wiki Finalization
|
||||
|
||||
1. Which signals are allowed to use reconstruction-aware smart storage, and which must remain raw or near-raw for audit/compliance reasons?
|
||||
2. How should `tagcodering` be exposed to runtime layers: direct database access, a dedicated API, or both?
|
||||
3. What exact responsibility split should EVOLV use between API synchronization and broker-based eventing?
|
||||
|
||||
## 12. Recommended Wiki Structure
|
||||
|
||||
The wiki should not be one long page. It should be split into:
|
||||
|
||||
1. platform overview with the main topology diagram
|
||||
2. edge-site-central runtime model
|
||||
3. telemetry and smart storage model
|
||||
4. security and access-boundary model
|
||||
5. configuration architecture centered on `tagcodering`
|
||||
|
||||
## 13. Next Step
|
||||
|
||||
Use this document as the architecture baseline. The companion markdown page in `architecture/` can then be shaped into a wiki-ready visual overview page with Mermaid diagrams and shorter human-readable sections.
|
||||
Reference in New Issue
Block a user