Compare commits
4 Commits
6a6c04d34b
...
dev-rene
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1c4a3f9685 | ||
|
|
9ca32dddfb | ||
|
|
75458713be | ||
|
|
99aedf46c3 |
@@ -0,0 +1,36 @@
|
|||||||
|
# DECISION-20260323-architecture-layering-resilience-and-config-authority
|
||||||
|
|
||||||
|
## Context
|
||||||
|
- Task/request: refine the EVOLV architecture baseline using the current stack drawings and owner guidance.
|
||||||
|
- Impacted files/contracts: architecture documentation, future wiki structure, telemetry/storage strategy, security boundaries, and configuration authority assumptions.
|
||||||
|
- Why a decision is required now: the architecture can no longer stay at a generic "Node-RED plus cloud" level; several operating principles were clarified by the owner and need to be treated as architectural defaults.
|
||||||
|
|
||||||
|
## Options
|
||||||
|
1. Keep the architecture intentionally broad and tool-centric
|
||||||
|
- Benefits: fewer early commitments.
|
||||||
|
- Risks: blurred boundaries for resilience, data ownership, and security; easier to drift into contradictory implementations.
|
||||||
|
- Rollout notes: wiki remains descriptive but not decision-shaping.
|
||||||
|
|
||||||
|
2. Adopt explicit defaults for resilience, API boundary, telemetry layering, and configuration authority
|
||||||
|
- Benefits: clearer target operating model; easier to design stack services and wiki pages consistently; aligns diagrams with intended operational behavior.
|
||||||
|
- Risks: some assumptions may outpace current implementation and therefore create an architecture debt backlog.
|
||||||
|
- Rollout notes: document gaps clearly and treat incomplete systems as planned workstreams rather than pretending they already exist.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
- Selected option: Option 2.
|
||||||
|
- Decision owner: repository owner confirmed during architecture review.
|
||||||
|
- Date: 2026-03-23.
|
||||||
|
- Rationale: the owner clarified concrete architecture goals that materially affect security, resilience, and platform structure. The documentation should encode those as defaults instead of leaving them implicit.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
- Compatibility impact: low immediate code impact, but future implementations should align to these defaults.
|
||||||
|
- Safety/security impact: improved boundary clarity by making central the integration entry point and keeping edge protected behind site/central mediation.
|
||||||
|
- Data/operations impact: multi-level InfluxDB and smart-storage behavior become first-class design concerns; `tagcodering` becomes the intended configuration backbone.
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
- Required code/doc updates: update the architecture review doc, add visual wiki-ready diagrams, and track follow-up work for incomplete `tagcodering` integration and telemetry policy design.
|
||||||
|
- Validation evidence required: architecture docs reflect the agreed principles and diagrams; no contradiction with current repo evidence for implemented components.
|
||||||
|
|
||||||
|
## Rollback / Migration
|
||||||
|
- Rollback strategy: return to a generic descriptive architecture document without explicit defaults.
|
||||||
|
- Migration/deprecation plan: implement these principles incrementally, starting with configuration authority, telemetry policy, and site/central API boundaries.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# DECISION-20260323-compose-secrets-via-env
|
||||||
|
|
||||||
|
## Context
|
||||||
|
- Task/request: harden the target-state stack example so credentials are not stored directly in `temp/cloud.yml`.
|
||||||
|
- Impacted files/contracts: `temp/cloud.yml`, deployment/operations practice for target-state infrastructure examples.
|
||||||
|
- Why a decision is required now: the repository contained inline credentials in a tracked compose file, which conflicts with the intended security posture and creates avoidable secret-leak risk.
|
||||||
|
|
||||||
|
## Options
|
||||||
|
1. Keep credentials inline in the compose file
|
||||||
|
- Benefits: simplest to run as a standalone example.
|
||||||
|
- Risks: secrets leak into git history, reviews, copies, and local machines; encourages unsafe operational practice.
|
||||||
|
- Rollout notes: none, but the risk remains permanent once committed.
|
||||||
|
|
||||||
|
2. Move credentials to server-side environment variables and keep only placeholders in compose
|
||||||
|
- Benefits: aligns the manifest with a safer deployment pattern; keeps tracked config portable across environments; supports secret rotation without editing the compose file.
|
||||||
|
- Risks: operators must manage `.env` or equivalent secret injection correctly.
|
||||||
|
- Rollout notes: provide an example env file and document that the real `.env` stays on the server and out of version control.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
- Selected option: Option 2.
|
||||||
|
- Decision owner: repository owner confirmed during task discussion.
|
||||||
|
- Date: 2026-03-23.
|
||||||
|
- Rationale: the target architecture should model the right operational pattern. Inline secrets in repository-tracked compose files are not acceptable for EVOLV's intended OT/IT deployment posture.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
- Compatibility impact: low; operators now need to supply environment variables when deploying `temp/cloud.yml`.
|
||||||
|
- Safety/security impact: improved secret hygiene and lower credential exposure risk.
|
||||||
|
- Data/operations impact: deployment requires an accompanying `.env` on the server or explicit `--env-file` usage.
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
- Required code/doc updates: replace inline secrets in `temp/cloud.yml`; add `temp/cloud.env.example`; keep the real `.env` untracked on the server.
|
||||||
|
- Validation evidence required: inspect compose file for `${...}` placeholders and verify no real credentials remain in tracked files touched by this change.
|
||||||
|
|
||||||
|
## Rollback / Migration
|
||||||
|
- Rollback strategy: reintroduce inline values, though this is not recommended.
|
||||||
|
- Migration/deprecation plan: create a server-local `.env` from `temp/cloud.env.example`, fill in real values, and run compose from that environment.
|
||||||
123
.agents/improvements/EXAMPLE_FLOW_TEMPLATE.md
Normal file
123
.agents/improvements/EXAMPLE_FLOW_TEMPLATE.md
Normal file
@@ -0,0 +1,123 @@
|
|||||||
|
# EVOLV Example Flow Template Standard
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Every EVOLV node MUST have example flows in its `examples/` directory. Node-RED automatically discovers these and shows them in **Import > Examples > EVOLV**.
|
||||||
|
|
||||||
|
## Naming Convention
|
||||||
|
|
||||||
|
```
|
||||||
|
examples/
|
||||||
|
01 - Basic Manual Control.json # Tier 1: inject-based, zero deps
|
||||||
|
02 - Integration with Parent Node.json # Tier 2: parent-child wiring
|
||||||
|
03 - Dashboard Visualization.json # Tier 3: FlowFuse dashboard (optional)
|
||||||
|
```
|
||||||
|
|
||||||
|
The filename (minus `.json`) becomes the menu label in Node-RED.
|
||||||
|
|
||||||
|
## Tier 1: Basic (inject-based, zero external dependencies)
|
||||||
|
|
||||||
|
**Purpose:** Demonstrate all key functionality using only core Node-RED nodes.
|
||||||
|
|
||||||
|
**Required elements:**
|
||||||
|
- 1x `comment` node (top-left): title + 2-3 line description of what the flow demonstrates
|
||||||
|
- 1x `comment` node (near inputs): "HOW TO USE: 1. Deploy flow. 2. Click inject nodes..."
|
||||||
|
- `inject` nodes for each control action (labeled clearly)
|
||||||
|
- The EVOLV node under test with **realistic, working configuration**
|
||||||
|
- 3x `debug` nodes: "Port 0: Process", "Port 1: InfluxDB", "Port 2: Parent"
|
||||||
|
- Optional: 1x `function` node to format output readably (keep under 20 lines)
|
||||||
|
|
||||||
|
**Forbidden:** No dashboard nodes. No FlowFuse widgets. No HTTP nodes. No third-party nodes.
|
||||||
|
|
||||||
|
**Config rules:**
|
||||||
|
- All required config fields filled with realistic values
|
||||||
|
- Model/curve fields set to existing models in the library
|
||||||
|
- `enableLog: true, logLevel: "info"` so users can see what happens
|
||||||
|
- Unit fields explicitly set (not empty strings)
|
||||||
|
|
||||||
|
**Layout rules:**
|
||||||
|
- Comment nodes: top-left
|
||||||
|
- Input section: left side (x: 100-400)
|
||||||
|
- EVOLV node: center (x: 500-600)
|
||||||
|
- Debug/output: right side (x: 700-900)
|
||||||
|
- Y spacing: ~60px between nodes
|
||||||
|
|
||||||
|
## Tier 2: Integration (parent-child relationships)
|
||||||
|
|
||||||
|
**Purpose:** Show how nodes connect as parent-child via Port 2.
|
||||||
|
|
||||||
|
**Required elements:**
|
||||||
|
- 1x `comment` node: what relationship is being demonstrated
|
||||||
|
- Parent node + child node(s) properly wired
|
||||||
|
- Port 2 of child → Port 0 input of parent (registration pathway)
|
||||||
|
- `inject` nodes to send control commands to parent
|
||||||
|
- `inject` nodes to send measurement/state to children
|
||||||
|
- `debug` nodes on all ports of both parent and children
|
||||||
|
|
||||||
|
**Node-specific integration patterns:**
|
||||||
|
- `machineGroupControl` → 2x `rotatingMachine`
|
||||||
|
- `pumpingStation` → 1x `rotatingMachine` + 1x `measurement` (assetType: "flow")
|
||||||
|
- `valveGroupControl` → 2x `valve`
|
||||||
|
- `reactor` → `settler` (downstream cascade)
|
||||||
|
- `measurement` → any parent node
|
||||||
|
|
||||||
|
## Tier 3: Dashboard Visualization (optional)
|
||||||
|
|
||||||
|
**Purpose:** Rich interactive demo with FlowFuse dashboard.
|
||||||
|
|
||||||
|
**Allowed additional dependencies:** FlowFuse dashboard nodes only (`@flowfuse/node-red-dashboard`).
|
||||||
|
|
||||||
|
**Required elements:**
|
||||||
|
- 1x `comment` node: "Requires @flowfuse/node-red-dashboard"
|
||||||
|
- Auto-initialization: `inject` node with "Inject once after 1 second" for default mode/state
|
||||||
|
- Dashboard controls clearly labeled
|
||||||
|
- Charts with proper axis labels and units
|
||||||
|
- Keep parser/formatter functions under 40 lines (split if needed)
|
||||||
|
- No null message outputs (filter before sending to charts)
|
||||||
|
|
||||||
|
## Comment Node Standard
|
||||||
|
|
||||||
|
Every comment node must use this format:
|
||||||
|
|
||||||
|
```
|
||||||
|
Title: [Node Name] - [Flow Tier]
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
[2-3 line description]
|
||||||
|
|
||||||
|
Prerequisites: [list any requirements]
|
||||||
|
```
|
||||||
|
|
||||||
|
## ID Naming Convention
|
||||||
|
|
||||||
|
Use predictable, readable IDs for all nodes (not random hex):
|
||||||
|
|
||||||
|
```
|
||||||
|
{nodeName}_{tier}_{purpose}
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- rm_basic_tab (rotatingMachine, basic flow, tab)
|
||||||
|
- rm_basic_node (the actual rotatingMachine node)
|
||||||
|
- rm_basic_debug_port0 (debug on port 0)
|
||||||
|
- rm_basic_inject_start (inject for startup)
|
||||||
|
- rm_basic_comment_title (title comment)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validation Checklist
|
||||||
|
|
||||||
|
Before committing an example flow:
|
||||||
|
|
||||||
|
- [ ] Can be imported into clean Node-RED + EVOLV (no other packages needed for Tier 1/2)
|
||||||
|
- [ ] All nodes show correct status after deploy (no red triangles)
|
||||||
|
- [ ] Comment nodes present and descriptive
|
||||||
|
- [ ] All 3 output ports wired to something (debug at minimum)
|
||||||
|
- [ ] IDs follow naming convention (no random hex)
|
||||||
|
- [ ] Node config uses realistic values (not empty strings or defaults)
|
||||||
|
- [ ] File named per convention (01/02/03 prefix)
|
||||||
|
|
||||||
|
## Gitea Wiki Integration
|
||||||
|
|
||||||
|
Each node's wiki gets an "Examples" page that:
|
||||||
|
1. Lists all available example flows with descriptions
|
||||||
|
2. Links to the raw .json file in the repo
|
||||||
|
3. Describes prerequisites and step-by-step usage
|
||||||
|
4. Shows expected behavior after deploy
|
||||||
@@ -22,3 +22,6 @@ Lifecycle:
|
|||||||
| IMP-20260219-022 | 2026-02-19 | generalFunctions/outliers | `DynamicClusterDeviation.update()` emits verbose `console.log` traces on each call with no log-level guard, unsafe for production telemetry volume. | `nodes/generalFunctions/src/outliers/outlierDetection.js:7` | open |
|
| IMP-20260219-022 | 2026-02-19 | generalFunctions/outliers | `DynamicClusterDeviation.update()` emits verbose `console.log` traces on each call with no log-level guard, unsafe for production telemetry volume. | `nodes/generalFunctions/src/outliers/outlierDetection.js:7` | open |
|
||||||
| IMP-20260224-006 | 2026-02-24 | rotatingMachine prediction fallback | When only one pressure side is available, predictor uses absolute pressure as surrogate differential, which can materially bias flow prediction under varying suction/discharge conditions. | `nodes/rotatingMachine/src/specificClass.js:573`, `nodes/rotatingMachine/src/specificClass.js:588` | open |
|
| IMP-20260224-006 | 2026-02-24 | rotatingMachine prediction fallback | When only one pressure side is available, predictor uses absolute pressure as surrogate differential, which can materially bias flow prediction under varying suction/discharge conditions. | `nodes/rotatingMachine/src/specificClass.js:573`, `nodes/rotatingMachine/src/specificClass.js:588` | open |
|
||||||
| IMP-20260224-012 | 2026-02-24 | cross-node unit architecture | Canonical unit-anchor strategy is implemented in rotatingMachine plus phase-1 controllers (`machineGroupControl`, `pumpingStation`, `valve`, `valveGroupControl`); continue rollout to remaining nodes so all runtime paths use canonical storage + explicit ingress/egress units. | `nodes/machineGroupControl/src/specificClass.js:42`, `nodes/pumpingStation/src/specificClass.js:48`, `nodes/valve/src/specificClass.js:87`, `nodes/valveGroupControl/src/specificClass.js:78` | open |
|
| IMP-20260224-012 | 2026-02-24 | cross-node unit architecture | Canonical unit-anchor strategy is implemented in rotatingMachine plus phase-1 controllers (`machineGroupControl`, `pumpingStation`, `valve`, `valveGroupControl`); continue rollout to remaining nodes so all runtime paths use canonical storage + explicit ingress/egress units. | `nodes/machineGroupControl/src/specificClass.js:42`, `nodes/pumpingStation/src/specificClass.js:48`, `nodes/valve/src/specificClass.js:87`, `nodes/valveGroupControl/src/specificClass.js:78` | open |
|
||||||
|
| IMP-20260323-001 | 2026-03-23 | architecture/security | `temp/cloud.yml` stores environment credentials directly in a repository-tracked target-state stack example; replace with env placeholders/secret injection and split illustrative architecture from deployable manifests. | `temp/cloud.yml:1` | open |
|
||||||
|
| IMP-20260323-002 | 2026-03-23 | architecture/configuration | Intended database-backed configuration authority (`tagcodering`) is not yet visibly integrated as the primary runtime config backbone in this repository; define access pattern, schema ownership, and rollout path for edge/site/central consumers. | `architecture/stack-architecture-review.md:1` | open |
|
||||||
|
| IMP-20260323-003 | 2026-03-23 | architecture/telemetry | Multi-level smart-storage strategy is a stated architecture goal, but signal classes, reconstruction guarantees, and authoritative-layer rules are not yet formalized; define telemetry policy before broad deployment. | `architecture/stack-architecture-review.md:1` | open |
|
||||||
|
|||||||
77
.claude/settings.local.json
Normal file
77
.claude/settings.local.json
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
{
|
||||||
|
"permissions": {
|
||||||
|
"allow": [
|
||||||
|
"Bash(node --test:*)",
|
||||||
|
"Bash(node -c:*)",
|
||||||
|
"Bash(npm:*)",
|
||||||
|
"Bash(git:*)",
|
||||||
|
"Bash(ls:*)",
|
||||||
|
"Bash(tree:*)",
|
||||||
|
"Bash(wc:*)",
|
||||||
|
"Bash(head:*)",
|
||||||
|
"Bash(tail:*)",
|
||||||
|
"Bash(sort:*)",
|
||||||
|
"Bash(find:*)",
|
||||||
|
"Bash(echo:*)",
|
||||||
|
"Bash(cat:*)",
|
||||||
|
"Bash(cut:*)",
|
||||||
|
"Bash(xargs:*)",
|
||||||
|
"WebSearch",
|
||||||
|
"WebFetch(domain:nodered.org)",
|
||||||
|
"WebFetch(domain:docs.influxdata.com)",
|
||||||
|
"WebFetch(domain:github.com)",
|
||||||
|
"WebFetch(domain:docs.anthropic.com)",
|
||||||
|
"WebFetch(domain:nodejs.org)",
|
||||||
|
"WebFetch(domain:www.npmjs.com)",
|
||||||
|
"WebFetch(domain:developer.mozilla.org)",
|
||||||
|
"WebFetch(domain:flowfuse.com)",
|
||||||
|
"WebFetch(domain:www.coolprop.org)",
|
||||||
|
"WebFetch(domain:en.wikipedia.org)",
|
||||||
|
"WebFetch(domain:www.engineeringtoolbox.com)",
|
||||||
|
"mcp__ide__getDiagnostics",
|
||||||
|
"Bash(chmod +x:*)",
|
||||||
|
"Bash(docker compose:*)",
|
||||||
|
"Bash(docker:*)",
|
||||||
|
"Bash(npm run docker:*)",
|
||||||
|
"Bash(sh:*)",
|
||||||
|
"Bash(curl:*)",
|
||||||
|
"Bash(# Check Node-RED context for the parse function to see if it received data\ndocker compose exec -T nodered sh -c 'curl -sf \"http://localhost:1880/context/node/demo_fn_ps_west_parse\" 2>/dev/null' | python3 -c \"\nimport json, sys\ntry:\n data = json.load\\(sys.stdin\\)\n print\\(json.dumps\\(data, indent=2\\)[:800]\\)\nexcept Exception as e: print\\(f'Error: {e}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Check what the deployed flow looks like for link out type nodes\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\n# All node types and their counts\nfrom collections import Counter\ntypes = Counter\\(n.get\\('type',''\\) for n in flows if 'type' in n\\)\nfor t, c in sorted\\(types.items\\(\\)\\):\n if 'link' in t.lower\\(\\):\n print\\(f'{t}: {c}'\\)\nprint\\('---'\\)\n# Show existing link out nodes\nfor n in flows:\n if n.get\\('type'\\) == 'link out':\n print\\(f' {n[\\\\\"id\\\\\"]}: links={n.get\\(\\\\\"links\\\\\",[]\\)}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Full count of all deployed node types\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfrom collections import Counter\ntypes = Counter\\(n.get\\('type',''\\) for n in flows if 'type' in n\\)\nfor t, c in sorted\\(types.items\\(\\)\\):\n print\\(f'{t:30s}: {c}'\\)\nprint\\(f'Total nodes: {len\\(flows\\)}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Check exact registered node type names\ncurl -sf http://localhost:1880/nodes 2>/dev/null | python3 -c \"\nimport json, sys\nnodes = json.load\\(sys.stdin\\)\nfor mod in nodes:\n if 'EVOLV' in json.dumps\\(mod\\) or 'evolv' in json.dumps\\(mod\\).lower\\(\\):\n if isinstance\\(mod, dict\\) and 'types' in mod:\n for t in mod['types']:\n print\\(f'Registered type: {t}'\\)\n elif isinstance\\(mod, dict\\) and 'nodes' in mod:\n for n in mod['nodes']:\n for t in n.get\\('types', []\\):\n print\\(f'Registered type: {t}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Get node types from the /nodes endpoint properly\ndocker compose exec -T nodered sh -c 'curl -sf http://localhost:1880/nodes' | python3 -c \"\nimport json, sys\ndata = json.load\\(sys.stdin\\)\n# Find EVOLV node types\nfor module in data:\n if isinstance\\(module, dict\\):\n name = module.get\\('name', module.get\\('module', ''\\)\\)\n if 'EVOLV' in str\\(name\\).upper\\(\\) or 'evolv' in str\\(name\\).lower\\(\\):\n print\\(f'Module: {name}'\\)\n for node_set in module.get\\('nodes', []\\):\n for t in node_set.get\\('types', []\\):\n print\\(f' Type: {t}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Get raw flow data directly from inside the container\ndocker compose exec -T nodered sh -c 'curl -sf http://localhost:1880/flows 2>/dev/null' | python3 -c \"\nimport json, sys\ndata = json.load\\(sys.stdin\\)\nprint\\(f'Total entries: {len\\(data\\)}'\\)\nprint\\(f'Type: {type\\(data\\)}'\\)\nif isinstance\\(data, list\\):\n print\\('First 3:'\\)\n for n in data[:3]:\n print\\(f' {n.get\\(\\\\\"id\\\\\",\\\\\"?\\\\\"\\)}: type={n.get\\(\\\\\"type\\\\\",\\\\\"?\\\\\"\\)}'\\)\n # Count\n from collections import Counter\n types = Counter\\(n.get\\('type',''\\) for n in data\\)\n for t, c in sorted\\(types.items\\(\\)\\):\n print\\(f' {t}: {c}'\\)\nelif isinstance\\(data, dict\\):\n print\\(f'Keys: {list\\(data.keys\\(\\)\\)}'\\)\n if 'flows' in data:\n flows = data['flows']\n print\\(f'Flows count: {len\\(flows\\)}'\\)\n from collections import Counter\n types = Counter\\(n.get\\('type',''\\) for n in flows\\)\n for t, c in sorted\\(types.items\\(\\)\\):\n print\\(f' {t}: {c}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(# Check individual tab flows\ndocker compose exec -T nodered sh -c 'curl -sf http://localhost:1880/flow/demo_tab_wwtp' | python3 -c \"\nimport json, sys\ndata = json.load\\(sys.stdin\\)\nif isinstance\\(data, dict\\):\n print\\(f'Tab: {data.get\\(\\\\\"label\\\\\",\\\\\"?\\\\\"\\)}'\\)\n nodes = data.get\\('nodes', []\\)\n print\\(f'Nodes: {len\\(nodes\\)}'\\)\n from collections import Counter\n types = Counter\\(n.get\\('type',''\\) for n in nodes\\)\n for t, c in sorted\\(types.items\\(\\)\\):\n print\\(f' {t}: {c}'\\)\nelse:\n print\\(data\\)\n\" 2>&1)",
|
||||||
|
"Bash(sleep 5:*)",
|
||||||
|
"Bash(sleep 15:*)",
|
||||||
|
"Bash(# Get all dashboard UIDs and update the bucket variable from lvl2 to telemetry\ncurl -sf -H \"Authorization: Bearer glsa_4tbdInvrkQ6c7J6N3InjSsH8de83vZ66_9db7efa3\" \\\\\n \"http://localhost:3000/api/search?type=dash-db\" | python3 -c \"\nimport json, sys\ndashboards = json.load\\(sys.stdin\\)\nfor d in dashboards:\n print\\(d['uid']\\)\n\" 2>&1)",
|
||||||
|
"Bash(sleep 20:*)",
|
||||||
|
"Bash(# Check reactor parse function context\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\n# Find parse functions by name\nfor n in flows:\n if n.get\\('type'\\) == 'function' and 'reactor' in n.get\\('name',''\\).lower\\(\\):\n print\\(f\\\\\"Reactor parse: id={n['id']}, name={n.get\\('name'\\)}\\\\\"\\)\" 2>&1)",
|
||||||
|
"Bash(# Check if reactor node is sending output — look at debug info\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\n# Find the reactor node and its wires\nfor n in flows:\n if n.get\\('type'\\) == 'reactor':\n print\\(f\\\\\"Reactor: id={n['id']}, name={n.get\\('name',''\\)}\\\\\"\\)\n wires = n.get\\('wires', []\\)\n for i, port in enumerate\\(wires\\):\n print\\(f' Port {i}: {port}'\\)\n if n.get\\('type'\\) == 'link out' and 'reactor' in n.get\\('name',''\\).lower\\(\\):\n print\\(f\\\\\"Link-out reactor: id={n['id']}, name={n.get\\('name',''\\)}, links={n.get\\('links',[]\\)}\\\\\"\\)\" 2>&1)",
|
||||||
|
"Bash(# Check measurement node wiring and output\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('type'\\) == 'measurement':\n print\\(f\\\\\"Measurement: id={n['id']}, name={n.get\\('name',''\\)}\\\\\"\\)\n wires = n.get\\('wires', []\\)\n for i, port in enumerate\\(wires\\):\n print\\(f' Port {i}: {port}'\\)\n if n.get\\('type'\\) == 'link out' and 'meas' in n.get\\('name',''\\).lower\\(\\):\n print\\(f\\\\\"Link-out meas: id={n['id']}, name={n.get\\('name',''\\)}, links={n.get\\('links',[]\\)}\\\\\"\\)\" 2>&1)",
|
||||||
|
"Bash(# Check reactor node config and measurement configs\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('type'\\) == 'reactor':\n print\\('=== REACTOR CONFIG ==='\\)\n for k,v in sorted\\(n.items\\(\\)\\):\n if k not in \\('wires','x','y','z'\\):\n print\\(f' {k}: {v}'\\)\n if n.get\\('type'\\) == 'measurement' and n.get\\('id'\\) == 'demo_meas_flow':\n print\\('=== MEASUREMENT FT-001 CONFIG ==='\\)\n for k,v in sorted\\(n.items\\(\\)\\):\n if k not in \\('wires','x','y','z'\\):\n print\\(f' {k}: {v}'\\)\" 2>&1)",
|
||||||
|
"Bash(# Check what inject/input nodes target the measurement nodes\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\n\n# Find all nodes that wire INTO the measurement nodes\nmeas_ids = {'demo_meas_flow', 'demo_meas_do', 'demo_meas_nh4'}\nfor n in flows:\n wires = n.get\\('wires', []\\)\n for port_idx, port_wires in enumerate\\(wires\\):\n for target in port_wires:\n if target in meas_ids:\n print\\(f'{n.get\\(\\\\\"type\\\\\"\\)}:{n.get\\(\\\\\"name\\\\\",\\\\\"\\\\\"\\)} \\(id={n.get\\(\\\\\"id\\\\\"\\)}\\) port {port_idx} → {target}'\\)\n\n# Check inject nodes that send to measurements \nprint\\(\\)\nprint\\('=== Inject nodes ==='\\)\nfor n in flows:\n if n.get\\('type'\\) == 'inject':\n wires = n.get\\('wires', []\\)\n all_targets = [t for port in wires for t in port]\n print\\(f'inject: {n.get\\(\\\\\"name\\\\\",\\\\\"\\\\\"\\)} id={n.get\\(\\\\\"id\\\\\"\\)} → targets={all_targets} repeat={n.get\\(\\\\\"repeat\\\\\",\\\\\"\\\\\"\\)} topic={n.get\\(\\\\\"topic\\\\\",\\\\\"\\\\\"\\)}'\\)\" 2>&1)",
|
||||||
|
"Bash(# Check the simulator function code for measurements\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('id'\\) in \\('demo_fn_sim_flow', 'demo_fn_sim_do', 'demo_fn_sim_nh4'\\):\n print\\(f'=== {n.get\\(\\\\\"name\\\\\"\\)} ==='\\)\n print\\(n.get\\('func',''\\)\\)\n print\\(\\)\" 2>&1)",
|
||||||
|
"Bash(# Check what the reactor tick inject sends\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('id'\\) == 'demo_inj_reactor_tick':\n print\\('=== Reactor tick inject ==='\\)\n for k,v in sorted\\(n.items\\(\\)\\):\n if k not in \\('x','y','z','wires'\\):\n print\\(f' {k}: {v}'\\)\n if n.get\\('id'\\) == 'demo_inj_meas_flow':\n print\\('=== Flow sensor inject ==='\\)\n for k,v in sorted\\(n.items\\(\\)\\):\n if k not in \\('x','y','z','wires'\\):\n print\\(f' {k}: {v}'\\)\" 2>&1)",
|
||||||
|
"Bash(# Check measurement parse function code\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('id'\\) == 'demo_fn_reactor_parse':\n print\\('=== Parse Reactor ==='\\)\n print\\(n.get\\('func',''\\)\\)\n print\\(\\)\n if n.get\\('id'\\) == 'demo_fn_meas_parse':\n print\\('=== Parse Measurements ==='\\)\n print\\(n.get\\('func',''\\)\\)\n print\\(\\)\n if n.get\\('type'\\) == 'function' and 'meas' in n.get\\('name',''\\).lower\\(\\) and 'parse' in n.get\\('name',''\\).lower\\(\\):\n print\\(f'=== {n.get\\(\\\\\"name\\\\\"\\)} \\(id={n.get\\(\\\\\"id\\\\\"\\)}\\) ==='\\)\n print\\(n.get\\('func',''\\)\\)\n print\\(\\)\" 2>&1)",
|
||||||
|
"Bash(# Check the link node pairs are properly paired\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nnodes = {n['id']: n for n in flows if 'id' in n}\n\nlink_outs = [n for n in flows if n.get\\('type'\\) == 'link out']\nlink_ins = [n for n in flows if n.get\\('type'\\) == 'link in']\n\nprint\\('=== Link-out nodes ==='\\)\nfor lo in link_outs:\n links = lo.get\\('links', []\\)\n targets = [nodes.get\\(l, {}\\).get\\('name', f'MISSING:{l}'\\) for l in links]\n tab = nodes.get\\(lo.get\\('z',''\\), {}\\).get\\('label', '?'\\)\n print\\(f' [{tab}] {lo.get\\(\\\\\"name\\\\\",\\\\\"\\\\\"\\)} \\(id={lo[\\\\\"id\\\\\"]}\\) → {targets}'\\)\n\nprint\\(\\)\nprint\\('=== Link-in nodes ==='\\) \nfor li in link_ins:\n links = li.get\\('links', []\\)\n tab = nodes.get\\(li.get\\('z',''\\), {}\\).get\\('label', '?'\\)\n print\\(f' [{tab}] {li.get\\(\\\\\"name\\\\\",\\\\\"\\\\\"\\)} \\(id={li[\\\\\"id\\\\\"]}\\) links={links}'\\)\" 2>&1)",
|
||||||
|
"Bash(sleep 8:*)",
|
||||||
|
"Bash(# Check the InfluxDB convert function and HTTP request config\ncurl -sf http://localhost:1880/flows 2>/dev/null | python3 -c \"\nimport json, sys\nflows = json.load\\(sys.stdin\\)\nfor n in flows:\n if n.get\\('id'\\) == 'demo_fn_influx_convert':\n print\\('=== InfluxDB Convert Function ==='\\)\n print\\(f'func: {n.get\\(\\\\\"func\\\\\",\\\\\"\\\\\"\\)}'\\)\n print\\(f'wires: {n.get\\(\\\\\"wires\\\\\",[]\\)}'\\)\n print\\(\\)\n if n.get\\('id'\\) == 'demo_http_influx':\n print\\('=== Write InfluxDB HTTP ==='\\)\n for k,v in sorted\\(n.items\\(\\)\\):\n if k not in \\('x','y','z','wires'\\):\n print\\(f' {k}: {v}'\\)\n print\\(f' wires: {n.get\\(\\\\\"wires\\\\\",[]\\)}'\\)\n\" 2>&1)",
|
||||||
|
"Bash(echo Grafana API not accessible:*)",
|
||||||
|
"Bash(python3 -c \":*)",
|
||||||
|
"Bash(__NEW_LINE_6565c53f4a65adcb__ echo \"\")",
|
||||||
|
"Bash(__NEW_LINE_43bd4a070667d63e__ echo \"\")",
|
||||||
|
"Bash(node:*)",
|
||||||
|
"Bash(python3:*)",
|
||||||
|
"WebFetch(domain:dashboard.flowfuse.com)",
|
||||||
|
"Bash(do echo:*)",
|
||||||
|
"Bash(__NEW_LINE_5a355214e3d8caae__ git:*)",
|
||||||
|
"Bash(git add:*)",
|
||||||
|
"Bash(__NEW_LINE_4762b8ca1fb65139__ for:*)",
|
||||||
|
"Bash(docker.exe ps:*)",
|
||||||
|
"Bash(docker.exe logs:*)",
|
||||||
|
"Bash(docker.exe compose:*)",
|
||||||
|
"Bash(docker.exe exec:*)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
270
architecture/deployment-blueprint.md
Normal file
270
architecture/deployment-blueprint.md
Normal file
@@ -0,0 +1,270 @@
|
|||||||
|
# EVOLV Deployment Blueprint
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document turns the current EVOLV architecture into a concrete deployment model.
|
||||||
|
|
||||||
|
It focuses on:
|
||||||
|
|
||||||
|
- target infrastructure layout
|
||||||
|
- container/service topology
|
||||||
|
- environment and secret boundaries
|
||||||
|
- rollout order from edge to site to central
|
||||||
|
|
||||||
|
It is the local source document behind the wiki deployment pages.
|
||||||
|
|
||||||
|
## 1. Deployment Principles
|
||||||
|
|
||||||
|
- edge-first operation: plant logic must continue when central is unavailable
|
||||||
|
- site mediation: site services protect field systems and absorb plant-specific complexity
|
||||||
|
- central governance: external APIs, analytics, IAM, CI/CD, and shared dashboards terminate centrally
|
||||||
|
- layered telemetry: InfluxDB exists where operationally justified at edge, site, and central
|
||||||
|
- configuration authority: `tagcodering` should become the source of truth for configuration
|
||||||
|
- secrets hygiene: tracked manifests contain variables only; secrets live in server-side env or secret stores
|
||||||
|
|
||||||
|
## 2. Layered Deployment Model
|
||||||
|
|
||||||
|
### 2.1 Edge node
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- interface with PLCs and field assets
|
||||||
|
- execute local Node-RED logic
|
||||||
|
- retain local telemetry for resilience and digital-twin use cases
|
||||||
|
|
||||||
|
Recommended services:
|
||||||
|
|
||||||
|
- `evolv-edge-nodered`
|
||||||
|
- `evolv-edge-influxdb`
|
||||||
|
- optional `evolv-edge-grafana`
|
||||||
|
- optional `evolv-edge-broker`
|
||||||
|
|
||||||
|
Should not host:
|
||||||
|
|
||||||
|
- public API ingress
|
||||||
|
- central IAM
|
||||||
|
- source control or CI/CD
|
||||||
|
|
||||||
|
### 2.2 Site node
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- aggregate one or more edge nodes
|
||||||
|
- host plant-local dashboards and engineering visibility
|
||||||
|
- mediate traffic between edge and central
|
||||||
|
|
||||||
|
Recommended services:
|
||||||
|
|
||||||
|
- `evolv-site-nodered` or `coresync-site`
|
||||||
|
- `evolv-site-influxdb`
|
||||||
|
- `evolv-site-grafana`
|
||||||
|
- optional `evolv-site-broker`
|
||||||
|
|
||||||
|
### 2.3 Central platform
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- fleet-wide analytics
|
||||||
|
- API and integration ingress
|
||||||
|
- engineering lifecycle and releases
|
||||||
|
- identity and governance
|
||||||
|
|
||||||
|
Recommended services:
|
||||||
|
|
||||||
|
- reverse proxy / ingress
|
||||||
|
- API gateway
|
||||||
|
- IAM
|
||||||
|
- central InfluxDB
|
||||||
|
- central Grafana
|
||||||
|
- Gitea
|
||||||
|
- CI/CD runner/controller
|
||||||
|
- optional broker for asynchronous site/central workflows
|
||||||
|
- configuration services over `tagcodering`
|
||||||
|
|
||||||
|
## 3. Target Container Topology
|
||||||
|
|
||||||
|
### 3.1 Edge host
|
||||||
|
|
||||||
|
Minimum viable edge stack:
|
||||||
|
|
||||||
|
```text
|
||||||
|
edge-host-01
|
||||||
|
- Node-RED
|
||||||
|
- InfluxDB
|
||||||
|
- optional Grafana
|
||||||
|
```
|
||||||
|
|
||||||
|
Preferred production edge stack:
|
||||||
|
|
||||||
|
```text
|
||||||
|
edge-host-01
|
||||||
|
- Node-RED
|
||||||
|
- InfluxDB
|
||||||
|
- local health/export service
|
||||||
|
- optional local broker
|
||||||
|
- optional local dashboard service
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 Site host
|
||||||
|
|
||||||
|
Minimum viable site stack:
|
||||||
|
|
||||||
|
```text
|
||||||
|
site-host-01
|
||||||
|
- Site Node-RED / CoreSync
|
||||||
|
- Site InfluxDB
|
||||||
|
- Site Grafana
|
||||||
|
```
|
||||||
|
|
||||||
|
Preferred production site stack:
|
||||||
|
|
||||||
|
```text
|
||||||
|
site-host-01
|
||||||
|
- Site Node-RED / CoreSync
|
||||||
|
- Site InfluxDB
|
||||||
|
- Site Grafana
|
||||||
|
- API relay / sync service
|
||||||
|
- optional site broker
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.3 Central host group
|
||||||
|
|
||||||
|
Central should not be one giant undifferentiated host forever. It should trend toward at least these responsibility groups:
|
||||||
|
|
||||||
|
```text
|
||||||
|
central-ingress
|
||||||
|
- reverse proxy
|
||||||
|
- API gateway
|
||||||
|
- IAM
|
||||||
|
|
||||||
|
central-observability
|
||||||
|
- central InfluxDB
|
||||||
|
- Grafana
|
||||||
|
|
||||||
|
central-engineering
|
||||||
|
- Gitea
|
||||||
|
- CI/CD
|
||||||
|
- deployment orchestration
|
||||||
|
|
||||||
|
central-config
|
||||||
|
- tagcodering-backed config services
|
||||||
|
```
|
||||||
|
|
||||||
|
For early rollout these may be colocated, but the responsibility split should remain clear.
|
||||||
|
|
||||||
|
## 4. Compose Strategy
|
||||||
|
|
||||||
|
The current repository shows:
|
||||||
|
|
||||||
|
- `docker-compose.yml` as a development stack
|
||||||
|
- `temp/cloud.yml` as a broad central-stack example
|
||||||
|
|
||||||
|
For production, EVOLV should not rely on one flat compose file for every layer.
|
||||||
|
|
||||||
|
Recommended split:
|
||||||
|
|
||||||
|
- `compose.edge.yml`
|
||||||
|
- `compose.site.yml`
|
||||||
|
- `compose.central.yml`
|
||||||
|
- optional overlay files for site-specific differences
|
||||||
|
|
||||||
|
Benefits:
|
||||||
|
|
||||||
|
- clearer ownership per layer
|
||||||
|
- smaller blast radius during updates
|
||||||
|
- easier secret and env separation
|
||||||
|
- easier rollout per site
|
||||||
|
|
||||||
|
## 5. Environment And Secrets Strategy
|
||||||
|
|
||||||
|
### 5.1 Current baseline
|
||||||
|
|
||||||
|
`temp/cloud.yml` now uses environment variables instead of inline credentials. That is the minimum acceptable baseline.
|
||||||
|
|
||||||
|
### 5.2 Recommended production rule
|
||||||
|
|
||||||
|
- tracked compose files contain `${VARIABLE}` placeholders only
|
||||||
|
- real secrets live in server-local `.env` files or a managed secret store
|
||||||
|
- no shared default production passwords in git
|
||||||
|
- separate env files per layer and per environment
|
||||||
|
|
||||||
|
Suggested structure:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/evolv/
|
||||||
|
compose.edge.yml
|
||||||
|
compose.site.yml
|
||||||
|
compose.central.yml
|
||||||
|
env/
|
||||||
|
edge.env
|
||||||
|
site.env
|
||||||
|
central.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. Recommended Network Flow
|
||||||
|
|
||||||
|
### 6.1 Northbound
|
||||||
|
|
||||||
|
- edge publishes or syncs upward to site
|
||||||
|
- site aggregates and forwards selected data to central
|
||||||
|
- central exposes APIs and dashboards to approved consumers
|
||||||
|
|
||||||
|
### 6.2 Southbound
|
||||||
|
|
||||||
|
- central issues advice, approved config, or mediated requests
|
||||||
|
- site validates and relays to edge where appropriate
|
||||||
|
- edge remains the execution point near PLCs
|
||||||
|
|
||||||
|
### 6.3 Forbidden direct path
|
||||||
|
|
||||||
|
- enterprise or internet clients should not directly query PLC-connected edge runtimes
|
||||||
|
|
||||||
|
## 7. Rollout Order
|
||||||
|
|
||||||
|
### Phase 1: Edge baseline
|
||||||
|
|
||||||
|
- deploy edge Node-RED
|
||||||
|
- deploy local InfluxDB
|
||||||
|
- validate PLC connectivity
|
||||||
|
- validate local telemetry and resilience
|
||||||
|
|
||||||
|
### Phase 2: Site mediation
|
||||||
|
|
||||||
|
- deploy site Node-RED / CoreSync
|
||||||
|
- connect one or more edge nodes
|
||||||
|
- validate site-local dashboards and outage behavior
|
||||||
|
|
||||||
|
### Phase 3: Central services
|
||||||
|
|
||||||
|
- deploy ingress, IAM, API, Grafana, central InfluxDB
|
||||||
|
- deploy Gitea and CI/CD services
|
||||||
|
- validate controlled northbound access
|
||||||
|
|
||||||
|
### Phase 4: Configuration backbone
|
||||||
|
|
||||||
|
- connect runtime layers to `tagcodering`
|
||||||
|
- reduce config duplication in flows
|
||||||
|
- formalize config promotion and rollback
|
||||||
|
|
||||||
|
### Phase 5: Smart telemetry policy
|
||||||
|
|
||||||
|
- classify signals
|
||||||
|
- define reconstruction rules
|
||||||
|
- define authoritative layer per horizon
|
||||||
|
- validate analytics and auditability
|
||||||
|
|
||||||
|
## 8. Immediate Technical Recommendations
|
||||||
|
|
||||||
|
- treat `docker/settings.js` as development-only and create hardened production settings separately
|
||||||
|
- split deployment manifests by layer
|
||||||
|
- define env files per layer and environment
|
||||||
|
- formalize healthchecks and backup procedures for every persistent service
|
||||||
|
- define whether broker usage is required at edge, site, central, or only selectively
|
||||||
|
|
||||||
|
## 9. Next Technical Work Items
|
||||||
|
|
||||||
|
1. create draft `compose.edge.yml`, `compose.site.yml`, and `compose.central.yml`
|
||||||
|
2. define server directory layout and env-file conventions
|
||||||
|
3. define production Node-RED settings profile
|
||||||
|
4. define site-to-central sync path
|
||||||
|
5. define deployment and rollback runbook
|
||||||
624
architecture/stack-architecture-review.md
Normal file
624
architecture/stack-architecture-review.md
Normal file
@@ -0,0 +1,624 @@
|
|||||||
|
# EVOLV Architecture Review
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document captures:
|
||||||
|
|
||||||
|
- the architecture implemented in this repository today
|
||||||
|
- the broader edge/site/central architecture shown in the drawings under `temp/`
|
||||||
|
- the key strengths and weaknesses of that direction
|
||||||
|
- the currently preferred target stack based on owner decisions from this review
|
||||||
|
|
||||||
|
It is the local staging document for a later wiki update.
|
||||||
|
|
||||||
|
## Evidence Used
|
||||||
|
|
||||||
|
Implemented stack evidence:
|
||||||
|
|
||||||
|
- `docker-compose.yml`
|
||||||
|
- `docker/settings.js`
|
||||||
|
- `docker/grafana/provisioning/datasources/influxdb.yaml`
|
||||||
|
- `package.json`
|
||||||
|
- `nodes/*`
|
||||||
|
|
||||||
|
Target-state evidence:
|
||||||
|
|
||||||
|
- `temp/fullStack.pdf`
|
||||||
|
- `temp/edge.pdf`
|
||||||
|
- `temp/CoreSync.drawio.pdf`
|
||||||
|
- `temp/cloud.yml`
|
||||||
|
|
||||||
|
Owner decisions from this review:
|
||||||
|
|
||||||
|
- local InfluxDB is required for operational resilience
|
||||||
|
- central acts as the advisory/intelligence and API-entry layer, not as a direct field caller
|
||||||
|
- intended configuration authority is the database-backed `tagcodering` model
|
||||||
|
- architecture wiki pages should be visual, not text-only
|
||||||
|
|
||||||
|
## 1. What Exists Today
|
||||||
|
|
||||||
|
### 1.1 Product/runtime layer
|
||||||
|
|
||||||
|
The codebase is currently a modular Node-RED package for wastewater/process automation:
|
||||||
|
|
||||||
|
- EVOLV ships custom Node-RED nodes for plant assets and process logic
|
||||||
|
- nodes emit both process/control messages and telemetry-oriented outputs
|
||||||
|
- shared helper logic lives in `nodes/generalFunctions/`
|
||||||
|
- Grafana-facing integration exists through `dashboardAPI` and Influx-oriented outputs
|
||||||
|
|
||||||
|
### 1.2 Implemented development stack
|
||||||
|
|
||||||
|
The concrete development stack in this repository is:
|
||||||
|
|
||||||
|
- Node-RED
|
||||||
|
- InfluxDB 2.x
|
||||||
|
- Grafana
|
||||||
|
|
||||||
|
That gives a clear local flow:
|
||||||
|
|
||||||
|
1. EVOLV logic runs in Node-RED.
|
||||||
|
2. Telemetry is emitted in a time-series-oriented shape.
|
||||||
|
3. InfluxDB stores the telemetry.
|
||||||
|
4. Grafana renders operational dashboards.
|
||||||
|
|
||||||
|
### 1.3 Existing runtime pattern in the nodes
|
||||||
|
|
||||||
|
A recurring EVOLV pattern is:
|
||||||
|
|
||||||
|
- output 0: process/control message
|
||||||
|
- output 1: Influx/telemetry message
|
||||||
|
- output 2: registration/control plumbing where relevant
|
||||||
|
|
||||||
|
So even in its current implemented form, EVOLV is not only a Node-RED project. It is already a control-plus-observability platform, with Node-RED as orchestration/runtime and InfluxDB/Grafana as telemetry and visualization services.
|
||||||
|
|
||||||
|
## 2. What The Drawings Describe
|
||||||
|
|
||||||
|
Across `temp/fullStack.pdf` and `temp/CoreSync.drawio.pdf`, the intended platform is broader and layered.
|
||||||
|
|
||||||
|
### 2.1 Edge / OT layer
|
||||||
|
|
||||||
|
The drawings consistently place these capabilities at the edge:
|
||||||
|
|
||||||
|
- PLC / OPC UA connectivity
|
||||||
|
- Node-RED container as protocol translator and logic runtime
|
||||||
|
- local broker in some variants
|
||||||
|
- local InfluxDB / Prometheus style storage in some variants
|
||||||
|
- local Grafana/SCADA in some variants
|
||||||
|
|
||||||
|
This is the plant-side operational layer.
|
||||||
|
|
||||||
|
### 2.2 Site / local server layer
|
||||||
|
|
||||||
|
The CoreSync drawings also show a site aggregation layer:
|
||||||
|
|
||||||
|
- RWZI-local server
|
||||||
|
- Node-RED / CoreSync services
|
||||||
|
- site-local broker
|
||||||
|
- site-local database
|
||||||
|
- upward API-based synchronization
|
||||||
|
|
||||||
|
This layer decouples field assets from central services and absorbs plant-specific complexity.
|
||||||
|
|
||||||
|
### 2.3 Central / cloud layer
|
||||||
|
|
||||||
|
The broader stack drawings and `temp/cloud.yml` show a central platform layer with:
|
||||||
|
|
||||||
|
- Gitea
|
||||||
|
- Jenkins
|
||||||
|
- reverse proxy / ingress
|
||||||
|
- Grafana
|
||||||
|
- InfluxDB
|
||||||
|
- Node-RED
|
||||||
|
- RabbitMQ / messaging
|
||||||
|
- VPN / tunnel concepts
|
||||||
|
- Keycloak in the drawing
|
||||||
|
- Portainer in the drawing
|
||||||
|
|
||||||
|
This is a platform-services layer, not just an application runtime.
|
||||||
|
|
||||||
|
## 3. Architecture Decisions From This Review
|
||||||
|
|
||||||
|
These decisions now shape the preferred EVOLV target architecture.
|
||||||
|
|
||||||
|
### 3.1 Local telemetry is mandatory for resilience
|
||||||
|
|
||||||
|
Local InfluxDB is not optional. It is required so that:
|
||||||
|
|
||||||
|
- operations continue when central SCADA or central services are down
|
||||||
|
- local dashboards and advanced digital-twin workflows can still consume recent and relevant process history
|
||||||
|
- local edge/site layers can make smarter decisions without depending on round-trips to central
|
||||||
|
|
||||||
|
### 3.2 Multi-level InfluxDB is part of the architecture
|
||||||
|
|
||||||
|
InfluxDB should exist on multiple levels where it adds operational value:
|
||||||
|
|
||||||
|
- edge/local for resilience and near-real-time replay
|
||||||
|
- site for plant-level history, diagnostics, and resilience
|
||||||
|
- central for fleet-wide analytics, benchmarking, and advisory intelligence
|
||||||
|
|
||||||
|
This is not just copy-paste storage at each level. The design intent is event-driven and selective.
|
||||||
|
|
||||||
|
### 3.3 Storage should be smart, not only deadband-driven
|
||||||
|
|
||||||
|
The target is not simple "store every point" or only a fixed deadband rule such as 1%.
|
||||||
|
|
||||||
|
The desired storage approach is:
|
||||||
|
|
||||||
|
- observe signal slope and change behavior
|
||||||
|
- preserve points where state is changing materially
|
||||||
|
- store fewer points where the signal can be reconstructed downstream with sufficient fidelity
|
||||||
|
- carry enough metadata or conventions so reconstruction quality is auditable
|
||||||
|
|
||||||
|
This implies EVOLV should evolve toward smart storage and signal-aware retention rather than naive event dumping.
|
||||||
|
|
||||||
|
### 3.4 Central is the intelligence and API-entry layer
|
||||||
|
|
||||||
|
Central may advise and coordinate edge/site layers, but external API requests should not hit field-edge systems directly.
|
||||||
|
|
||||||
|
The intended pattern is:
|
||||||
|
|
||||||
|
- external and enterprise integrations terminate centrally
|
||||||
|
- central evaluates, aggregates, authorizes, and advises
|
||||||
|
- site/edge layers receive mediated requests, policies, or setpoints
|
||||||
|
- field-edge remains protected behind an intermediate layer
|
||||||
|
|
||||||
|
This aligns with the stated security direction.
|
||||||
|
|
||||||
|
### 3.5 Configuration source of truth should be database-backed
|
||||||
|
|
||||||
|
The intended configuration authority is the database-backed `tagcodering` model, which already exists but is not yet complete enough to serve as the fully realized source of truth.
|
||||||
|
|
||||||
|
That means the architecture should assume:
|
||||||
|
|
||||||
|
- asset and machine metadata belong in `tagcodering`
|
||||||
|
- Node-RED flows should consume configuration rather than silently becoming the only configuration store
|
||||||
|
- more work is still needed before this behaves as the intended central configuration backbone
|
||||||
|
|
||||||
|
## 4. Visual Model
|
||||||
|
|
||||||
|
### 4.1 Platform topology
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph OT["OT / Field"]
|
||||||
|
PLC["PLC / IO"]
|
||||||
|
DEV["Sensors / Machines"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph EDGE["Edge Layer"]
|
||||||
|
ENR["Edge Node-RED"]
|
||||||
|
EDB["Local InfluxDB"]
|
||||||
|
EUI["Local Grafana / Local Monitoring"]
|
||||||
|
EBR["Optional Local Broker"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph SITE["Site Layer"]
|
||||||
|
SNR["Site Node-RED / CoreSync"]
|
||||||
|
SDB["Site InfluxDB"]
|
||||||
|
SUI["Site Grafana / SCADA Support"]
|
||||||
|
SBR["Site Broker"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph CENTRAL["Central Layer"]
|
||||||
|
API["API / Integration Gateway"]
|
||||||
|
INTEL["Overview Intelligence / Advisory Logic"]
|
||||||
|
CDB["Central InfluxDB"]
|
||||||
|
CGR["Central Grafana"]
|
||||||
|
CFG["Tagcodering Config Model"]
|
||||||
|
GIT["Gitea"]
|
||||||
|
CI["CI/CD"]
|
||||||
|
IAM["IAM / Keycloak"]
|
||||||
|
end
|
||||||
|
|
||||||
|
DEV --> PLC
|
||||||
|
PLC --> ENR
|
||||||
|
ENR --> EDB
|
||||||
|
ENR --> EUI
|
||||||
|
ENR --> EBR
|
||||||
|
ENR <--> SNR
|
||||||
|
EDB <--> SDB
|
||||||
|
SNR --> SDB
|
||||||
|
SNR --> SUI
|
||||||
|
SNR --> SBR
|
||||||
|
SNR <--> API
|
||||||
|
API --> INTEL
|
||||||
|
API <--> CFG
|
||||||
|
SDB <--> CDB
|
||||||
|
INTEL --> SNR
|
||||||
|
CGR --> CDB
|
||||||
|
CI --> GIT
|
||||||
|
IAM --> API
|
||||||
|
IAM --> CGR
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 Command and access boundary
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
EXT["External APIs / Enterprise Requests"] --> API["Central API Gateway"]
|
||||||
|
API --> AUTH["AuthN/AuthZ / Policy Checks"]
|
||||||
|
AUTH --> INTEL["Central Advisory / Decision Support"]
|
||||||
|
INTEL --> SITE["Site Integration Layer"]
|
||||||
|
SITE --> EDGE["Edge Runtime"]
|
||||||
|
EDGE --> PLC["PLC / Field Assets"]
|
||||||
|
|
||||||
|
EXT -. no direct access .-> EDGE
|
||||||
|
EXT -. no direct access .-> PLC
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.3 Smart telemetry flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
RAW["Raw Signal"] --> EDGELOGIC["Edge Signal Evaluation"]
|
||||||
|
EDGELOGIC --> KEEP["Keep Critical Change Points"]
|
||||||
|
EDGELOGIC --> SKIP["Skip Reconstructable Flat Points"]
|
||||||
|
EDGELOGIC --> LOCAL["Local InfluxDB"]
|
||||||
|
LOCAL --> SITE["Site InfluxDB"]
|
||||||
|
SITE --> CENTRAL["Central InfluxDB"]
|
||||||
|
KEEP --> LOCAL
|
||||||
|
SKIP -. reconstruction assumptions / metadata .-> SITE
|
||||||
|
CENTRAL --> DASH["Fleet Dashboards / Analytics"]
|
||||||
|
```
|
||||||
|
|
||||||
|
## 5. Upsides Of This Direction
|
||||||
|
|
||||||
|
### 5.1 Strong separation between control and observability
|
||||||
|
|
||||||
|
Node-RED for runtime/orchestration and InfluxDB/Grafana for telemetry is still the right structural split:
|
||||||
|
|
||||||
|
- control stays close to the process
|
||||||
|
- telemetry storage/querying stays in time-series-native tooling
|
||||||
|
- dashboards do not need to overload Node-RED itself
|
||||||
|
|
||||||
|
### 5.2 Edge-first matches operational reality
|
||||||
|
|
||||||
|
For wastewater/process systems, edge-first remains correct:
|
||||||
|
|
||||||
|
- lower latency
|
||||||
|
- better degraded-mode behavior
|
||||||
|
- less dependence on WAN or central platform uptime
|
||||||
|
- clearer OT trust boundary
|
||||||
|
|
||||||
|
### 5.3 Site mediation improves safety and security
|
||||||
|
|
||||||
|
Using central as the enterprise/API entry point and site as the mediator improves posture:
|
||||||
|
|
||||||
|
- field systems are less exposed
|
||||||
|
- policy decisions can be centralized
|
||||||
|
- external integrations do not probe the edge directly
|
||||||
|
- site can continue operating even when upstream is degraded
|
||||||
|
|
||||||
|
### 5.4 Multi-level storage enables better analytics
|
||||||
|
|
||||||
|
Multiple Influx layers can support:
|
||||||
|
|
||||||
|
- local resilience
|
||||||
|
- site diagnostics
|
||||||
|
- fleet benchmarking
|
||||||
|
- smarter retention and reconstruction strategies
|
||||||
|
|
||||||
|
That is substantially more capable than a single central historian model.
|
||||||
|
|
||||||
|
### 5.5 `tagcodering` is the right long-term direction
|
||||||
|
|
||||||
|
A database-backed configuration authority is stronger than embedding configuration only in flows because it supports:
|
||||||
|
|
||||||
|
- machine metadata management
|
||||||
|
- controlled rollout of configuration changes
|
||||||
|
- clearer versioning and provenance
|
||||||
|
- future API-driven configuration services
|
||||||
|
|
||||||
|
## 6. Downsides And Risks
|
||||||
|
|
||||||
|
### 6.1 Smart storage raises algorithmic and governance complexity
|
||||||
|
|
||||||
|
Signal-aware storage and reconstruction is promising, but it creates architectural obligations:
|
||||||
|
|
||||||
|
- reconstruction rules must be explicit
|
||||||
|
- acceptable reconstruction error must be defined per signal type
|
||||||
|
- operators must know whether they see raw or reconstructed history
|
||||||
|
- compliance-relevant data may need stricter retention than operational convenience data
|
||||||
|
|
||||||
|
Without those rules, smart storage can become opaque and hard to trust.
|
||||||
|
|
||||||
|
### 6.2 Multi-level databases can create ownership confusion
|
||||||
|
|
||||||
|
If edge, site, and central all store telemetry, you must define:
|
||||||
|
|
||||||
|
- which layer is authoritative for which time horizon
|
||||||
|
- when backfill is allowed
|
||||||
|
- when data is summarized vs copied
|
||||||
|
- how duplicates or gaps are detected
|
||||||
|
|
||||||
|
Otherwise operations will argue over which trend is "the real one."
|
||||||
|
|
||||||
|
### 6.3 Central intelligence must remain advisory-first
|
||||||
|
|
||||||
|
Central guidance can become valuable, but direct closed-loop dependency on central would be risky.
|
||||||
|
|
||||||
|
The architecture should therefore preserve:
|
||||||
|
|
||||||
|
- local control authority at edge/site
|
||||||
|
- bounded and explicit central advice
|
||||||
|
- safe behavior if central recommendations stop arriving
|
||||||
|
|
||||||
|
### 6.4 `tagcodering` is not yet complete enough to lean on blindly
|
||||||
|
|
||||||
|
It is the right target, but its current partial state means there is still architecture debt:
|
||||||
|
|
||||||
|
- incomplete config workflows
|
||||||
|
- likely mismatch between desired and implemented schema behavior
|
||||||
|
- temporary duplication between flows, node config, and database-held metadata
|
||||||
|
|
||||||
|
This should be treated as a core platform workstream, not a side issue.
|
||||||
|
|
||||||
|
### 6.5 Broker responsibilities are still not crisp enough
|
||||||
|
|
||||||
|
The materials still reference MQTT/AMQP/RabbitMQ/brokers without one stable responsibility split. That needs to be resolved before large-scale deployment.
|
||||||
|
|
||||||
|
Questions still open:
|
||||||
|
|
||||||
|
- command bus or event bus?
|
||||||
|
- site-only or cross-site?
|
||||||
|
- telemetry transport or only synchronization/eventing?
|
||||||
|
- durability expectations and replay behavior?
|
||||||
|
|
||||||
|
## 7. Security And Regulatory Positioning
|
||||||
|
|
||||||
|
### 7.1 Purdue-style layering is a good fit
|
||||||
|
|
||||||
|
EVOLV's preferred structure aligns well with a Purdue-style OT/IT layering approach:
|
||||||
|
|
||||||
|
- PLCs and field assets stay at the operational edge
|
||||||
|
- edge runtimes stay close to the process
|
||||||
|
- site systems mediate between OT and broader enterprise concerns
|
||||||
|
- central services host APIs, identity, analytics, and engineering workflows
|
||||||
|
|
||||||
|
That is important because it supports segmented trust boundaries instead of direct enterprise-to-field reach-through.
|
||||||
|
|
||||||
|
### 7.2 NIS2 alignment
|
||||||
|
|
||||||
|
Directive (EU) 2022/2555 (NIS2) requires cybersecurity risk-management measures, incident handling, and stronger governance for covered entities.
|
||||||
|
|
||||||
|
This architecture supports that by:
|
||||||
|
|
||||||
|
- limiting direct exposure of field systems
|
||||||
|
- separating operational layers
|
||||||
|
- enabling central policy and oversight
|
||||||
|
- preserving local operation during upstream failure
|
||||||
|
|
||||||
|
### 7.3 CER alignment
|
||||||
|
|
||||||
|
Directive (EU) 2022/2557 (Critical Entities Resilience Directive) focuses on resilience of essential services.
|
||||||
|
|
||||||
|
The edge-plus-site approach supports that direction because:
|
||||||
|
|
||||||
|
- local/site layers can continue during central disruption
|
||||||
|
- essential service continuity does not depend on one central runtime
|
||||||
|
- degraded-mode behavior can be explicitly designed per layer
|
||||||
|
|
||||||
|
### 7.4 Cyber Resilience Act alignment
|
||||||
|
|
||||||
|
Regulation (EU) 2024/2847 (Cyber Resilience Act) creates cybersecurity requirements for products with digital elements.
|
||||||
|
|
||||||
|
For EVOLV, that means the platform should keep strengthening:
|
||||||
|
|
||||||
|
- secure configuration handling
|
||||||
|
- vulnerability and update management
|
||||||
|
- release traceability
|
||||||
|
- lifecycle ownership of components and dependencies
|
||||||
|
|
||||||
|
### 7.5 GDPR alignment where personal data is present
|
||||||
|
|
||||||
|
Regulation (EU) 2016/679 (GDPR) applies whenever EVOLV processes personal data.
|
||||||
|
|
||||||
|
The architecture helps by:
|
||||||
|
|
||||||
|
- centralizing ingress
|
||||||
|
- reducing unnecessary propagation of data to field layers
|
||||||
|
- making access, retention, and audit boundaries easier to define
|
||||||
|
|
||||||
|
### 7.6 What can and cannot be claimed
|
||||||
|
|
||||||
|
The defensible claim is that EVOLV can be deployed in a way that supports compliance with strict European cybersecurity and resilience expectations.
|
||||||
|
|
||||||
|
The non-defensible claim is that EVOLV is automatically compliant purely because of the architecture diagram.
|
||||||
|
|
||||||
|
Actual compliance still depends on implementation and operations, including:
|
||||||
|
|
||||||
|
- access control
|
||||||
|
- patch and vulnerability management
|
||||||
|
- incident response
|
||||||
|
- logging and audit evidence
|
||||||
|
- retention policy
|
||||||
|
- data classification
|
||||||
|
|
||||||
|
## 8. Recommended Ideal Stack
|
||||||
|
|
||||||
|
The ideal EVOLV stack should be layered around operational boundaries, not around tools.
|
||||||
|
|
||||||
|
### 7.1 Layer A: Edge execution
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- connect to PLCs and field assets
|
||||||
|
- execute time-sensitive local logic
|
||||||
|
- preserve operation during WAN/central loss
|
||||||
|
- provide local telemetry access for resilience and digital-twin use cases
|
||||||
|
|
||||||
|
Recommended components:
|
||||||
|
|
||||||
|
- Node-RED runtime for EVOLV edge flows
|
||||||
|
- OPC UA and protocol adapters
|
||||||
|
- local InfluxDB
|
||||||
|
- optional local Grafana for local engineering/monitoring
|
||||||
|
- optional local broker only when multiple participants need decoupling
|
||||||
|
|
||||||
|
Principle:
|
||||||
|
|
||||||
|
- edge remains safe and useful when disconnected
|
||||||
|
|
||||||
|
### 7.2 Layer B: Site integration
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- aggregate multiple edge systems at plant/site level
|
||||||
|
- host plant-local dashboards and diagnostics
|
||||||
|
- mediate between raw OT detail and central standardization
|
||||||
|
- serve as the protected step between field systems and central requests
|
||||||
|
|
||||||
|
Recommended components:
|
||||||
|
|
||||||
|
- site Node-RED / CoreSync services
|
||||||
|
- site InfluxDB
|
||||||
|
- site Grafana / SCADA-supporting dashboards
|
||||||
|
- site broker where asynchronous eventing is justified
|
||||||
|
|
||||||
|
Principle:
|
||||||
|
|
||||||
|
- site absorbs plant complexity and protects field assets
|
||||||
|
|
||||||
|
### 7.3 Layer C: Central platform
|
||||||
|
|
||||||
|
Purpose:
|
||||||
|
|
||||||
|
- fleet-wide analytics
|
||||||
|
- shared dashboards
|
||||||
|
- engineering lifecycle
|
||||||
|
- enterprise/API entry point
|
||||||
|
- overview intelligence and advisory logic
|
||||||
|
|
||||||
|
Recommended components:
|
||||||
|
|
||||||
|
- Gitea
|
||||||
|
- CI/CD
|
||||||
|
- central InfluxDB
|
||||||
|
- central Grafana
|
||||||
|
- API/integration gateway
|
||||||
|
- IAM
|
||||||
|
- VPN/private connectivity
|
||||||
|
- `tagcodering`-backed configuration services
|
||||||
|
|
||||||
|
Principle:
|
||||||
|
|
||||||
|
- central coordinates, advises, and governs; it is not the direct field caller
|
||||||
|
|
||||||
|
### 7.4 Cross-cutting platform services
|
||||||
|
|
||||||
|
These should be explicit architecture elements:
|
||||||
|
|
||||||
|
- secrets management
|
||||||
|
- certificate management
|
||||||
|
- backup/restore
|
||||||
|
- audit logging
|
||||||
|
- monitoring/alerting of the platform itself
|
||||||
|
- versioned configuration and schema management
|
||||||
|
- rollout/rollback strategy
|
||||||
|
|
||||||
|
## 9. Recommended Opinionated Choices
|
||||||
|
|
||||||
|
### 8.1 Keep Node-RED as the orchestration layer, not the whole platform
|
||||||
|
|
||||||
|
Node-RED should own:
|
||||||
|
|
||||||
|
- process orchestration
|
||||||
|
- protocol mediation
|
||||||
|
- edge/site logic
|
||||||
|
- KPI production
|
||||||
|
|
||||||
|
It should not become the sole owner of:
|
||||||
|
|
||||||
|
- identity
|
||||||
|
- long-term configuration authority
|
||||||
|
- secret management
|
||||||
|
- compliance/audit authority
|
||||||
|
|
||||||
|
### 8.2 Use InfluxDB by function and horizon
|
||||||
|
|
||||||
|
Recommended split:
|
||||||
|
|
||||||
|
- edge: resilience, local replay, digital-twin input
|
||||||
|
- site: plant diagnostics and local continuity
|
||||||
|
- central: fleet analytics, advisory intelligence, benchmarking, and long-term cross-site views
|
||||||
|
|
||||||
|
### 8.3 Prefer smart telemetry retention over naive point dumping
|
||||||
|
|
||||||
|
Recommended rule:
|
||||||
|
|
||||||
|
- keep information-rich points
|
||||||
|
- reduce information-poor flat spans
|
||||||
|
- document reconstruction assumptions
|
||||||
|
- define signal-class-specific fidelity expectations
|
||||||
|
|
||||||
|
This needs design discipline, but it is a real differentiator if executed well.
|
||||||
|
|
||||||
|
### 8.4 Put enterprise/API ingress at central, not at edge
|
||||||
|
|
||||||
|
This should become a hard architectural rule:
|
||||||
|
|
||||||
|
- external requests land centrally
|
||||||
|
- central authenticates and authorizes
|
||||||
|
- central or site mediates downward
|
||||||
|
- edge never becomes the exposed public integration surface
|
||||||
|
|
||||||
|
### 8.5 Make `tagcodering` the target configuration backbone
|
||||||
|
|
||||||
|
The architecture should be designed so that `tagcodering` can mature into:
|
||||||
|
|
||||||
|
- machine and asset registry
|
||||||
|
- configuration source of truth
|
||||||
|
- site/central configuration exchange point
|
||||||
|
- API-served configuration source for runtime layers
|
||||||
|
|
||||||
|
## 10. Suggested Phasing
|
||||||
|
|
||||||
|
### Phase 1: Stabilize contracts
|
||||||
|
|
||||||
|
- define topic and payload contracts
|
||||||
|
- define telemetry classes and reconstruction policy
|
||||||
|
- define asset, machine, and site identity model
|
||||||
|
- define `tagcodering` scope and schema ownership
|
||||||
|
|
||||||
|
### Phase 2: Harden local/site resilience
|
||||||
|
|
||||||
|
- formalize edge and site runtime patterns
|
||||||
|
- define local telemetry retention and replay behavior
|
||||||
|
- define central-loss behavior
|
||||||
|
- define dashboard behavior during isolation
|
||||||
|
|
||||||
|
### Phase 3: Harden central platform
|
||||||
|
|
||||||
|
- IAM
|
||||||
|
- API gateway
|
||||||
|
- central observability
|
||||||
|
- CI/CD
|
||||||
|
- backup and disaster recovery
|
||||||
|
- config services over `tagcodering`
|
||||||
|
|
||||||
|
### Phase 4: Introduce selective synchronization and intelligence
|
||||||
|
|
||||||
|
- event-driven telemetry propagation rules
|
||||||
|
- smart-storage promotion/backfill policies
|
||||||
|
- advisory services from central
|
||||||
|
- auditability of downward recommendations and configuration changes
|
||||||
|
|
||||||
|
## 11. Immediate Open Questions Before Wiki Finalization
|
||||||
|
|
||||||
|
1. Which signals are allowed to use reconstruction-aware smart storage, and which must remain raw or near-raw for audit/compliance reasons?
|
||||||
|
2. How should `tagcodering` be exposed to runtime layers: direct database access, a dedicated API, or both?
|
||||||
|
3. What exact responsibility split should EVOLV use between API synchronization and broker-based eventing?
|
||||||
|
|
||||||
|
## 12. Recommended Wiki Structure
|
||||||
|
|
||||||
|
The wiki should not be one long page. It should be split into:
|
||||||
|
|
||||||
|
1. platform overview with the main topology diagram
|
||||||
|
2. edge-site-central runtime model
|
||||||
|
3. telemetry and smart storage model
|
||||||
|
4. security and access-boundary model
|
||||||
|
5. configuration architecture centered on `tagcodering`
|
||||||
|
|
||||||
|
## 13. Next Step
|
||||||
|
|
||||||
|
Use this document as the architecture baseline. The companion markdown page in `architecture/` can then be shaped into a wiki-ready visual overview page with Mermaid diagrams and shorter human-readable sections.
|
||||||
150
architecture/wiki-platform-overview.md
Normal file
150
architecture/wiki-platform-overview.md
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
# EVOLV Platform Architecture
|
||||||
|
|
||||||
|
## At A Glance
|
||||||
|
|
||||||
|
EVOLV is not only a Node-RED package. It is a layered automation platform:
|
||||||
|
|
||||||
|
- edge for plant-side execution
|
||||||
|
- site for local aggregation and resilience
|
||||||
|
- central for coordination, analytics, APIs, and governance
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph EDGE["Edge"]
|
||||||
|
PLC["PLC / IO"]
|
||||||
|
ENR["Node-RED"]
|
||||||
|
EDB["Local InfluxDB"]
|
||||||
|
EUI["Local Monitoring"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph SITE["Site"]
|
||||||
|
SNR["CoreSync / Site Node-RED"]
|
||||||
|
SDB["Site InfluxDB"]
|
||||||
|
SUI["Site Dashboards"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph CENTRAL["Central"]
|
||||||
|
API["API Gateway"]
|
||||||
|
CFG["Tagcodering"]
|
||||||
|
CDB["Central InfluxDB"]
|
||||||
|
CGR["Grafana"]
|
||||||
|
INTEL["Overview Intelligence"]
|
||||||
|
GIT["Gitea + CI/CD"]
|
||||||
|
end
|
||||||
|
|
||||||
|
PLC --> ENR
|
||||||
|
ENR --> EDB
|
||||||
|
ENR --> EUI
|
||||||
|
ENR <--> SNR
|
||||||
|
EDB <--> SDB
|
||||||
|
SNR --> SUI
|
||||||
|
SNR <--> API
|
||||||
|
API <--> CFG
|
||||||
|
API --> INTEL
|
||||||
|
SDB <--> CDB
|
||||||
|
CDB --> CGR
|
||||||
|
GIT --> ENR
|
||||||
|
GIT --> SNR
|
||||||
|
```
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
### 1. Edge-first operation
|
||||||
|
|
||||||
|
The edge layer must remain useful and safe when central systems are down.
|
||||||
|
|
||||||
|
That means:
|
||||||
|
|
||||||
|
- local logic remains operational
|
||||||
|
- local telemetry remains queryable
|
||||||
|
- local dashboards can keep working
|
||||||
|
|
||||||
|
### 2. Multi-level telemetry
|
||||||
|
|
||||||
|
InfluxDB is expected on multiple levels:
|
||||||
|
|
||||||
|
- local for resilience and digital-twin use
|
||||||
|
- site for plant diagnostics
|
||||||
|
- central for fleet analytics and advisory logic
|
||||||
|
|
||||||
|
### 3. Smart storage
|
||||||
|
|
||||||
|
Telemetry should not be stored only with naive deadband rules.
|
||||||
|
|
||||||
|
The target model is signal-aware:
|
||||||
|
|
||||||
|
- preserve critical change points
|
||||||
|
- reduce low-information flat sections
|
||||||
|
- allow downstream reconstruction where justified
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
SIG["Process Signal"] --> EVAL["Slope / Event Evaluation"]
|
||||||
|
EVAL --> KEEP["Keep critical points"]
|
||||||
|
EVAL --> REDUCE["Reduce reconstructable points"]
|
||||||
|
KEEP --> L0["Local InfluxDB"]
|
||||||
|
REDUCE --> L0
|
||||||
|
L0 --> L1["Site InfluxDB"]
|
||||||
|
L1 --> L2["Central InfluxDB"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Central is the safe entry point
|
||||||
|
|
||||||
|
External systems should enter through central APIs, not by directly calling field-edge systems.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
EXT["External Request"] --> API["Central API Gateway"]
|
||||||
|
API --> AUTH["Auth / Policy"]
|
||||||
|
AUTH --> SITE["Site Layer"]
|
||||||
|
SITE --> EDGE["Edge Layer"]
|
||||||
|
EDGE --> PLC["Field Assets"]
|
||||||
|
|
||||||
|
EXT -. blocked .-> EDGE
|
||||||
|
EXT -. blocked .-> PLC
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Configuration belongs in `tagcodering`
|
||||||
|
|
||||||
|
The intended configuration source of truth is the database-backed `tagcodering` model:
|
||||||
|
|
||||||
|
- machine metadata
|
||||||
|
- asset configuration
|
||||||
|
- runtime-consumable configuration
|
||||||
|
- future central/site configuration services
|
||||||
|
|
||||||
|
This already exists partially but still needs more work before it fully serves that role.
|
||||||
|
|
||||||
|
## Layer Roles
|
||||||
|
|
||||||
|
### Edge
|
||||||
|
|
||||||
|
- PLC connectivity
|
||||||
|
- local logic
|
||||||
|
- protocol translation
|
||||||
|
- local telemetry buffering
|
||||||
|
- local monitoring and digital-twin support
|
||||||
|
|
||||||
|
### Site
|
||||||
|
|
||||||
|
- aggregation of edge systems
|
||||||
|
- local dashboards and diagnostics
|
||||||
|
- mediation between OT and central
|
||||||
|
- protected handoff for central requests
|
||||||
|
|
||||||
|
### Central
|
||||||
|
|
||||||
|
- enterprise/API gateway
|
||||||
|
- fleet dashboards
|
||||||
|
- analytics and intelligence
|
||||||
|
- source control and CI/CD
|
||||||
|
- configuration governance through `tagcodering`
|
||||||
|
|
||||||
|
## Why This Matters
|
||||||
|
|
||||||
|
This architecture gives EVOLV:
|
||||||
|
|
||||||
|
- better resilience
|
||||||
|
- safer external integration
|
||||||
|
- better data quality for analytics
|
||||||
|
- a path from Node-RED package to platform
|
||||||
Submodule nodes/dashboardAPI updated: 547333be7d...89d2260351
Submodule nodes/generalFunctions updated: c60aa40666...27a6d3c709
Submodule nodes/machineGroupControl updated: f8012c8bad...b337bf9eb7
Submodule nodes/measurement updated: c587ed9c7b...43b5269f0b
Submodule nodes/monster updated: 38013a86db...32ebfd7154
Submodule nodes/reactor updated: 460b872053...2c69a5a0c1
Submodule nodes/rotatingMachine updated: 33f3c2ef61...6b2a8239f2
Submodule nodes/valve updated: d56f8a382c...6287708c1e
Submodule nodes/valveGroupControl updated: cbe868a148...5e1f3946bf
24
temp/cloud.env.example
Normal file
24
temp/cloud.env.example
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
# Copy this file to `.env` on the target server and populate real values there.
|
||||||
|
# Keep the real `.env` out of version control.
|
||||||
|
|
||||||
|
INFLUXDB_ADMIN_USER=replace-me
|
||||||
|
INFLUXDB_ADMIN_PASSWORD=replace-me
|
||||||
|
INFLUXDB_BUCKET=lvl0
|
||||||
|
INFLUXDB_ORG=wbd
|
||||||
|
|
||||||
|
GF_SECURITY_ADMIN_USER=replace-me
|
||||||
|
GF_SECURITY_ADMIN_PASSWORD=replace-me
|
||||||
|
|
||||||
|
NPM_DB_MYSQL_HOST=db
|
||||||
|
NPM_DB_MYSQL_PORT=3306
|
||||||
|
NPM_DB_MYSQL_USER=npm
|
||||||
|
NPM_DB_MYSQL_PASSWORD=replace-me
|
||||||
|
NPM_DB_MYSQL_NAME=npm
|
||||||
|
|
||||||
|
MYSQL_ROOT_PASSWORD=replace-me
|
||||||
|
MYSQL_DATABASE=npm
|
||||||
|
MYSQL_USER=npm
|
||||||
|
MYSQL_PASSWORD=replace-me
|
||||||
|
|
||||||
|
RABBITMQ_DEFAULT_USER=replace-me
|
||||||
|
RABBITMQ_DEFAULT_PASS=replace-me
|
||||||
117
temp/cloud.yml
Normal file
117
temp/cloud.yml
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
services:
|
||||||
|
node-red:
|
||||||
|
image: nodered/node-red:latest
|
||||||
|
container_name: node-red
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "1880:1880"
|
||||||
|
volumes:
|
||||||
|
- node_red_data:/data
|
||||||
|
|
||||||
|
influxdb:
|
||||||
|
image: influxdb:2.7
|
||||||
|
container_name: influxdb
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "8086:8086"
|
||||||
|
environment:
|
||||||
|
- INFLUXDB_ADMIN_USER=${INFLUXDB_ADMIN_USER}
|
||||||
|
- INFLUXDB_ADMIN_PASSWORD=${INFLUXDB_ADMIN_PASSWORD}
|
||||||
|
- INFLUXDB_BUCKET=${INFLUXDB_BUCKET}
|
||||||
|
- INFLUXDB_ORG=${INFLUXDB_ORG}
|
||||||
|
volumes:
|
||||||
|
- influxdb_data:/var/lib/influxdb2
|
||||||
|
|
||||||
|
grafana:
|
||||||
|
image: grafana/grafana:latest
|
||||||
|
container_name: grafana
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "3000:3000"
|
||||||
|
environment:
|
||||||
|
- GF_SECURITY_ADMIN_USER=${GF_SECURITY_ADMIN_USER}
|
||||||
|
- GF_SECURITY_ADMIN_PASSWORD=${GF_SECURITY_ADMIN_PASSWORD}
|
||||||
|
volumes:
|
||||||
|
- grafana_data:/var/lib/grafana
|
||||||
|
depends_on:
|
||||||
|
- influxdb
|
||||||
|
|
||||||
|
jenkins:
|
||||||
|
image: jenkins/jenkins:lts
|
||||||
|
container_name: jenkins
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "8080:8080" # Web
|
||||||
|
- "50000:50000" # Agents
|
||||||
|
volumes:
|
||||||
|
- jenkins_home:/var/jenkins_home
|
||||||
|
|
||||||
|
gitea:
|
||||||
|
image: gitea/gitea:latest
|
||||||
|
container_name: gitea
|
||||||
|
restart: always
|
||||||
|
environment:
|
||||||
|
- USER_UID=1000
|
||||||
|
- USER_GID=1000
|
||||||
|
ports:
|
||||||
|
- "3001:3000" # Webinterface (anders dan Grafana)
|
||||||
|
- "222:22" # SSH voor Git
|
||||||
|
volumes:
|
||||||
|
- gitea_data:/data
|
||||||
|
|
||||||
|
proxymanager:
|
||||||
|
image: jc21/nginx-proxy-manager:latest
|
||||||
|
container_name: proxymanager
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "80:80" # HTTP
|
||||||
|
- "443:443" # HTTPS
|
||||||
|
- "81:81" # Admin UI
|
||||||
|
environment:
|
||||||
|
DB_MYSQL_HOST: ${NPM_DB_MYSQL_HOST:-db}
|
||||||
|
DB_MYSQL_PORT: ${NPM_DB_MYSQL_PORT:-3306}
|
||||||
|
DB_MYSQL_USER: ${NPM_DB_MYSQL_USER}
|
||||||
|
DB_MYSQL_PASSWORD: ${NPM_DB_MYSQL_PASSWORD}
|
||||||
|
DB_MYSQL_NAME: ${NPM_DB_MYSQL_NAME}
|
||||||
|
volumes:
|
||||||
|
- proxymanager_data:/data
|
||||||
|
- proxymanager_letsencrypt:/etc/letsencrypt
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
depends_on:
|
||||||
|
- db
|
||||||
|
|
||||||
|
db:
|
||||||
|
image: jc21/mariadb-aria:latest
|
||||||
|
container_name: proxymanager_db
|
||||||
|
restart: always
|
||||||
|
environment:
|
||||||
|
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
|
||||||
|
MYSQL_DATABASE: ${MYSQL_DATABASE}
|
||||||
|
MYSQL_USER: ${MYSQL_USER}
|
||||||
|
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
|
||||||
|
volumes:
|
||||||
|
- proxymanager_db_data:/var/lib/mysql
|
||||||
|
|
||||||
|
rabbitmq:
|
||||||
|
image: rabbitmq:3-management
|
||||||
|
container_name: rabbitmq
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "5672:5672" # AMQP protocol voor apps
|
||||||
|
- "15672:15672" # Management webinterface
|
||||||
|
environment:
|
||||||
|
- RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
|
||||||
|
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
|
||||||
|
volumes:
|
||||||
|
- rabbitmq_data:/var/lib/rabbitmq
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
rabbitmq_data:
|
||||||
|
node_red_data:
|
||||||
|
influxdb_data:
|
||||||
|
grafana_data:
|
||||||
|
jenkins_home:
|
||||||
|
gitea_data:
|
||||||
|
proxymanager_data:
|
||||||
|
proxymanager_letsencrypt:
|
||||||
|
proxymanager_db_data:
|
||||||
Reference in New Issue
Block a user