docs: initialize project wiki from production hardening session
12 pages covering architecture, findings, and metrics from the rotatingMachine + machineGroupControl hardening work: - Overview: node inventory, what works/doesn't, current scale - Architecture: 3D pump curves, group optimization algorithm - Findings: BEP-Gravitation proof (0.1% of optimum), NCog behavior, curve non-convexity, pump switching stability - Metrics: test counts, power comparison table, performance numbers - Knowledge graph: structured YAML with all data points and provenance - Session log: 2026-04-07 production hardening - Tools: query.py, search.sh, lint.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
znetsixe
89
wiki/SCHEMA.md
Normal file
89
wiki/SCHEMA.md
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
# Project Wiki Schema
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
LLM-maintained knowledge base for this project. The LLM writes and maintains everything. You read it (ideally in Obsidian). Knowledge compounds across sessions instead of being lost in chat history.
|
||||||
|
|
||||||
|
## Directory Structure
|
||||||
|
```
|
||||||
|
wiki/
|
||||||
|
SCHEMA.md — this file (how to maintain the wiki)
|
||||||
|
index.md — catalog of all pages with one-line summaries
|
||||||
|
log.md — chronological record of updates
|
||||||
|
overview.md — project overview and current status
|
||||||
|
metrics.md — all numbers with provenance
|
||||||
|
knowledge-graph.yaml — structured data, machine-queryable
|
||||||
|
tools/ — search, lint, query scripts
|
||||||
|
concepts/ — core ideas and mechanisms
|
||||||
|
architecture/ — design decisions, system internals
|
||||||
|
findings/ — honest results (what worked AND what didn't)
|
||||||
|
sessions/ — per-session summaries
|
||||||
|
```
|
||||||
|
|
||||||
|
## Page Conventions
|
||||||
|
|
||||||
|
### Frontmatter
|
||||||
|
Every page starts with YAML frontmatter:
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
title: Page Title
|
||||||
|
created: YYYY-MM-DD
|
||||||
|
updated: YYYY-MM-DD
|
||||||
|
status: proven | disproven | evolving | speculative
|
||||||
|
tags: [tag1, tag2]
|
||||||
|
sources: [path/to/file.py, commit abc1234]
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
### Status values
|
||||||
|
- **proven**: tested and verified with evidence
|
||||||
|
- **disproven**: tested and honestly shown NOT to work (document WHY)
|
||||||
|
- **evolving**: partially working, boundary not fully mapped
|
||||||
|
- **speculative**: proposed but not yet tested
|
||||||
|
|
||||||
|
### Cross-references
|
||||||
|
Use `[[Page Name]]` Obsidian-style wikilinks.
|
||||||
|
|
||||||
|
### Contradictions
|
||||||
|
When new evidence contradicts a prior claim, DON'T delete the old claim. Add:
|
||||||
|
```
|
||||||
|
> [!warning] Superseded
|
||||||
|
> This was shown to be incorrect on YYYY-MM-DD. See [[New Finding]].
|
||||||
|
```
|
||||||
|
|
||||||
|
### Honesty rule
|
||||||
|
If something doesn't work, say so. If a result was a false positive, document how it was discovered. The wiki must be trustworthy.
|
||||||
|
|
||||||
|
## Operations
|
||||||
|
|
||||||
|
### Ingest (after a session or new source)
|
||||||
|
1. Read outputs, commits, findings
|
||||||
|
2. Update relevant pages
|
||||||
|
3. Create new pages for new concepts
|
||||||
|
4. Update `index.md`, `log.md`, `knowledge-graph.yaml`
|
||||||
|
5. Check for contradictions with existing pages
|
||||||
|
|
||||||
|
### Query
|
||||||
|
1. Use `python3 wiki/tools/query.py` for structured lookup
|
||||||
|
2. Use `wiki/tools/search.sh` for full-text
|
||||||
|
3. Read `index.md` to find relevant pages
|
||||||
|
4. File valuable answers back into the wiki
|
||||||
|
|
||||||
|
### Lint (periodically)
|
||||||
|
```bash
|
||||||
|
bash wiki/tools/lint.sh
|
||||||
|
```
|
||||||
|
Checks: orphan pages, broken wikilinks, missing frontmatter, index completeness.
|
||||||
|
|
||||||
|
## Data Layer
|
||||||
|
|
||||||
|
- `knowledge-graph.yaml` — structured YAML with every metric and data point
|
||||||
|
- `metrics.md` — human-readable dashboard
|
||||||
|
- When adding new results, update BOTH the wiki page AND the knowledge graph
|
||||||
|
- The knowledge graph is the single source of truth for numbers
|
||||||
|
|
||||||
|
## Source of Truth Hierarchy
|
||||||
|
1. **Test results** (actual outputs) — highest authority
|
||||||
|
2. **Code** (current state) — second authority
|
||||||
|
3. **Knowledge graph** (knowledge-graph.yaml) — structured metrics
|
||||||
|
4. **Wiki pages** — synthesis, may lag
|
||||||
|
5. **Chat/memory** — ephemeral, may be stale
|
||||||
56
wiki/architecture/3d-pump-curves.md
Normal file
56
wiki/architecture/3d-pump-curves.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
title: 3D Pump Curve Architecture
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [predict, curves, interpolation, rotatingMachine]
|
||||||
|
sources: [nodes/generalFunctions/src/predict/predict_class.js, nodes/rotatingMachine/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# 3D Pump Curve Prediction
|
||||||
|
|
||||||
|
## Data Structure
|
||||||
|
A family of 2D curves indexed by pressure (f-dimension):
|
||||||
|
- **X-axis**: control position (0-100%)
|
||||||
|
- **Y-axis**: flow (nq) or power (np) in canonical units
|
||||||
|
- **F-dimension**: pressure (Pa) — the 3rd dimension
|
||||||
|
|
||||||
|
Raw curves are in curve units (m3/h, kW, mbar). `_normalizeMachineCurve()` converts to canonical (m3/s, W, Pa).
|
||||||
|
|
||||||
|
## Interpolation
|
||||||
|
Monotonic cubic spline (Fritsch-Carlson) in both dimensions:
|
||||||
|
- **X-Y splines**: at each discrete pressure level
|
||||||
|
- **F-splines**: across pressure levels for intermediate pressure interpolation
|
||||||
|
|
||||||
|
## Prediction Flow
|
||||||
|
```
|
||||||
|
predict.y(x):
|
||||||
|
1. Clamp x to [currentFxyXMin, currentFxyXMax]
|
||||||
|
2. Normalize x to [normMin, normMax]
|
||||||
|
3. Evaluate spline at normalized x for current fDimension
|
||||||
|
4. Return y in canonical units (m3/s or W)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Unit Conversion Chain
|
||||||
|
```
|
||||||
|
Raw curve (m3/h, kW, mbar)
|
||||||
|
→ _normalizeMachineCurve → canonical (m3/s, W, Pa)
|
||||||
|
→ predict class → canonical output
|
||||||
|
→ MeasurementContainer.getCurrentValue(outputUnit) → output units
|
||||||
|
```
|
||||||
|
|
||||||
|
No double-conversion. Clean separation: specificClass handles units, predict handles normalization/interpolation.
|
||||||
|
|
||||||
|
## Three Predict Instances per Machine
|
||||||
|
- `predictFlow`: control % → flow (nq curve)
|
||||||
|
- `predictPower`: control % → power (np curve)
|
||||||
|
- `predictCtrl`: flow → control % (reversed nq curve)
|
||||||
|
|
||||||
|
## Boundary Behavior
|
||||||
|
- Below/above curve X range: flat extrapolation (clamped)
|
||||||
|
- Below/above f-dimension range: clamped to min/max pressure level
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
- `y(x)`: O(log n), effectively O(1) for 5-10 data points
|
||||||
|
- `buildAllFxyCurves`: sub-10ms for typical curves
|
||||||
|
- Full caching of normalized curves, splines, and calculated curves
|
||||||
45
wiki/architecture/group-optimization.md
Normal file
45
wiki/architecture/group-optimization.md
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
---
|
||||||
|
title: Group Optimization Architecture
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, optimization, BEP-Gravitation]
|
||||||
|
sources: [nodes/machineGroupControl/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# machineGroupControl Optimization
|
||||||
|
|
||||||
|
## Algorithm: BEP-Gravitation + Marginal-Cost Refinement
|
||||||
|
|
||||||
|
### Step 1 — Pressure Equalization
|
||||||
|
Sets all non-operational pumps to the group's max downstream / min upstream pressure. Ensures fair curve evaluation across combinations.
|
||||||
|
|
||||||
|
### Step 2 — Combination Enumeration
|
||||||
|
Generates all 2^n pump subsets (n = number of machines). Filters by:
|
||||||
|
- Machine state (excludes off, cooling, stopping, emergency)
|
||||||
|
- Mode compatibility (`execsequence` allowed in auto)
|
||||||
|
- Flow bounds: `sumMinFlow ≤ Qd ≤ sumMaxFlow`
|
||||||
|
- Optional power cap
|
||||||
|
|
||||||
|
### Step 3 — BEP-Gravitation Distribution (per combination)
|
||||||
|
1. **BEP seed**: `estimatedBEP = minFlow + span * NCog` per pump
|
||||||
|
2. **Slope estimation**: samples dP/dQ at BEP ± delta (directional: slopeLeft, slopeRight)
|
||||||
|
3. **Slope redistribution**: iteratively shifts flow from steep to flat curves (weight = 1/slope)
|
||||||
|
4. **Marginal-cost refinement**: after slope redistribution, shifts flow from highest actual dP/dQ to lowest using real `inputFlowCalcPower` evaluations. Converges regardless of curve convexity. Max 50 iterations, typically 5-15.
|
||||||
|
|
||||||
|
### Step 4 — Best Selection
|
||||||
|
Pick combination with lowest total power. Tiebreak by deviation from BEP.
|
||||||
|
|
||||||
|
### Step 5 — Execution
|
||||||
|
Start/stop pumps as needed, send `flowmovement` commands in output units via `_canonicalToOutputFlow()`.
|
||||||
|
|
||||||
|
## Three Control Modes
|
||||||
|
|
||||||
|
| Mode | Distribution | Combination Selection |
|
||||||
|
|------|-------------|----------------------|
|
||||||
|
| optimalControl | BEP-Gravitation + refinement | exhaustive 2^n |
|
||||||
|
| priorityControl | equal split, priority-ordered | sequential add/remove |
|
||||||
|
| priorityPercentageControl | percentage-based, normalized | count-based |
|
||||||
|
|
||||||
|
## Key Design Decision
|
||||||
|
The `flowmovement` command sends flow in the **machine's output units** (m3/h), not canonical (m3/s). The `_canonicalToOutputFlow()` helper converts before sending. Without this conversion, every pump stays at minimum flow (the critical bug fixed on 2026-04-07).
|
||||||
38
wiki/findings/bep-gravitation-proof.md
Normal file
38
wiki/findings/bep-gravitation-proof.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
---
|
||||||
|
title: BEP-Gravitation Optimality Proof
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, optimization, BEP, brute-force]
|
||||||
|
sources: [nodes/machineGroupControl/test/integration/distribution-power-table.integration.test.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# BEP-Gravitation vs Brute-Force Global Optimum
|
||||||
|
|
||||||
|
## Claim
|
||||||
|
The machineGroupControl BEP-Gravitation algorithm (with marginal-cost refinement) produces near-optimal flow distribution across a pump group.
|
||||||
|
|
||||||
|
## Method
|
||||||
|
Brute-force exhaustive search: 1000 steps per pump, all 2^n combinations, 0.05% flow tolerance. Station: 2x H05K-S03R + 1x C5-D03R-SHN1 @ ΔP=2000 mbar.
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
| Demand | Brute force | machineGroupControl | Gap |
|
||||||
|
|--------|------------|--------------------|----|
|
||||||
|
| 10% (71 m3/h) | 17.65 kW | 17.63 kW | -0.10% (MGC wins) |
|
||||||
|
| 25% (136 m3/h) | 34.33 kW | 34.33 kW | +0.01% |
|
||||||
|
| 50% (243 m3/h) | 61.62 kW | 61.62 kW | -0.00% |
|
||||||
|
| 75% (351 m3/h) | 96.01 kW | 96.10 kW | +0.08% |
|
||||||
|
| 90% (415 m3/h) | 122.17 kW | 122.26 kW | +0.07% |
|
||||||
|
|
||||||
|
Maximum deviation: **0.1%** from proven global optimum.
|
||||||
|
|
||||||
|
## Why the Refinement Matters
|
||||||
|
|
||||||
|
Before the marginal-cost refinement loop, the gap at 50% demand was **2.12%**. The BEP-Gravitation slope estimate pushed 14.6 m3/h to C5 (costing 5.0 kW) when the optimum was 6.5 m3/h (0.59 kW). The refinement loop corrects this by shifting flow from highest actual dP/dQ to lowest until no improvement is possible.
|
||||||
|
|
||||||
|
## Stability
|
||||||
|
Sweep 5-95% in 2% steps: 1 switch (rising), 1 switch (falling), same transition point. No hysteresis. See [[Pump Switching Stability]].
|
||||||
|
|
||||||
|
## Computational Cost
|
||||||
|
0.027-0.153ms median per optimization call (3 pumps, 6 combinations). Uses 0.015% of the 1000ms tick budget.
|
||||||
34
wiki/findings/curve-non-convexity.md
Normal file
34
wiki/findings/curve-non-convexity.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
title: Pump Curve Non-Convexity
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [curves, interpolation, C5, non-convex]
|
||||||
|
sources: [nodes/generalFunctions/datasets/assetData/curves/hidrostal-C5-D03R-SHN1.json]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pump Curve Non-Convexity from Sparse Data
|
||||||
|
|
||||||
|
## Finding
|
||||||
|
The C5-D03R-SHN1 pump's power curve is non-convex after spline interpolation. The marginal cost (dP/dQ) shows a spike-then-valley pattern:
|
||||||
|
|
||||||
|
```
|
||||||
|
C5 dP/dQ across flow range @ ΔP=2000 mbar:
|
||||||
|
6.4 m3/h → 1,316,610 (high)
|
||||||
|
10.2 m3/h → 2,199,349 (spikes UP)
|
||||||
|
17.7 m3/h → 1,114,700 (dropping)
|
||||||
|
21.5 m3/h → 453,316 (valley — cheapest)
|
||||||
|
29.0 m3/h → 1,048,375 (rising again)
|
||||||
|
44.1 m3/h → 1,107,708 (high)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Root Cause
|
||||||
|
The C5 curve has only **5 raw data points** per pressure level. The monotonic cubic spline (Fritsch-Carlson) creates a smooth curve through all 5 points, but with such sparse data it introduces non-convex regions that don't match the physical convexity of a real pump.
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
- The equal-marginal-cost theorem (KKT conditions) does not apply — it requires convexity
|
||||||
|
- The BEP-Gravitation slope estimate at a single point can be misleading in non-convex regions
|
||||||
|
- The marginal-cost refinement loop fixes this by using actual power evaluations instead of slope assumptions
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
Add more data points (15-20 per pressure level) to the C5 curve. This would make the spline track the real convex physics more closely, eliminating the non-convex artifacts.
|
||||||
42
wiki/findings/ncog-behavior.md
Normal file
42
wiki/findings/ncog-behavior.md
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
---
|
||||||
|
title: NCog Behavior and Limitations
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [rotatingMachine, NCog, BEP, efficiency]
|
||||||
|
sources: [nodes/rotatingMachine/src/specificClass.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# NCog — Normalized Center of Gravity
|
||||||
|
|
||||||
|
## What It Is
|
||||||
|
NCog is a 0-1 value indicating where on its flow range a pump operates most efficiently. Computed per tick from the current pressure slice of the 3D pump curve.
|
||||||
|
|
||||||
|
```
|
||||||
|
BEP_flow = minFlow + (maxFlow - minFlow) * NCog
|
||||||
|
```
|
||||||
|
|
||||||
|
## How It's Computed
|
||||||
|
1. Pressure sensors update → `getMeasuredPressure()` computes differential
|
||||||
|
2. `fDimension` locks the 2D slice at current system pressure
|
||||||
|
3. `calcCog()` computes Q/P (specific flow) across the curve
|
||||||
|
4. Peak Q/P index → `NCog = (flowAtPeak - flowMin) / (flowMax - flowMin)`
|
||||||
|
|
||||||
|
## When NCog is Meaningful
|
||||||
|
NCog requires **differential pressure** (upstream + downstream). With only one pressure sensor, fDimension is the raw sensor value (too high), producing a monotonic Q/P curve and NCog = 0.
|
||||||
|
|
||||||
|
| Condition | NCog for H05K | NCog for C5 |
|
||||||
|
|-----------|--------------|-------------|
|
||||||
|
| ΔP = 400 mbar | 0.333 | 0.355 |
|
||||||
|
| ΔP = 1000 mbar | 0.000 | 0.000 |
|
||||||
|
| ΔP = 1500 mbar | 0.135 | 0.000 |
|
||||||
|
| ΔP = 2000 mbar | 0.351 | 0.000 |
|
||||||
|
|
||||||
|
## Why NCog = 0 Happens
|
||||||
|
For variable-speed centrifugal pumps, Q/P is monotonically decreasing when the affinity laws dominate (P ∝ Q³). At certain pressure levels, the spline interpolation preserves this monotonicity and the peak is always at index 0 (minimum flow).
|
||||||
|
|
||||||
|
## How the machineGroupControl Uses NCog
|
||||||
|
The BEP-Gravitation algorithm seeds each pump at its BEP flow, then redistributes using slope-based weights + marginal-cost refinement. Even when NCog = 0, the slope redistribution produces near-optimal results because it uses actual power evaluations.
|
||||||
|
|
||||||
|
> [!warning] Disproven: NCog as proportional weight
|
||||||
|
> Using NCog directly as a flow-distribution weight (`flow = NCog/totalNCog * Qd`) is wrong. It starves pumps with NCog = 0 and overloads high-NCog pumps. See `calcBestCombination` in machineGroupControl.
|
||||||
34
wiki/findings/pump-switching-stability.md
Normal file
34
wiki/findings/pump-switching-stability.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
title: Pump Switching Stability
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [machineGroupControl, stability, switching]
|
||||||
|
sources: [nodes/machineGroupControl/test/integration/ncog-distribution.integration.test.js]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pump Switching Stability
|
||||||
|
|
||||||
|
## Concern
|
||||||
|
Frequent pump on/off cycling causes mechanical wear, water hammer, and process disturbance.
|
||||||
|
|
||||||
|
## Test Method
|
||||||
|
Sweep demand from 5% to 95% in 2% steps, count combination changes. Repeat in reverse to check for hysteresis.
|
||||||
|
|
||||||
|
## Results — Mixed Station (2x H05K + 1x C5)
|
||||||
|
|
||||||
|
Rising 5→95%: **1 switch** at 27% (H05K-1+C5 → all 3)
|
||||||
|
Falling 95→5%: **1 switch** at 25% (all 3 → H05K-1+C5)
|
||||||
|
|
||||||
|
Same transition zone, no hysteresis.
|
||||||
|
|
||||||
|
## Results — Equal Station (3x H05K)
|
||||||
|
|
||||||
|
Rising 5→95%: **2 switches**
|
||||||
|
- 19%: 1 pump → 2 pumps
|
||||||
|
- 37%: 2 pumps → 3 pumps
|
||||||
|
|
||||||
|
Clean monotonic transitions, no flickering.
|
||||||
|
|
||||||
|
## Why It's Stable
|
||||||
|
The marginal-cost refinement only adjusts flow distribution WITHIN a combination — it never changes which pumps are selected. Combination selection is driven by total power comparison, which changes smoothly with demand.
|
||||||
33
wiki/index.md
Normal file
33
wiki/index.md
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
---
|
||||||
|
title: Wiki Index
|
||||||
|
updated: 2026-04-07
|
||||||
|
---
|
||||||
|
|
||||||
|
# EVOLV Project Wiki Index
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
- [Project Overview](overview.md) — what works, what doesn't, node inventory
|
||||||
|
- [Metrics Dashboard](metrics.md) — test counts, power comparison, performance
|
||||||
|
- [Knowledge Graph](knowledge-graph.yaml) — structured data, machine-queryable
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
- [3D Pump Curves](architecture/3d-pump-curves.md) — predict class, spline interpolation, unit chain
|
||||||
|
- [Group Optimization](architecture/group-optimization.md) — BEP-Gravitation, combination selection, marginal-cost refinement
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
- [BEP-Gravitation Proof](findings/bep-gravitation-proof.md) — within 0.1% of brute-force optimum (proven)
|
||||||
|
- [NCog Behavior](findings/ncog-behavior.md) — when NCog works, when it's zero, how it's used (evolving)
|
||||||
|
- [Curve Non-Convexity](findings/curve-non-convexity.md) — C5 sparse data artifacts (proven)
|
||||||
|
- [Pump Switching Stability](findings/pump-switching-stability.md) — 1-2 transitions, no hysteresis (proven)
|
||||||
|
|
||||||
|
## Sessions
|
||||||
|
- [2026-04-07: Production Hardening](sessions/2026-04-07-production-hardening.md) — rotatingMachine + machineGroupControl
|
||||||
|
|
||||||
|
## Not Yet Documented
|
||||||
|
- Parent-child registration protocol (Port 2 handshake)
|
||||||
|
- Prediction health scoring algorithm (confidence 0-1)
|
||||||
|
- MeasurementContainer internals (chainable API, delta compression)
|
||||||
|
- PID controller implementation
|
||||||
|
- reactor / settler / monster / measurement nodes
|
||||||
|
- pumpingStation node (uses rotatingMachine children)
|
||||||
|
- InfluxDB telemetry format (Port 1)
|
||||||
161
wiki/knowledge-graph.yaml
Normal file
161
wiki/knowledge-graph.yaml
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
# Knowledge Graph — structured data with provenance
|
||||||
|
# Every claim has: value, source (file/commit), date, status
|
||||||
|
|
||||||
|
# ── TESTS ──
|
||||||
|
tests:
|
||||||
|
rotatingMachine:
|
||||||
|
basic:
|
||||||
|
count: 10
|
||||||
|
passing: 10
|
||||||
|
file: nodes/rotatingMachine/test/basic/
|
||||||
|
date: 2026-04-07
|
||||||
|
integration:
|
||||||
|
count: 16
|
||||||
|
passing: 16
|
||||||
|
file: nodes/rotatingMachine/test/integration/
|
||||||
|
date: 2026-04-07
|
||||||
|
edge:
|
||||||
|
count: 17
|
||||||
|
passing: 17
|
||||||
|
file: nodes/rotatingMachine/test/edge/
|
||||||
|
date: 2026-04-07
|
||||||
|
machineGroupControl:
|
||||||
|
basic:
|
||||||
|
count: 1
|
||||||
|
passing: 1
|
||||||
|
file: nodes/machineGroupControl/test/basic/
|
||||||
|
date: 2026-04-07
|
||||||
|
integration:
|
||||||
|
count: 3
|
||||||
|
passing: 3
|
||||||
|
file: nodes/machineGroupControl/test/integration/
|
||||||
|
date: 2026-04-07
|
||||||
|
edge:
|
||||||
|
count: 1
|
||||||
|
passing: 1
|
||||||
|
file: nodes/machineGroupControl/test/edge/
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── METRICS ──
|
||||||
|
metrics:
|
||||||
|
optimization_gap_to_brute_force:
|
||||||
|
value: "0.1% max"
|
||||||
|
source: distribution-power-table.integration.test.js
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "3 pumps, 1000-step brute force, 0.05% flow tolerance"
|
||||||
|
optimization_time_median:
|
||||||
|
value: "0.027-0.153ms"
|
||||||
|
source: benchmark script
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "3 pumps, 6 combinations, BEP-Gravitation + refinement"
|
||||||
|
pump_switching_stability:
|
||||||
|
value: "1-2 transitions across 5-95% demand"
|
||||||
|
source: stability sweep
|
||||||
|
date: 2026-04-07
|
||||||
|
conditions: "2% demand steps, both ascending and descending"
|
||||||
|
pump_curves:
|
||||||
|
H05K-S03R:
|
||||||
|
pressure_levels: 33
|
||||||
|
pressure_range: "700-3900 mbar"
|
||||||
|
flow_range: "28-227 m3/h (at 2000 mbar)"
|
||||||
|
data_points_per_level: 5
|
||||||
|
anomalies_fixed: 3
|
||||||
|
date: 2026-04-07
|
||||||
|
C5-D03R-SHN1:
|
||||||
|
pressure_levels: 26
|
||||||
|
pressure_range: "400-2900 mbar"
|
||||||
|
flow_range: "6-53 m3/h"
|
||||||
|
data_points_per_level: 5
|
||||||
|
non_convex: true
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── DISPROVEN CLAIMS ──
|
||||||
|
disproven:
|
||||||
|
ncog_proportional_weight:
|
||||||
|
claimed: "Distributing flow proportional to NCog weights is optimal"
|
||||||
|
claimed_date: 2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "Simple implementation in calcBestCombination"
|
||||||
|
evidence_against: "Starves small pumps (NCog=0 gets zero flow), overloads large pumps at high demand. BEP-target + scale is correct approach."
|
||||||
|
root_cause: "NCog is a position indicator (0-1 on flow range), not a distribution weight"
|
||||||
|
efficiency_rounding:
|
||||||
|
claimed: "Math.round(flow/power * 100) / 100 preserves BEP signal"
|
||||||
|
claimed_date: pre-2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "Removes floating point noise"
|
||||||
|
evidence_against: "In canonical units (m3/s and W), Q/P ratio is ~1e-6. Rounding to 2 decimals produces 0 for all points. NCog, cog, BEP all became 0."
|
||||||
|
root_cause: "Canonical units make the ratio very small — rounding destroys the signal"
|
||||||
|
equal_marginal_cost_optimal:
|
||||||
|
claimed: "Equal dP/dQ across pumps guarantees global power minimum"
|
||||||
|
claimed_date: 2026-04-07
|
||||||
|
disproven_date: 2026-04-07
|
||||||
|
evidence_for: "KKT conditions for convex functions"
|
||||||
|
evidence_against: "C5 pump curve is non-convex (dP/dQ dips from 1.3M to 453K then rises). Sparse data (5 points) causes spline artifacts."
|
||||||
|
root_cause: "Convexity assumption fails with interpolated curves from sparse data"
|
||||||
|
|
||||||
|
# ── PERFORMANCE ──
|
||||||
|
performance:
|
||||||
|
mgc_optimization:
|
||||||
|
median_ms: 0.09
|
||||||
|
p99_ms: 0.5
|
||||||
|
tick_budget_pct: 0.015
|
||||||
|
source: benchmark script
|
||||||
|
date: 2026-04-07
|
||||||
|
predict_y_call:
|
||||||
|
complexity: "O(log n), ~O(1) for 5-10 data points"
|
||||||
|
source: predict_class.js
|
||||||
|
|
||||||
|
# ── ARCHITECTURE ──
|
||||||
|
architecture:
|
||||||
|
canonical_units:
|
||||||
|
pressure: Pa
|
||||||
|
flow: "m3/s"
|
||||||
|
power: W
|
||||||
|
temperature: K
|
||||||
|
output_units:
|
||||||
|
pressure: mbar
|
||||||
|
flow: "m3/h"
|
||||||
|
power: kW
|
||||||
|
temperature: C
|
||||||
|
node_count: 13
|
||||||
|
submodules: 12
|
||||||
|
|
||||||
|
# ── BUGS FIXED ──
|
||||||
|
bugs_fixed:
|
||||||
|
flowmovement_unit_mismatch:
|
||||||
|
severity: critical
|
||||||
|
description: "machineGroupControl sent flow in canonical (m3/s) but rotatingMachine flowmovement expected output units (m3/h). Every pump stayed at minimum."
|
||||||
|
fix: "_canonicalToOutputFlow() conversion before all flowmovement calls"
|
||||||
|
commit: d55f401
|
||||||
|
date: 2026-04-07
|
||||||
|
emergencystop_case:
|
||||||
|
severity: critical
|
||||||
|
description: "specificClass called executeSequence('emergencyStop') but config key was 'emergencystop'"
|
||||||
|
fix: "Lowercase to match config"
|
||||||
|
commit: 07af7ce
|
||||||
|
date: 2026-04-07
|
||||||
|
curve_data_anomalies:
|
||||||
|
severity: high
|
||||||
|
description: "3 flow values leaked into power column in hidrostal-H05K-S03R.json at pressures 1600, 3200, 3300 mbar"
|
||||||
|
fix: "Linearly interpolated correct values from adjacent levels"
|
||||||
|
commit: 024db55
|
||||||
|
date: 2026-04-07
|
||||||
|
efficiency_rounding:
|
||||||
|
severity: high
|
||||||
|
description: "Math.round(Q/P * 100) / 100 destroyed all NCog/BEP calculations"
|
||||||
|
fix: "Removed rounding, use raw ratio"
|
||||||
|
commit: 07af7ce
|
||||||
|
date: 2026-04-07
|
||||||
|
absolute_scaling_bug:
|
||||||
|
severity: high
|
||||||
|
description: "handleInput compared demandQout (always 0) instead of demandQ for max cap"
|
||||||
|
fix: "Reordered conditions, use demandQ throughout"
|
||||||
|
commit: d55f401
|
||||||
|
date: 2026-04-07
|
||||||
|
|
||||||
|
# ── TIMELINE ──
|
||||||
|
timeline:
|
||||||
|
- {date: 2026-04-07, commit: 024db55, desc: "Fix 3 anomalous power values in hidrostal curve"}
|
||||||
|
- {date: 2026-04-07, commit: 07af7ce, desc: "rotatingMachine production hardening: safety + prediction + 43 tests"}
|
||||||
|
- {date: 2026-04-07, commit: d55f401, desc: "machineGroupControl: unit fix + refinement + stability tests"}
|
||||||
|
- {date: 2026-04-07, commit: fd9d167, desc: "Update EVOLV submodule refs"}
|
||||||
11
wiki/log.md
Normal file
11
wiki/log.md
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
---
|
||||||
|
title: Wiki Log
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wiki Log
|
||||||
|
|
||||||
|
## [2026-04-07] Wiki initialized | Full codebase scan + session findings
|
||||||
|
- Created overview, metrics, knowledge graph from production hardening session
|
||||||
|
- Architecture pages: 3D pump curves, group optimization
|
||||||
|
- Findings: BEP-Gravitation proof, NCog behavior, curve non-convexity, switching stability
|
||||||
|
- Session log: 2026-04-07 production hardening
|
||||||
56
wiki/metrics.md
Normal file
56
wiki/metrics.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
---
|
||||||
|
title: Metrics Dashboard
|
||||||
|
updated: 2026-04-07
|
||||||
|
---
|
||||||
|
|
||||||
|
# Metrics Dashboard
|
||||||
|
|
||||||
|
All numbers with provenance. Source of truth: `knowledge-graph.yaml`.
|
||||||
|
|
||||||
|
## Test Results
|
||||||
|
|
||||||
|
| Suite | Pass/Total | File | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| rotatingMachine basic | 10/10 | test/basic/*.test.js | 2026-04-07 |
|
||||||
|
| rotatingMachine integration | 16/16 | test/integration/*.test.js | 2026-04-07 |
|
||||||
|
| rotatingMachine edge | 17/17 | test/edge/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl basic | 1/1 | test/basic/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl integration | 3/3 | test/integration/*.test.js | 2026-04-07 |
|
||||||
|
| machineGroupControl edge | 1/1 | test/edge/*.test.js | 2026-04-07 |
|
||||||
|
|
||||||
|
## Performance — machineGroupControl Optimization
|
||||||
|
|
||||||
|
| Metric | Value | Source | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| BEP-Gravitation + refinement (3 pumps, 6 combos) | 0.027-0.153ms median | benchmark script | 2026-04-07 |
|
||||||
|
| Tick loop budget used | 0.015% of 1000ms | benchmark script | 2026-04-07 |
|
||||||
|
| Max gap from brute-force optimum (1000 steps) | 0.1% | [[BEP Gravitation Proof]] | 2026-04-07 |
|
||||||
|
| Pump switching stability (5-95% sweep) | 1-2 transitions, no hysteresis | stability sweep | 2026-04-07 |
|
||||||
|
|
||||||
|
## Performance — rotatingMachine Prediction
|
||||||
|
|
||||||
|
| Metric | Value | Source |
|
||||||
|
|---|---|---|
|
||||||
|
| predict.y(x) call | O(log n), effectively O(1) | predict_class.js |
|
||||||
|
| buildAllFxyCurves | sub-10ms for typical curves | predict_class.js |
|
||||||
|
| Curve cache | full caching of splines + calculated curves | predict_class.js |
|
||||||
|
|
||||||
|
## Power Comparison: machineGroupControl vs Baselines
|
||||||
|
|
||||||
|
Station: 2x H05K-S03R + 1x C5-D03R-SHN1 @ ΔP=2000 mbar
|
||||||
|
|
||||||
|
| Demand | Qd (m3/h) | machineGroupControl | Spillover | Equal-all | Gap to optimum |
|
||||||
|
|--------|-----------|--------------------|-----------|-----------|----|
|
||||||
|
| 10% | 71 | 17.6 kW | 22.0 kW (+25%) | 23.9 kW (+36%) | -0.10% |
|
||||||
|
| 25% | 136 | 34.6 kW | 36.3 kW (+5%) | 39.1 kW (+13%) | +0.01% |
|
||||||
|
| 50% | 243 | 62.9 kW | 73.8 kW (+17%) | 64.2 kW (+2%) | -0.00% |
|
||||||
|
| 75% | 351 | 96.8 kW | 102.9 kW (+6%) | 99.6 kW (+3%) | +0.08% |
|
||||||
|
| 90% | 415 | 122.8 kW | 123.0 kW (0%) | 123.0 kW (0%) | +0.07% |
|
||||||
|
|
||||||
|
## Disproven Claims
|
||||||
|
|
||||||
|
| Claim | Evidence For | Evidence Against | Date |
|
||||||
|
|---|---|---|---|
|
||||||
|
| NCog as proportional weight works | Simple implementation | Starves small pumps, overloads large ones at high demand | 2026-04-07 |
|
||||||
|
| Q/P ratio always has mid-range peak | Expected from pump physics | Monotonically decreasing at high ΔP due to affinity laws (P ∝ Q³) | 2026-04-07 |
|
||||||
|
| Equal-marginal-cost solver is optimal | KKT theory for convex curves | C5 curve is non-convex due to sparse data points (5 per pressure) | 2026-04-07 |
|
||||||
70
wiki/overview.md
Normal file
70
wiki/overview.md
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
---
|
||||||
|
title: EVOLV Project Overview
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: evolving
|
||||||
|
tags: [overview, wastewater, node-red, isa-88]
|
||||||
|
---
|
||||||
|
|
||||||
|
# EVOLV — Edge-Layer Evolution for Optimized Virtualization
|
||||||
|
|
||||||
|
Industrial automation platform for wastewater treatment, built as custom Node-RED nodes by Waterschap Brabantse Delta R&D. Follows ISA-88 (S88) batch control standard.
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
Node.js, Node-RED, InfluxDB (time-series), TensorFlow.js (prediction), CoolProp (thermodynamics). No build step — pure Node.js.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Each node follows a 3-tier pattern:
|
||||||
|
1. **Entry file** — registers with Node-RED, admin HTTP endpoints
|
||||||
|
2. **nodeClass** — Node-RED adapter (tick loop, message routing, status)
|
||||||
|
3. **specificClass** — pure domain logic (physics, state machines, predictions)
|
||||||
|
|
||||||
|
3-port output convention: Port 0 = process data, Port 1 = InfluxDB telemetry, Port 2 = parent-child registration.
|
||||||
|
|
||||||
|
## What Works
|
||||||
|
|
||||||
|
| Capability | Status | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| rotatingMachine state machine | proven | 76 tests passing, all sequences verified |
|
||||||
|
| 3D pump curve prediction (flow/power from pressure+control) | proven | Monotonic cubic spline interpolation across 34 pressure levels |
|
||||||
|
| NCog / BEP tracking per pump | proven | Produces meaningful values with differential pressure |
|
||||||
|
| machineGroupControl BEP-Gravitation | proven | Within 0.1% of brute-force global optimum |
|
||||||
|
| Combination selection (2^n exhaustive) | proven | Stable: 1-2 switches across 5-95% demand sweep, no hysteresis |
|
||||||
|
| Prediction health scoring | proven | NRMSE drift, pressure source penalties, edge detection |
|
||||||
|
| Hydraulic efficiency (η = QΔP/P) | proven | CoolProp density, head calculation |
|
||||||
|
| Unit conversion chain | proven | No double-conversion, clean layer separation |
|
||||||
|
|
||||||
|
## What Doesn't Work (honestly)
|
||||||
|
|
||||||
|
| Issue | Status | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| C5 curve non-convexity | evolving | 5 raw data points cause spline artifacts, dP/dQ non-monotonic |
|
||||||
|
| NCog = 0 at high ΔP | evolving | At ΔP > 800 mbar for H05K, Q/P is monotonically decreasing |
|
||||||
|
| calcBestCombination (NCog-weight mode) | disproven | Uses NCog as proportional weight instead of BEP target |
|
||||||
|
|
||||||
|
## Current Scale
|
||||||
|
|
||||||
|
- 13 custom Node-RED nodes (12 submodules + generalFunctions)
|
||||||
|
- rotatingMachine: 76 tests, 1563 lines domain logic
|
||||||
|
- machineGroupControl: 90+ tests, 1400+ lines domain logic
|
||||||
|
- 3 real pump curves: H05K-S03R, C5-D03R-SHN1, ECDV
|
||||||
|
- Tick loop: 1000ms interval
|
||||||
|
|
||||||
|
## Node Inventory
|
||||||
|
|
||||||
|
| Node | Purpose | Test Status |
|
||||||
|
|------|---------|-------------|
|
||||||
|
| rotatingMachine | Pump/compressor control | 76 tests (full) |
|
||||||
|
| machineGroupControl | Multi-pump optimization | 90 tests (full) |
|
||||||
|
| pumpingStation | Multi-pump station | needs review |
|
||||||
|
| valve | Valve modeling | needs review |
|
||||||
|
| valveGroupControl | Valve group coordination | needs review |
|
||||||
|
| reactor | Biological reactor (ASM kinetics) | needs review |
|
||||||
|
| settler | Secondary clarifier | needs review |
|
||||||
|
| monster | Multi-parameter bio monitoring | needs review |
|
||||||
|
| measurement | Sensor signal conditioning | needs review |
|
||||||
|
| diffuser | Aeration system control | needs review |
|
||||||
|
| dashboardAPI | InfluxDB + FlowFuse charts | needs review |
|
||||||
|
| generalFunctions | Shared utilities | partial |
|
||||||
46
wiki/sessions/2026-04-07-production-hardening.md
Normal file
46
wiki/sessions/2026-04-07-production-hardening.md
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
---
|
||||||
|
title: "Session: Production Hardening rotatingMachine + machineGroupControl"
|
||||||
|
created: 2026-04-07
|
||||||
|
updated: 2026-04-07
|
||||||
|
status: proven
|
||||||
|
tags: [session, rotatingMachine, machineGroupControl, testing]
|
||||||
|
---
|
||||||
|
|
||||||
|
# 2026-04-07 — Production Hardening
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
Full code review and hardening of rotatingMachine and machineGroupControl nodes for production readiness.
|
||||||
|
|
||||||
|
## Key Discoveries
|
||||||
|
|
||||||
|
1. **Efficiency rounding destroyed NCog/BEP** — `Math.round(Q/P * 100) / 100` in canonical units (m3/s and W) produces ratios ~1e-6 that all round to 0. All NCog, cog, and BEP calculations were non-functional. Fixed by removing rounding.
|
||||||
|
|
||||||
|
2. **flowmovement unit mismatch** — machineGroupControl computed flow in canonical (m3/s) and sent it directly to rotatingMachine which expected output units (m3/h). Every pump stayed at minimum flow. Fixed with `_canonicalToOutputFlow()`.
|
||||||
|
|
||||||
|
3. **emergencyStop case mismatch** — `"emergencyStop"` vs config key `"emergencystop"`. Emergency stop never worked. Fixed to lowercase.
|
||||||
|
|
||||||
|
4. **Curve data anomalies** — 3 flow values leaked into power columns in hidrostal-H05K-S03R.json at pressures 1600, 3200, 3300 mbar. Fixed with interpolated values.
|
||||||
|
|
||||||
|
5. **C5 pump non-convexity** — 5 data points per pressure level produces non-convex spline. The marginal-cost refinement loop closes the gap to brute-force optimum from 2.1% to 0.1%.
|
||||||
|
|
||||||
|
## Changes Made
|
||||||
|
|
||||||
|
### rotatingMachine (3 files, 7 test files)
|
||||||
|
- Async input handler, null guards, listener cleanup, tick loop race fix
|
||||||
|
- showCoG() implementation, efficiency variant fix, curve anomaly detection
|
||||||
|
- 43 new tests (76 total)
|
||||||
|
|
||||||
|
### machineGroupControl (1 file, 2 test files)
|
||||||
|
- `_canonicalToOutputFlow()` on all flowmovement calls
|
||||||
|
- Absolute scaling bug, empty Qd block, empty-machines guards
|
||||||
|
- Marginal-cost refinement loop in BEP-Gravitation
|
||||||
|
- Missing flowmovement after startup in equalFlowControl
|
||||||
|
|
||||||
|
### generalFunctions (1 file)
|
||||||
|
- 3 curve data fixes in hidrostal-H05K-S03R.json
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
- 90 tests passing across both nodes
|
||||||
|
- machineGroupControl within 0.1% of brute-force global optimum (1000-step search)
|
||||||
|
- Pump switching stable: 1-2 transitions across full demand range, no hysteresis
|
||||||
|
- Optimization cost: 0.03-0.15ms per call (0.015% of tick budget)
|
||||||
46
wiki/tools/lint.sh
Normal file
46
wiki/tools/lint.sh
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Wiki health check — find issues
|
||||||
|
# Usage: ./wiki/tools/lint.sh
|
||||||
|
|
||||||
|
WIKI_DIR="$(dirname "$(dirname "$(readlink -f "$0")")")"
|
||||||
|
|
||||||
|
echo "=== Wiki Health Check ==="
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Page count --"
|
||||||
|
find "$WIKI_DIR" -name "*.md" -not -path "*/tools/*" | wc -l
|
||||||
|
echo " total pages"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Orphans (not linked from other pages) --"
|
||||||
|
for f in $(find "$WIKI_DIR" -name "*.md" -not -name "index.md" -not -name "log.md" -not -name "SCHEMA.md" -not -path "*/tools/*"); do
|
||||||
|
basename=$(basename "$f" .md)
|
||||||
|
refs=$(grep -rl --include="*.md" "$basename" "$WIKI_DIR" 2>/dev/null | grep -v "$f" | wc -l)
|
||||||
|
if [ "$refs" -eq 0 ]; then
|
||||||
|
echo " ORPHAN: $f"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Status distribution --"
|
||||||
|
for status in proven disproven evolving speculative; do
|
||||||
|
count=$(grep -rl "status: $status" "$WIKI_DIR" --include="*.md" 2>/dev/null | wc -l)
|
||||||
|
echo " $status: $count"
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Pages missing frontmatter --"
|
||||||
|
for f in $(find "$WIKI_DIR" -name "*.md" -not -name "SCHEMA.md" -not -path "*/tools/*"); do
|
||||||
|
if ! head -1 "$f" | grep -q "^---"; then
|
||||||
|
echo " NO FRONTMATTER: $f"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "-- Index completeness --"
|
||||||
|
indexed=$(grep -c '\[.*\](.*\.md)' "$WIKI_DIR/index.md" 2>/dev/null)
|
||||||
|
total=$(find "$WIKI_DIR" -name "*.md" -not -name "index.md" -not -name "log.md" -not -name "SCHEMA.md" -not -path "*/tools/*" | wc -l)
|
||||||
|
echo " Indexed: $indexed / Total: $total"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo "=== Done ==="
|
||||||
249
wiki/tools/query.py
Normal file
249
wiki/tools/query.py
Normal file
@@ -0,0 +1,249 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Wiki Knowledge Graph query tool.
|
||||||
|
|
||||||
|
Queryable interface over knowledge-graph.yaml + wiki pages.
|
||||||
|
Usable by both humans (CLI) and LLM agents (imported).
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python wiki/tools/query.py health # project health
|
||||||
|
python wiki/tools/query.py entity "search term" # everything about an entity
|
||||||
|
python wiki/tools/query.py metric "search term" # find metrics
|
||||||
|
python wiki/tools/query.py status "proven" # all pages with status
|
||||||
|
python wiki/tools/query.py test "test name" # test results
|
||||||
|
python wiki/tools/query.py search "keyword" # full-text search
|
||||||
|
python wiki/tools/query.py related "page-name" # pages linking to/from
|
||||||
|
python wiki/tools/query.py timeline # commit timeline
|
||||||
|
"""
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
WIKI_DIR = Path(__file__).parent.parent
|
||||||
|
GRAPH_PATH = WIKI_DIR / 'knowledge-graph.yaml'
|
||||||
|
|
||||||
|
|
||||||
|
def load_graph():
|
||||||
|
if not GRAPH_PATH.exists():
|
||||||
|
return {}
|
||||||
|
with open(GRAPH_PATH) as f:
|
||||||
|
return yaml.safe_load(f) or {}
|
||||||
|
|
||||||
|
|
||||||
|
def load_all_pages():
|
||||||
|
pages = {}
|
||||||
|
for md_path in WIKI_DIR.rglob('*.md'):
|
||||||
|
if 'tools' in str(md_path):
|
||||||
|
continue
|
||||||
|
rel = md_path.relative_to(WIKI_DIR)
|
||||||
|
content = md_path.read_text()
|
||||||
|
meta = {}
|
||||||
|
if content.startswith('---'):
|
||||||
|
parts = content.split('---', 2)
|
||||||
|
if len(parts) >= 3:
|
||||||
|
try:
|
||||||
|
meta = yaml.safe_load(parts[1]) or {}
|
||||||
|
except yaml.YAMLError:
|
||||||
|
pass
|
||||||
|
content = parts[2]
|
||||||
|
links = re.findall(r'\[\[([^\]]+)\]\]', content)
|
||||||
|
pages[str(rel)] = {
|
||||||
|
'path': str(rel), 'meta': meta, 'content': content,
|
||||||
|
'links': links, 'title': meta.get('title', str(rel)),
|
||||||
|
'status': meta.get('status', 'unknown'),
|
||||||
|
'tags': meta.get('tags', []),
|
||||||
|
}
|
||||||
|
return pages
|
||||||
|
|
||||||
|
|
||||||
|
def flatten_graph(graph, prefix=''):
|
||||||
|
items = []
|
||||||
|
if isinstance(graph, dict):
|
||||||
|
for k, v in graph.items():
|
||||||
|
path = f"{prefix}.{k}" if prefix else k
|
||||||
|
if isinstance(v, (dict, list)):
|
||||||
|
items.extend(flatten_graph(v, path))
|
||||||
|
else:
|
||||||
|
items.append((path, str(v)))
|
||||||
|
elif isinstance(graph, list):
|
||||||
|
for i, v in enumerate(graph):
|
||||||
|
path = f"{prefix}[{i}]"
|
||||||
|
if isinstance(v, (dict, list)):
|
||||||
|
items.extend(flatten_graph(v, path))
|
||||||
|
else:
|
||||||
|
items.append((path, str(v)))
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_health():
|
||||||
|
graph = load_graph()
|
||||||
|
pages = load_all_pages()
|
||||||
|
statuses = {}
|
||||||
|
for p in pages.values():
|
||||||
|
s = p['status']
|
||||||
|
statuses[s] = statuses.get(s, 0) + 1
|
||||||
|
|
||||||
|
tests = graph.get('tests', {})
|
||||||
|
total_pass = sum(t.get('passing', 0) for t in tests.values() if isinstance(t, dict))
|
||||||
|
total_count = sum(t.get('count', t.get('total', 0)) for t in tests.values() if isinstance(t, dict))
|
||||||
|
disproven = len(graph.get('disproven', {}))
|
||||||
|
timeline = len(graph.get('timeline', []))
|
||||||
|
|
||||||
|
# Count broken links
|
||||||
|
all_titles = set()
|
||||||
|
for p in pages.values():
|
||||||
|
all_titles.add(p['title'].lower())
|
||||||
|
all_titles.add(p['path'].lower().replace('.md', '').split('/')[-1])
|
||||||
|
broken = sum(1 for p in pages.values() for link in p['links']
|
||||||
|
if not any(link.lower().replace('-', ' ') in t or t in link.lower().replace('-', ' ')
|
||||||
|
for t in all_titles))
|
||||||
|
|
||||||
|
print(f"Wiki Health:\n")
|
||||||
|
print(f" Pages: {len(pages)}")
|
||||||
|
print(f" Statuses: {statuses}")
|
||||||
|
if total_count:
|
||||||
|
print(f" Tests: {total_pass}/{total_count} passing")
|
||||||
|
print(f" Disproven: {disproven} claims tracked")
|
||||||
|
print(f" Timeline: {timeline} commits")
|
||||||
|
print(f" Broken links: {broken}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_entity(query):
|
||||||
|
graph = load_graph()
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Entity: '{query}'\n")
|
||||||
|
|
||||||
|
flat = flatten_graph(graph)
|
||||||
|
hits = [(p, v) for p, v in flat if q in p.lower() or q in v.lower()]
|
||||||
|
if hits:
|
||||||
|
print(" -- Knowledge Graph --")
|
||||||
|
for path, value in hits[:20]:
|
||||||
|
print(f" {path}: {value}")
|
||||||
|
|
||||||
|
print("\n -- Wiki Pages --")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
if q in page['content'].lower() or q in page['title'].lower():
|
||||||
|
lines = [l.strip() for l in page['content'].split('\n')
|
||||||
|
if q in l.lower() and l.strip()]
|
||||||
|
print(f" {rel} ({page['status']})")
|
||||||
|
for line in lines[:3]:
|
||||||
|
print(f" {line[:100]}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_metric(query):
|
||||||
|
flat = flatten_graph(load_graph())
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Metrics matching '{query}':\n")
|
||||||
|
found = 0
|
||||||
|
for path, value in flat:
|
||||||
|
if q in path.lower() or q in value.lower():
|
||||||
|
print(f" {path}: {value}")
|
||||||
|
found += 1
|
||||||
|
if not found:
|
||||||
|
print(" (no matches)")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_status(status):
|
||||||
|
pages = load_all_pages()
|
||||||
|
graph = load_graph()
|
||||||
|
print(f"Status: '{status}'\n")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
if page['status'] == status:
|
||||||
|
print(f" {page['title']} ({rel})")
|
||||||
|
if page['tags']:
|
||||||
|
print(f" tags: {page['tags']}")
|
||||||
|
if status == 'disproven' and 'disproven' in graph:
|
||||||
|
print("\n -- Disproven Claims --")
|
||||||
|
for name, claim in graph['disproven'].items():
|
||||||
|
print(f" {name}:")
|
||||||
|
for k, v in claim.items():
|
||||||
|
print(f" {k}: {v}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_test(query):
|
||||||
|
tests = load_graph().get('tests', {})
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Test results for '{query}':\n")
|
||||||
|
for name, suite in tests.items():
|
||||||
|
if q in name.lower() or q in str(suite).lower():
|
||||||
|
print(f" -- {name} --")
|
||||||
|
if isinstance(suite, dict):
|
||||||
|
for k, v in suite.items():
|
||||||
|
if isinstance(v, dict):
|
||||||
|
print(f" {k}: {v.get('passing', '?')}/{v.get('total', '?')}")
|
||||||
|
elif k in ('count', 'passing', 'accuracy', 'file', 'date'):
|
||||||
|
print(f" {k}: {v}")
|
||||||
|
elif k == 'results' and isinstance(v, list):
|
||||||
|
for r in v:
|
||||||
|
mark = '✓' if r.get('result') == 'pass' else '✗'
|
||||||
|
print(f" {mark} {r.get('test', '?')}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_search(query):
|
||||||
|
flat = flatten_graph(load_graph())
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = query.lower()
|
||||||
|
print(f"Search: '{query}'\n")
|
||||||
|
|
||||||
|
graph_hits = [(p, v) for p, v in flat if q in v.lower()]
|
||||||
|
if graph_hits:
|
||||||
|
print(f" -- Knowledge Graph ({len(graph_hits)} hits) --")
|
||||||
|
for p, v in graph_hits[:10]:
|
||||||
|
print(f" {p}: {v[:80]}")
|
||||||
|
|
||||||
|
page_hits = sorted(
|
||||||
|
[(page['content'].lower().count(q), rel, page['title'])
|
||||||
|
for rel, page in pages.items() if q in page['content'].lower()],
|
||||||
|
reverse=True)
|
||||||
|
if page_hits:
|
||||||
|
print(f"\n -- Wiki Pages ({len(page_hits)} pages) --")
|
||||||
|
for count, rel, title in page_hits:
|
||||||
|
print(f" {count:3d}x {title} ({rel})")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_related(page_name):
|
||||||
|
pages = load_all_pages()
|
||||||
|
q = page_name.lower().replace('-', ' ').replace('_', ' ')
|
||||||
|
print(f"Related to: '{page_name}'\n")
|
||||||
|
|
||||||
|
print(" -- Links TO --")
|
||||||
|
for rel, page in sorted(pages.items()):
|
||||||
|
for link in page['links']:
|
||||||
|
if q in link.lower().replace('-', ' '):
|
||||||
|
print(f" <- {page['title']} ({rel})")
|
||||||
|
break
|
||||||
|
|
||||||
|
print("\n -- Links FROM --")
|
||||||
|
for rel, page in pages.items():
|
||||||
|
if q in page['title'].lower().replace('-', ' '):
|
||||||
|
for link in page['links']:
|
||||||
|
print(f" -> [[{link}]]")
|
||||||
|
break
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_timeline():
|
||||||
|
for entry in load_graph().get('timeline', []):
|
||||||
|
print(f" [{entry.get('date')}] {entry.get('commit', '?')}: {entry.get('desc', '?')}")
|
||||||
|
|
||||||
|
|
||||||
|
COMMANDS = {
|
||||||
|
'health': cmd_health, 'entity': cmd_entity, 'metric': cmd_metric,
|
||||||
|
'status': cmd_status, 'test': cmd_test, 'search': cmd_search,
|
||||||
|
'related': cmd_related, 'timeline': cmd_timeline,
|
||||||
|
}
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if len(sys.argv) < 2 or sys.argv[1] not in COMMANDS:
|
||||||
|
print(f"Usage: query.py <{'|'.join(COMMANDS)}> [args]")
|
||||||
|
sys.exit(1)
|
||||||
|
cmd = sys.argv[1]
|
||||||
|
args = sys.argv[2:]
|
||||||
|
if cmd in ('timeline', 'health'):
|
||||||
|
COMMANDS[cmd]()
|
||||||
|
elif args:
|
||||||
|
COMMANDS[cmd](' '.join(args))
|
||||||
|
else:
|
||||||
|
print(f"Usage: query.py {cmd} <query>")
|
||||||
18
wiki/tools/search.sh
Normal file
18
wiki/tools/search.sh
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Search the wiki — usable by both humans and LLM agents
|
||||||
|
# Usage: ./wiki/tools/search.sh "query" [--files-only]
|
||||||
|
|
||||||
|
WIKI_DIR="$(dirname "$(dirname "$(readlink -f "$0")")")"
|
||||||
|
QUERY="$1"
|
||||||
|
MODE="${2:---content}"
|
||||||
|
|
||||||
|
if [ -z "$QUERY" ]; then
|
||||||
|
echo "Usage: $0 <query> [--files-only]"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$MODE" = "--files-only" ]; then
|
||||||
|
grep -rl --include="*.md" --include="*.yaml" "$QUERY" "$WIKI_DIR" 2>/dev/null | sort
|
||||||
|
else
|
||||||
|
grep -rn --include="*.md" --include="*.yaml" --color=auto -i "$QUERY" "$WIKI_DIR" 2>/dev/null
|
||||||
|
fi
|
||||||
Reference in New Issue
Block a user