b8cb889d87
- Archived 20 pre-refactor pages to wiki/Archive/ with standard banners:
- All 6 architecture/ pages (old _loadConfig/_setupSpecificClass internals,
pre-refactor S88 hierarchy, deployment blueprint)
- All 3 sessions/ logs (Apr-07 + Apr-13 session summaries)
- findings/open-issues-2026-03.md (issues 1-5 all resolved by refactor)
- concepts/generalfunctions-api.md (missing BaseDomain/BaseNodeAdapter)
- concepts/sources-readme.md (empty PDF placeholder, never populated)
- manuals/nodes/rotatingMachine.md + measurement.md (superseded by per-repo wikis)
- Top-level SCHEMA.md, index.md, log.md, metrics.md, overview.md,
knowledge-graph.yaml (all Apr-07 snapshot, pre-refactor)
- Kept wiki/concepts/ domain pages (ASM, PID, pump-affinity, settling, etc.)
- Kept wiki/findings/ proven results (BEP, NCog, curve-non-convexity, stability)
- Kept wiki/manuals/node-red/* (FlowFuse + Node-RED runtime docs, still current)
- Kept wiki/tools/* (utility scripts)
- Updated wiki/Archive.md index with 20 rows
- Fixed wiki/Home.md: Tier 6 was wrongly marked done; corrected to pending;
Tier 9 updated to reflect 2026-05-11 in-progress wave
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
118 lines
6.9 KiB
Markdown
118 lines
6.9 KiB
Markdown
---
|
||
title: "Session: measurement node — dispatcher bug fix + digital/MQTT mode"
|
||
created: 2026-04-13
|
||
updated: 2026-04-13
|
||
status: proven
|
||
tags: [session, measurement, smoothing, outlier, mqtt, iot]
|
||
---
|
||
|
||
> **⚠️ ARCHIVED — pre-refactor (Tier 1–4, 2026-05)**
|
||
>
|
||
> This page describes the architecture before the platform refactor.
|
||
> The current page is the per-node wiki on **[gitea.wbd-rd.nl/RnD](https://gitea.wbd-rd.nl/RnD)** or **[Home](../Home)**.
|
||
>
|
||
> Kept for historical reference only. **Do not update.**
|
||
|
||
|
||
# 2026-04-13 — measurement trial-ready + digital mode
|
||
|
||
## Scope
|
||
|
||
Honest review of the `measurement` node. Benchmark every method, reason about keeping the node agnostic across analog and digital sources, add a digital (MQTT/IoT) mode without breaking analog.
|
||
|
||
## Findings
|
||
|
||
### Silent dispatcher bug (critical)
|
||
|
||
`validateEnum` in `generalFunctions` lowercases enum values (`zScore` → `zscore`, `lowPass` → `lowpass`). But `specificClass.outlierDetection` and `specificClass.applySmoothing` compared against camelCase keys. Effect:
|
||
|
||
- 5 of 11 smoothing methods silently fell through to a no-op: `lowPass`, `highPass`, `weightedMovingAverage`, `bandPass`, `savitzkyGolay`.
|
||
- 2 of 3 outlier methods silently disabled: `zScore`, `modifiedZScore`.
|
||
- Only `mean`, `median`, `sd`, `min`, `max`, `none`, `kalman`, `iqr` (the already-lowercase ones) actually worked.
|
||
|
||
Users who picked any camelCase method from the dropdown got the raw last value or no outlier filtering, with no error. Flows deployed before this session that relied on these filters got no filtering at all.
|
||
|
||
### Test coverage was thin
|
||
|
||
Pre-session: **12 tests** — 1 for scaling, 1 for outlier toggle, 1 for event emit, 3 for example flow shape, 1 constructor, 1 routing, 1 invalid payload, 2 other. Every smoothing method beyond `mean` and every outlier method beyond a toggle-flip was untested. The dispatcher bug would have been caught immediately by per-method unit tests.
|
||
|
||
### Analog-only input shape
|
||
|
||
The node only accepted scalar `msg.payload`. MQTT / IoT devices commonly publish a single JSON blob with many readings per message. Every user wanting that pattern had to fan out into N measurement nodes — ugly, and the device's shared timestamp is lost.
|
||
|
||
## Fixes + additions
|
||
|
||
### Dispatcher normalization (`specificClass.js`)
|
||
|
||
Both `outlierDetection()` and `applySmoothing()` now lowercase the configured method and the lookup table keys. Legacy camelCase config values and normalized lowercase config values both work.
|
||
|
||
### `MeasurementContainer.isUnitCompatible` permissive short-circuit
|
||
|
||
Previously: if the unit couldn't be described by the convert module, compatibility returned false regardless of type. This blocked user-defined types like `humidity` with unit `%`. Now: when `measureMap[type]` is undefined (unknown type), accept any unit. Known types still validate strictly.
|
||
|
||
### Digital mode (new)
|
||
|
||
`config.mode.current === 'digital'` opts into a new input shape. `config.channels` declares one entry per JSON key. The new `Channel` class (`src/channel.js`) is a self-contained per-channel pipeline — outlier → offset → scaling → smoothing → min/max → constrain → emit. Analog behaviour is preserved exactly; flows built before this session work unchanged.
|
||
|
||
## Test additions
|
||
|
||
Before → after: **12 → 71** tests.
|
||
|
||
New files:
|
||
- `test/basic/smoothing-methods.basic.test.js` — every smoothing method covered, 16 tests.
|
||
- `test/basic/outlier-detection.basic.test.js` — every outlier method + toggle + fall-through, 10 tests.
|
||
- `test/basic/scaling-and-interpolation.basic.test.js` — offset / interpolateLinear / constrain / handleScaling / updateMinMaxValues / updateOutputPercent / updateOutputAbs / getOutput, 10 tests.
|
||
- `test/basic/calibration-and-stability.basic.test.js` — calibrate / isStable / evaluateRepeatability / toggleSimulation / tick / simulateInput, 11 tests.
|
||
- `test/integration/digital-mode.integration.test.js` — 12 tests covering channel build, payload dispatch, multi-channel emit, unknown keys, per-channel scaling / smoothing / outlier, empty channels, malformed entries, non-numeric values, digital-output shape.
|
||
|
||
## E2E verification (Dockerized Node-RED)
|
||
|
||
### Analog baseline — `/tmp/m_e2e_baseline.py`
|
||
|
||
Deploys `examples/basic.flow.json`, fires `{topic:"measurement", payload:42}` repeatedly. Observed port-0 output: `mAbs` climbed 0 → 2.1 → 2.8 → 3.15 → 3.36 → 4.2 across five ticks as the mean window filled with 42s (scaling 0..100 → 0..10). Tick cadence 909–1001 ms (avg 981 ms). Registration at t=0.22 s.
|
||
|
||
### Digital end-to-end — `/tmp/m_digital_e2e.py`
|
||
|
||
Deploys a single measurement node in digital mode with three channels (`temperature` / `humidity` / `pressure`) and fires two MQTT-shaped payloads.
|
||
|
||
| Tick | Channel | mAbs | totalMinSmooth | totalMaxSmooth |
|
||
|---|---|---:|---:|---:|
|
||
| after inject 1 | temperature | 22.5 | 22.5 | 22.5 |
|
||
| after inject 1 | humidity | 45 | 45 | 45 |
|
||
| after inject 1 | pressure | 1013 | 1013 | 1013 |
|
||
| after inject 2 | temperature | 24 | 22.5 | 24 |
|
||
| after inject 2 | humidity | 42.5 | 42.5 | 45 |
|
||
| after inject 2 | pressure | 1014 | 1013 | 1014 |
|
||
|
||
Mean smoothing across a window of 3 computed per-channel, the `unknown` key in the payload ignored, all three events emitted on `<type>.measured.atequipment`.
|
||
|
||
## Files changed
|
||
|
||
```
|
||
nodes/generalFunctions/src/measurements/MeasurementContainer.js # permissive unit check for user-defined types
|
||
nodes/generalFunctions/src/configs/measurement.json # mode + channels schema
|
||
|
||
nodes/measurement/src/channel.js # new per-channel pipeline class
|
||
nodes/measurement/src/specificClass.js # dispatcher fix + digital dispatch
|
||
nodes/measurement/src/nodeClass.js # mode-aware input handler + tick
|
||
nodes/measurement/measurement.html # Mode dropdown + Channels JSON + help panel
|
||
nodes/measurement/README.md # rewrite
|
||
|
||
nodes/measurement/test/basic/smoothing-methods.basic.test.js # +16 tests
|
||
nodes/measurement/test/basic/outlier-detection.basic.test.js # +10 tests
|
||
nodes/measurement/test/basic/scaling-and-interpolation.basic.test.js # +10 tests
|
||
nodes/measurement/test/basic/calibration-and-stability.basic.test.js # +11 tests
|
||
nodes/measurement/test/integration/digital-mode.integration.test.js # +12 tests
|
||
```
|
||
|
||
## Production status
|
||
|
||
Trial-ready for both modes. Supervised trial recommended for digital-mode deployments until the channels-editor UI (currently a JSON textarea) lands.
|
||
|
||
## Follow-ups
|
||
|
||
- Repeatable-row editor widget for channels.
|
||
- `validateArray.minLength=0` evaluates as falsy; pre-existing generalFunctions bug affecting this node's `channels` and also `measurement.assetRegistration.childAssets`. Harmless warn at deploy time.
|
||
- Per-channel calibration + simulation for digital mode.
|
||
- Runtime channel reconfiguration via a dedicated topic (`addChannel` / `removeChannel`).
|