0300a76ae8
Some checks failed
CI / lint-and-test (push) Has been cancelled
Bumps: - nodes/generalFunctions 75d16c6 -> e50be2e (permissive unit check + measurement schema additions) - nodes/measurement f7c3dc2 -> 495b4cf (digital mode + dispatcher fix + 59 new tests + rewritten README + UI) Wiki: - wiki/manuals/nodes/measurement.md — new user manual covering analog and digital modes, topic reference, smoothing/outlier methods, unit policy, and the pre-fix dispatcher bug advisory. - wiki/sessions/2026-04-13-measurement-digital-mode.md — session note with findings, fix scope, test additions, and dual-mode E2E results. - wiki/index.md — links both pages and adds the missing 2026-04-13 rotatingMachine session entry that was omitted from the earlier commit. Status: measurement is now trial-ready in both analog and digital modes. 71/71 unit tests green (was 12), dual-mode E2E on live Dockerized Node-RED verifies analog regression and a three-channel MQTT-style payload (temperature/humidity/pressure) dispatching independently with per-channel smoothing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
110 lines
6.6 KiB
Markdown
110 lines
6.6 KiB
Markdown
---
|
||
title: "Session: measurement node — dispatcher bug fix + digital/MQTT mode"
|
||
created: 2026-04-13
|
||
updated: 2026-04-13
|
||
status: proven
|
||
tags: [session, measurement, smoothing, outlier, mqtt, iot]
|
||
---
|
||
|
||
# 2026-04-13 — measurement trial-ready + digital mode
|
||
|
||
## Scope
|
||
|
||
Honest review of the `measurement` node. Benchmark every method, reason about keeping the node agnostic across analog and digital sources, add a digital (MQTT/IoT) mode without breaking analog.
|
||
|
||
## Findings
|
||
|
||
### Silent dispatcher bug (critical)
|
||
|
||
`validateEnum` in `generalFunctions` lowercases enum values (`zScore` → `zscore`, `lowPass` → `lowpass`). But `specificClass.outlierDetection` and `specificClass.applySmoothing` compared against camelCase keys. Effect:
|
||
|
||
- 5 of 11 smoothing methods silently fell through to a no-op: `lowPass`, `highPass`, `weightedMovingAverage`, `bandPass`, `savitzkyGolay`.
|
||
- 2 of 3 outlier methods silently disabled: `zScore`, `modifiedZScore`.
|
||
- Only `mean`, `median`, `sd`, `min`, `max`, `none`, `kalman`, `iqr` (the already-lowercase ones) actually worked.
|
||
|
||
Users who picked any camelCase method from the dropdown got the raw last value or no outlier filtering, with no error. Flows deployed before this session that relied on these filters got no filtering at all.
|
||
|
||
### Test coverage was thin
|
||
|
||
Pre-session: **12 tests** — 1 for scaling, 1 for outlier toggle, 1 for event emit, 3 for example flow shape, 1 constructor, 1 routing, 1 invalid payload, 2 other. Every smoothing method beyond `mean` and every outlier method beyond a toggle-flip was untested. The dispatcher bug would have been caught immediately by per-method unit tests.
|
||
|
||
### Analog-only input shape
|
||
|
||
The node only accepted scalar `msg.payload`. MQTT / IoT devices commonly publish a single JSON blob with many readings per message. Every user wanting that pattern had to fan out into N measurement nodes — ugly, and the device's shared timestamp is lost.
|
||
|
||
## Fixes + additions
|
||
|
||
### Dispatcher normalization (`specificClass.js`)
|
||
|
||
Both `outlierDetection()` and `applySmoothing()` now lowercase the configured method and the lookup table keys. Legacy camelCase config values and normalized lowercase config values both work.
|
||
|
||
### `MeasurementContainer.isUnitCompatible` permissive short-circuit
|
||
|
||
Previously: if the unit couldn't be described by the convert module, compatibility returned false regardless of type. This blocked user-defined types like `humidity` with unit `%`. Now: when `measureMap[type]` is undefined (unknown type), accept any unit. Known types still validate strictly.
|
||
|
||
### Digital mode (new)
|
||
|
||
`config.mode.current === 'digital'` opts into a new input shape. `config.channels` declares one entry per JSON key. The new `Channel` class (`src/channel.js`) is a self-contained per-channel pipeline — outlier → offset → scaling → smoothing → min/max → constrain → emit. Analog behaviour is preserved exactly; flows built before this session work unchanged.
|
||
|
||
## Test additions
|
||
|
||
Before → after: **12 → 71** tests.
|
||
|
||
New files:
|
||
- `test/basic/smoothing-methods.basic.test.js` — every smoothing method covered, 16 tests.
|
||
- `test/basic/outlier-detection.basic.test.js` — every outlier method + toggle + fall-through, 10 tests.
|
||
- `test/basic/scaling-and-interpolation.basic.test.js` — offset / interpolateLinear / constrain / handleScaling / updateMinMaxValues / updateOutputPercent / updateOutputAbs / getOutput, 10 tests.
|
||
- `test/basic/calibration-and-stability.basic.test.js` — calibrate / isStable / evaluateRepeatability / toggleSimulation / tick / simulateInput, 11 tests.
|
||
- `test/integration/digital-mode.integration.test.js` — 12 tests covering channel build, payload dispatch, multi-channel emit, unknown keys, per-channel scaling / smoothing / outlier, empty channels, malformed entries, non-numeric values, digital-output shape.
|
||
|
||
## E2E verification (Dockerized Node-RED)
|
||
|
||
### Analog baseline — `/tmp/m_e2e_baseline.py`
|
||
|
||
Deploys `examples/basic.flow.json`, fires `{topic:"measurement", payload:42}` repeatedly. Observed port-0 output: `mAbs` climbed 0 → 2.1 → 2.8 → 3.15 → 3.36 → 4.2 across five ticks as the mean window filled with 42s (scaling 0..100 → 0..10). Tick cadence 909–1001 ms (avg 981 ms). Registration at t=0.22 s.
|
||
|
||
### Digital end-to-end — `/tmp/m_digital_e2e.py`
|
||
|
||
Deploys a single measurement node in digital mode with three channels (`temperature` / `humidity` / `pressure`) and fires two MQTT-shaped payloads.
|
||
|
||
| Tick | Channel | mAbs | totalMinSmooth | totalMaxSmooth |
|
||
|---|---|---:|---:|---:|
|
||
| after inject 1 | temperature | 22.5 | 22.5 | 22.5 |
|
||
| after inject 1 | humidity | 45 | 45 | 45 |
|
||
| after inject 1 | pressure | 1013 | 1013 | 1013 |
|
||
| after inject 2 | temperature | 24 | 22.5 | 24 |
|
||
| after inject 2 | humidity | 42.5 | 42.5 | 45 |
|
||
| after inject 2 | pressure | 1014 | 1013 | 1014 |
|
||
|
||
Mean smoothing across a window of 3 computed per-channel, the `unknown` key in the payload ignored, all three events emitted on `<type>.measured.atequipment`.
|
||
|
||
## Files changed
|
||
|
||
```
|
||
nodes/generalFunctions/src/measurements/MeasurementContainer.js # permissive unit check for user-defined types
|
||
nodes/generalFunctions/src/configs/measurement.json # mode + channels schema
|
||
|
||
nodes/measurement/src/channel.js # new per-channel pipeline class
|
||
nodes/measurement/src/specificClass.js # dispatcher fix + digital dispatch
|
||
nodes/measurement/src/nodeClass.js # mode-aware input handler + tick
|
||
nodes/measurement/measurement.html # Mode dropdown + Channels JSON + help panel
|
||
nodes/measurement/README.md # rewrite
|
||
|
||
nodes/measurement/test/basic/smoothing-methods.basic.test.js # +16 tests
|
||
nodes/measurement/test/basic/outlier-detection.basic.test.js # +10 tests
|
||
nodes/measurement/test/basic/scaling-and-interpolation.basic.test.js # +10 tests
|
||
nodes/measurement/test/basic/calibration-and-stability.basic.test.js # +11 tests
|
||
nodes/measurement/test/integration/digital-mode.integration.test.js # +12 tests
|
||
```
|
||
|
||
## Production status
|
||
|
||
Trial-ready for both modes. Supervised trial recommended for digital-mode deployments until the channels-editor UI (currently a JSON textarea) lands.
|
||
|
||
## Follow-ups
|
||
|
||
- Repeatable-row editor widget for channels.
|
||
- `validateArray.minLength=0` evaluates as falsy; pre-existing generalFunctions bug affecting this node's `channels` and also `measurement.assetRegistration.childAssets`. Harmless warn at deploy time.
|
||
- Per-channel calibration + simulation for digital mode.
|
||
- Runtime channel reconfiguration via a dedicated topic (`addChannel` / `removeChannel`).
|