RnD/measurement
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: digital (MQTT) mode + fix silent dispatcher bug for camelCase methods

Browse Source 
Runtime:
- Fix silent no-op when user selected any camelCase smoothing or outlier
  method from the editor. validateEnum in generalFunctions lowercases enum
  values (zScore -> zscore, lowPass -> lowpass, ...) but the dispatcher
  compared against camelCase keys. Effect: 5 of 11 smoothing methods
  (lowPass, highPass, weightedMovingAverage, bandPass, savitzkyGolay) and
  2 of 3 outlier methods (zScore, modifiedZScore) silently fell through.
  Users got the raw last value or no outlier filtering with no error log.
  Review any pre-2026-04-13 flows that relied on these methods.
  Fix: normalize method names to lowercase on both sides of the lookup.

- New Channel class (src/channel.js) — self-contained per-channel pipeline:
  outlier -> offset -> scaling -> smoothing -> min/max -> constrain -> emit.
  Pure domain logic, no Node-RED deps, reusable by future nodes that need
  the same signal-conditioning chain.

Digital mode:
- config.mode.current = 'digital' opts in. config.channels declares one
  entry per expected JSON key; each channel has its own type, position,
  unit, distance, and optional scaling/smoothing/outlierDetection blocks
  that override the top-level analog-mode fields. One MQTT-shaped payload
  ({t:22.5, h:45, p:1013}) dispatches N independent pipelines and emits N
  MeasurementContainer slots from a single input message.
- Backward compatible: absent mode config = analog = pre-digital behaviour.
  Every existing measurement flow keeps working unchanged.

UI:
- HTML editor: new Mode dropdown and Channels JSON textarea. The Node-RED
  help panel is rewritten end-to-end with topic reference, port contracts,
  per-mode configuration, smoothing/outlier method tables, and a note
  about the pre-fix behaviour.
- README.md rewritten (was a one-line stub).

Tests (12 -> 71, all green):
- test/basic/smoothing-methods.basic.test.js (+16): every smoothing method
  including the formerly-broken camelCase ones.
- test/basic/outlier-detection.basic.test.js (+10): every outlier method,
  fall-through, toggle.
- test/basic/scaling-and-interpolation.basic.test.js (+10): offset,
  interpolateLinear, constrain, handleScaling edge cases, min/max
  tracking, updateOutputPercent fallback, updateOutputAbs emit dedup.
- test/basic/calibration-and-stability.basic.test.js (+11): calibrate
  (stable and unstable), isStable, evaluateRepeatability refusals,
  toggleSimulation, tick simulation on/off.
- test/integration/digital-mode.integration.test.js (+12): channel build
  (including malformed entries), payload dispatch, multi-channel emit,
  unknown keys, per-channel scaling/smoothing/outlier, empty channels,
  non-numeric value rejection, getDigitalOutput shape, analog-default
  back-compat.

E2E verified on Dockerized Node-RED: analog regression unchanged; digital
mode deploys with three channels, dispatches MQTT-style payload, emits
per-channel events, accumulates per-channel smoothing, ignores unknown
keys.

Depends on generalFunctions commit e50be2e (permissive unit check +
mode/channels schema).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
znetsixe
2026-04-13 13:43:03 +02:00
parent 0918be7705
commit 495b4cf400
10 changed files with 1367 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

119
README.md
View File

@@ -1,3 +1,118 @@
# convert
# measurement
Makes unit conversions
Node-RED custom node for sensor signal conditioning. Takes raw input — either a single scalar (analog mode) or an MQTT-style JSON object with many keys (digital mode) — and produces scaled, smoothed, outlier-filtered measurements. Part of the [EVOLV](https://gitea.wbd-rd.nl/RnD/EVOLV) wastewater-automation platform.
Registers itself on port 2 as a child of a parent equipment (rotatingMachine, pumpingStation, reactor, etc.). The parent consumes measurements via shared `MeasurementContainer` events.
## Install
```bash
cd ~/.node-red
npm install github:gitea.wbd-rd.nl/RnD/measurement
```
Or pull the whole platform via the superproject. Restart Node-RED and the node appears in the palette under **EVOLV**.
## Two input modes
### Analog mode (default)
One scalar per message — the classic PLC / 4-20mA pattern.
```json
{ "topic": "measurement", "payload": 42 }
```
The node runs one offset → scaling → smoothing → outlier pipeline and emits exactly one MeasurementContainer slot. Every existing flow built before digital mode keeps working unchanged.
### Digital mode (MQTT / IoT)
One object per message, many keys:
```json
{ "topic": "measurement",
"payload": { "temperature": 22.5, "humidity": 45, "pressure": 1013 } }
```
Each key maps to its own **channel** with independently-configured scaling, smoothing, outlier detection, type, position, unit, and distance. A single inbound message therefore emits N MeasurementContainer slots — one per channel — so a downstream parent sees everything at once.
Pick the mode in the editor or via `msg.mode`. Analog is the default; digital requires populating `channels` (see *Configuration*).
## Input topics
| Topic | Payload | Effect |
|---|---|---|
| `measurement` | analog mode: `number` or numeric `string` — stored as `inputValue` and consumed on the next tick. digital mode: `object` keyed by channel names. | drives the pipeline |
| `simulator` | — | toggles the simulator flag |
| `outlierDetection` | — | toggles outlier detection |
| `calibrate` | — | adjust the scaling offset so current output matches `inputMin` (scaling on) or `absMin` (scaling off). Requires a stable window. |
## Output ports
| Port | Label | Payload |
|---|---|---|
| 0 | `process` | analog: `{mAbs, mPercent, totalMinValue, totalMaxValue, totalMinSmooth, totalMaxSmooth}`. digital: `{channels: {<key>: {mAbs, mPercent, ...}}}`. Delta-compressed — only changed fields emit each tick. |
| 1 | `dbase` | InfluxDB line-protocol telemetry |
| 2 | `parent` | `{topic:"registerChild", payload:<nodeId>, positionVsParent, distance}` emitted once ~180ms after deploy |
## Configuration
### Common (both modes)
- **Asset** (menu): supplier, category, `assetType` (measurement type in the container — `pressure`, `flow`, `temperature`, `power`, or any user-defined type like `humidity`), model, unit.
- **Logger** (menu): log level + enable flag.
- **Position** (menu): `upstream` / `atEquipment` / `downstream` relative to parent; optional distance offset.
### Analog-mode fields
| Field | Purpose |
|---|---|
| `Scaling` (checkbox) | enables linear source→process interpolation |
| `Source Min / Max` | input-side range (e.g. 4–20 mA) |
| `Input Offset` | additive bias applied before scaling |
| `Process Min / Max` | output-side range (e.g. 0–3000 mbar) |
| `Simulator` (checkbox) | internal random-walk source |
| `Smoothing` | one of: `none`, `mean`, `min`, `max`, `sd`, `lowPass`, `highPass`, `weightedMovingAverage`, `bandPass`, `median`, `kalman`, `savitzkyGolay` |
| `Window` | sample count for the smoothing window |
### Digital-mode fields
- **Mode**: set to `digital`.
- **Channels**: JSON array, one entry per channel. Each entry:
```json
{
"key": "temperature",
"type": "temperature",
"position": "atEquipment",
"unit": "C",
"scaling": { "enabled": false, "inputMin": 0, "inputMax": 1, "absMin": -50, "absMax": 150, "offset": 0 },
"smoothing": { "smoothWindow": 5, "smoothMethod": "mean" },
"outlierDetection": { "enabled": true, "method": "zScore", "threshold": 3 }
}
```
`scaling`, `smoothing`, `outlierDetection` are optional — the node falls back to the top-level analog-mode equivalents when missing. `key` is the JSON field name inside `msg.payload`; `type` is the MeasurementContainer axis (can be any string — unknown types are accepted).
## State and emit contract
Every channel runs the same pipeline: `outlier → offset → scaling → smoothing → min/max tracking → constrain → emit`. Output is rounded to two decimals. MeasurementContainer events follow the pattern `<type>.<variant>.<position>` all lowercase, e.g. `temperature.measured.atequipment`.
Unknown measurement types (anything not in the container's built-in measureMap — `pressure`, `flow`, `power`, `temperature`, `volume`, `length`, `mass`, `energy`) are accepted without unit compatibility checks. Known types still validate strictly.
## Testing
```bash
cd nodes/measurement
npm test
```
71 tests cover every smoothing method, every outlier strategy, scaling, interpolation, constrain, calibration, stability, simulation, output-percent fallback, per-channel pipelines, digital payload dispatch, registration events, and example-flow shape.
## Production status
Last reviewed **2026-04-13**. See the project memory file `node_measurement.md` for the current verdict, benchmarks, and wishlist.
## License
SEE LICENSE. Author: Rene De Ren, Waterschap Brabantse Delta R&D.