Migrate to new Gitea instance (gitea.wbd-rd.nl)
- Update all submodule URLs from gitea.centraal.wbd-rd.nl to gitea.wbd-rd.nl - Add settler as proper submodule in .gitmodules - Add agent skills, function anchors, decisions, and improvements - Add Docker configuration and scripts - Add manuals and third_party docs - Update .gitignore with secrets and build artifacts - Remove stale .tgz build artifact Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
znetsixe
157
third_party/docs/signal-processing-sensors.md
vendored
Normal file
157
third_party/docs/signal-processing-sensors.md
vendored
Normal file
@@ -0,0 +1,157 @@
|
||||
# Sensor Signal Conditioning & Data Quality
|
||||
|
||||
> **Used by**: `instrumentation-measurement` agent, `measurement` node
|
||||
> **Validation**: Verified against IEC 61298, sensor manufacturer literature, and signal processing references
|
||||
|
||||
## Signal Conditioning Pipeline
|
||||
|
||||
```
|
||||
Raw Signal → Scaling → Filtering → Outlier Rejection → Quality Flagging → Output
|
||||
```
|
||||
|
||||
## Scaling: Engineering Unit Conversion
|
||||
|
||||
### 4-20 mA Standard
|
||||
```
|
||||
value = range_min + (I - 4) / (20 - 4) · (range_max - range_min)
|
||||
```
|
||||
|
||||
Where I is the measured current in mA.
|
||||
|
||||
### Key Rules
|
||||
- 0 mA = wire break (fault condition)
|
||||
- < 4 mA = under-range or fault
|
||||
- 4 mA = range minimum (0%)
|
||||
- 20 mA = range maximum (100%)
|
||||
- > 20 mA = over-range or fault
|
||||
- NAMUR NE43 recommends 3.8 mA and 20.5 mA as fault thresholds
|
||||
|
||||
## Filtering Methods
|
||||
|
||||
### Moving Average
|
||||
```
|
||||
y[k] = (1/N) · Σ x[k-i] for i = 0 to N-1
|
||||
```
|
||||
- Simple, effective for white noise
|
||||
- Introduces phase lag proportional to (N-1)/2 samples
|
||||
- Good for steady-state signals, poor for fast transients
|
||||
|
||||
### Exponential Moving Average (EMA)
|
||||
```
|
||||
y[k] = α · x[k] + (1-α) · y[k-1]
|
||||
```
|
||||
Where α = 2/(N+1) or α = Δt/(τ + Δt) for time-constant-based tuning.
|
||||
- Less memory than moving average
|
||||
- Equivalent to first-order low-pass filter
|
||||
- τ (time constant) sets the cutoff frequency: f_c = 1/(2π·τ)
|
||||
|
||||
### Savitzky-Golay Filter
|
||||
- Fits a polynomial to a window of data points, uses the polynomial value as the filtered output
|
||||
- Preserves higher-order moments (peaks, edges) better than moving average
|
||||
- Configurable by window size and polynomial order
|
||||
- Typical: window = 5-11 points, order = 2-3
|
||||
|
||||
## Outlier Detection
|
||||
|
||||
### Z-Score Method
|
||||
```
|
||||
z = |x - μ| / σ
|
||||
outlier if z > threshold (typically 3.0)
|
||||
```
|
||||
- Assumes normal distribution
|
||||
- Sensitive to the outliers themselves (they inflate σ)
|
||||
|
||||
### Modified Z-Score (MAD-based)
|
||||
```
|
||||
MAD = median(|x_i - median(x)|)
|
||||
modified_z = 0.6745 · (x - median(x)) / MAD
|
||||
outlier if |modified_z| > threshold (typically 3.5)
|
||||
```
|
||||
- Robust to outliers (uses median instead of mean)
|
||||
- **Recommended for process measurements** where occasional spikes are common
|
||||
- 0.6745 is the 75th percentile of the standard normal distribution
|
||||
|
||||
### IQR Method
|
||||
```
|
||||
Q1 = 25th percentile, Q3 = 75th percentile
|
||||
IQR = Q3 - Q1
|
||||
outlier if x < Q1 - 1.5·IQR or x > Q3 + 1.5·IQR
|
||||
```
|
||||
- Non-parametric, no distribution assumption
|
||||
- Common in exploratory data analysis
|
||||
- Less suitable for real-time streaming (needs window of data)
|
||||
|
||||
## NRMSE for Drift Detection
|
||||
|
||||
Normalized Root Mean Square Error compares a recent measurement window against a reference window to detect sensor drift.
|
||||
|
||||
### Calculation
|
||||
```
|
||||
RMSE = √(Σ(x_i - x_ref_i)² / N)
|
||||
NRMSE = RMSE / (x_max - x_min) or RMSE / x_mean
|
||||
```
|
||||
|
||||
### Thresholds (typical for process sensors)
|
||||
| NRMSE Range | Quality | Action |
|
||||
|-------------|---------|--------|
|
||||
| 0 – 0.05 | Good | Normal operation |
|
||||
| 0.05 – 0.15 | Uncertain | Flag for review, increase monitoring |
|
||||
| 0.15 – 0.30 | Poor | Alarm, reduce weight in control loops |
|
||||
| > 0.30 | Bad | Remove from control, maintenance required |
|
||||
|
||||
### Reference Window Selection
|
||||
- Calibration data (gold standard)
|
||||
- Post-maintenance baseline
|
||||
- Rolling reference from a known-good period
|
||||
- Multi-sensor cross-validation
|
||||
|
||||
## Sensor Accuracy Classes
|
||||
|
||||
### IEC 61298 Framework
|
||||
IEC 61298 "Process measurement and control devices — General methods and procedures for evaluating performance" defines standardized test methods for evaluating sensor accuracy under reference and influence conditions.
|
||||
|
||||
### Key Performance Metrics
|
||||
- **Accuracy**: Closeness of measured value to true value (includes systematic and random errors)
|
||||
- **Repeatability**: Closeness of successive measurements under identical conditions
|
||||
- **Hysteresis**: Maximum difference between upscale and downscale readings
|
||||
- **Linearity**: Maximum deviation from a straight line between zero and span
|
||||
- **Deadband**: Smallest change in input that produces a detectable output change
|
||||
|
||||
### Common Accuracy Specifications
|
||||
| Sensor Type | Typical Accuracy | Response Time |
|
||||
|-------------|-----------------|---------------|
|
||||
| Pressure transmitter | ±0.04 – 0.1% FS | < 100 ms |
|
||||
| Flow meter (electromagnetic) | ±0.2 – 0.5% of reading | 1-3 s |
|
||||
| Temperature (RTD/Pt100) | ±0.1 – 0.3°C | 5-30 s (depends on housing) |
|
||||
| Level (ultrasonic) | ±0.25% FS | 1-5 s |
|
||||
| pH | ±0.02 – 0.1 pH | 10-60 s |
|
||||
| Dissolved oxygen | ±1-2% of reading | 30-90 s (membrane) |
|
||||
| Turbidity (nephelometric) | ±2% of reading | 5-15 s |
|
||||
| Ammonia (ion-selective) | ±5-10% of reading | 60-180 s |
|
||||
|
||||
## Sensor States and Warmup
|
||||
|
||||
### State Machine
|
||||
```
|
||||
Maintenance → Warmup → Active → Cooldown → Maintenance
|
||||
```
|
||||
|
||||
### Warmup Behavior
|
||||
- **Duration**: Varies by sensor type (seconds for pressure, minutes for pH, hours for dissolved oxygen)
|
||||
- **During warmup**: Measurements flagged as "uncertain" quality
|
||||
- **Completion criterion**: Readings stabilize within defined tolerance for a minimum duration
|
||||
- **EVOLV convention**: Warmup state prevents measurements from propagating to control loops
|
||||
|
||||
### Stabilization Detection
|
||||
```
|
||||
stable if std_dev(last_N_readings) < threshold for T_stable seconds
|
||||
```
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. IEC 61298 series (2008). "Process measurement and control devices — General methods and procedures for evaluating performance"
|
||||
2. IEC 61326-2-3. "Electrical equipment for measurement, control and laboratory use — EMC requirements — Part 2-3: Particular requirements — Transducers with integrated or remote signal conditioning"
|
||||
3. NAMUR NE43 (2003). "Standardization of the Signal Level for the Failure Information of Digital Transmitters"
|
||||
4. Bently Nevada / Baker Hughes. "Fundamentals of Rotating Machinery Diagnostics"
|
||||
5. Oppenheim, A.V. & Willsky, A.S. (1997). "Signals & Systems" 2nd ed., Prentice Hall
|
||||
6. Press, W.H. et al. (2007). "Numerical Recipes" 3rd ed., Chapter 14 (Savitzky-Golay filters)
|
||||
Reference in New Issue
Block a user