EVOLV/wiki/concepts/signal-processing-sensors.md

Sensor Signal Conditioning & Data Quality

Used by: instrumentation-measurement agent, measurement node Validation: Verified against IEC 61298, sensor manufacturer literature, and signal processing references

Signal Conditioning Pipeline

Raw Signal → Scaling → Filtering → Outlier Rejection → Quality Flagging → Output

Scaling: Engineering Unit Conversion

4-20 mA Standard

value = range_min + (I - 4) / (20 - 4) · (range_max - range_min)

Where I is the measured current in mA.

Key Rules

• 0 mA = wire break (fault condition)
• < 4 mA = under-range or fault
• 4 mA = range minimum (0%)
• 20 mA = range maximum (100%)

• 20 mA = over-range or fault

• NAMUR NE43 recommends 3.8 mA and 20.5 mA as fault thresholds

Filtering Methods

Moving Average

y[k] = (1/N) · Σ x[k-i]   for i = 0 to N-1

• Simple, effective for white noise
• Introduces phase lag proportional to (N-1)/2 samples
• Good for steady-state signals, poor for fast transients

Exponential Moving Average (EMA)

y[k] = α · x[k] + (1-α) · y[k-1]

Where α = 2/(N+1) or α = Δt/(τ + Δt) for time-constant-based tuning.

• Less memory than moving average
• Equivalent to first-order low-pass filter
• τ (time constant) sets the cutoff frequency: f_c = 1/(2π·τ)

Savitzky-Golay Filter

• Fits a polynomial to a window of data points, uses the polynomial value as the filtered output
• Preserves higher-order moments (peaks, edges) better than moving average
• Configurable by window size and polynomial order
• Typical: window = 5-11 points, order = 2-3

Outlier Detection

Z-Score Method

z = |x - μ| / σ
outlier if z > threshold (typically 3.0)

• Assumes normal distribution
• Sensitive to the outliers themselves (they inflate σ)

Modified Z-Score (MAD-based)

MAD = median(|x_i - median(x)|)
modified_z = 0.6745 · (x - median(x)) / MAD
outlier if |modified_z| > threshold (typically 3.5)

• Robust to outliers (uses median instead of mean)
• Recommended for process measurements where occasional spikes are common
• 0.6745 is the 75th percentile of the standard normal distribution

IQR Method

Q1 = 25th percentile, Q3 = 75th percentile
IQR = Q3 - Q1
outlier if x < Q1 - 1.5·IQR or x > Q3 + 1.5·IQR

• Non-parametric, no distribution assumption
• Common in exploratory data analysis
• Less suitable for real-time streaming (needs window of data)

NRMSE for Drift Detection

Normalized Root Mean Square Error compares a recent measurement window against a reference window to detect sensor drift.

Calculation

RMSE = √(Σ(x_i - x_ref_i)² / N)
NRMSE = RMSE / (x_max - x_min)   or   RMSE / x_mean

Thresholds (typical for process sensors)

NRMSE Range	Quality	Action
0 – 0.05	Good	Normal operation
0.05 – 0.15	Uncertain	Flag for review, increase monitoring
0.15 – 0.30	Poor	Alarm, reduce weight in control loops
> 0.30	Bad	Remove from control, maintenance required

Reference Window Selection

• Calibration data (gold standard)
• Post-maintenance baseline
• Rolling reference from a known-good period
• Multi-sensor cross-validation

Sensor Accuracy Classes

IEC 61298 Framework

IEC 61298 "Process measurement and control devices — General methods and procedures for evaluating performance" defines standardized test methods for evaluating sensor accuracy under reference and influence conditions.

Key Performance Metrics

• Accuracy: Closeness of measured value to true value (includes systematic and random errors)
• Repeatability: Closeness of successive measurements under identical conditions
• Hysteresis: Maximum difference between upscale and downscale readings
• Linearity: Maximum deviation from a straight line between zero and span
• Deadband: Smallest change in input that produces a detectable output change

Common Accuracy Specifications

Sensor Type	Typical Accuracy	Response Time
Pressure transmitter	±0.04 – 0.1% FS	< 100 ms
Flow meter (electromagnetic)	±0.2 – 0.5% of reading	1-3 s
Temperature (RTD/Pt100)	±0.1 – 0.3°C	5-30 s (depends on housing)
Level (ultrasonic)	±0.25% FS	1-5 s
pH	±0.02 – 0.1 pH	10-60 s
Dissolved oxygen	±1-2% of reading	30-90 s (membrane)
Turbidity (nephelometric)	±2% of reading	5-15 s
Ammonia (ion-selective)	±5-10% of reading	60-180 s

Sensor States and Warmup

State Machine

Maintenance → Warmup → Active → Cooldown → Maintenance

Warmup Behavior

• Duration: Varies by sensor type (seconds for pressure, minutes for pH, hours for dissolved oxygen)
• During warmup: Measurements flagged as "uncertain" quality
• Completion criterion: Readings stabilize within defined tolerance for a minimum duration
• EVOLV convention: Warmup state prevents measurements from propagating to control loops

Stabilization Detection

stable if std_dev(last_N_readings) < threshold for T_stable seconds

Authoritative References

1. IEC 61298 series (2008). "Process measurement and control devices — General methods and procedures for evaluating performance"
2. IEC 61326-2-3. "Electrical equipment for measurement, control and laboratory use — EMC requirements — Part 2-3: Particular requirements — Transducers with integrated or remote signal conditioning"
3. NAMUR NE43 (2003). "Standardization of the Signal Level for the Failure Information of Digital Transmitters"
4. Bently Nevada / Baker Hughes. "Fundamentals of Rotating Machinery Diagnostics"
5. Oppenheim, A.V. & Willsky, A.S. (1997). "Signals & Systems" 2nd ed., Prentice Hall
6. Press, W.H. et al. (2007). "Numerical Recipes" 3rd ed., Chapter 14 (Savitzky-Golay filters)