Migrate to new Gitea instance (gitea.wbd-rd.nl)
- Update all submodule URLs from gitea.centraal.wbd-rd.nl to gitea.wbd-rd.nl - Add settler as proper submodule in .gitmodules - Add agent skills, function anchors, decisions, and improvements - Add Docker configuration and scripts - Add manuals and third_party docs - Update .gitignore with secrets and build artifacts - Remove stale .tgz build artifact Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
40
third_party/docs/README.md
vendored
Normal file
40
third_party/docs/README.md
vendored
Normal file
@@ -0,0 +1,40 @@
|
||||
# EVOLV Scientific & Technical Reference Library
|
||||
|
||||
## Purpose
|
||||
|
||||
This directory contains curated reference documents for EVOLV's domain-specialist agents. These summaries distill authoritative sources into actionable knowledge that agents should consult **before making scientific or engineering claims**.
|
||||
|
||||
## How Agents Should Use This
|
||||
|
||||
1. **Before making domain claims**: Read the relevant reference doc to verify your reasoning
|
||||
2. **Cite sources**: When referencing scientific facts, point to the specific reference doc and its cited sources
|
||||
3. **Acknowledge uncertainty**: If the reference docs don't cover a topic, say so rather than guessing
|
||||
4. **Cross-reference with skills**: Combine these references with `.agents/skills/` SKILL.md files for implementation context
|
||||
|
||||
## Index
|
||||
|
||||
| File | Domain | Used By Agents |
|
||||
|------|--------|---------------|
|
||||
| [`asm-models.md`](asm-models.md) | Activated Sludge Models (ASM1-ASM3) | biological-process-engineer |
|
||||
| [`settling-models.md`](settling-models.md) | Sludge Settling & Clarifier Models | biological-process-engineer |
|
||||
| [`pump-affinity-laws.md`](pump-affinity-laws.md) | Pump Affinity Laws & Curve Theory | mechanical-process-engineer |
|
||||
| [`pid-control-theory.md`](pid-control-theory.md) | PID Control for Process Applications | mechanical-process-engineer, node-red-runtime |
|
||||
| [`signal-processing-sensors.md`](signal-processing-sensors.md) | Sensor Signal Conditioning | instrumentation-measurement |
|
||||
| [`wastewater-compliance-nl.md`](wastewater-compliance-nl.md) | Dutch Wastewater Regulations | commissioning-compliance, biological-process-engineer |
|
||||
| [`influxdb-schema-design.md`](influxdb-schema-design.md) | InfluxDB Time-Series Best Practices | telemetry-database |
|
||||
| [`ot-security-iec62443.md`](ot-security-iec62443.md) | OT Security Standards | ot-security-integration |
|
||||
|
||||
## Sources Directory
|
||||
|
||||
The `sources/` subdirectory is for placing actual PDFs of scientific papers, standards, and technical manuals. Agents should prefer these curated summaries but can reference originals when available.
|
||||
|
||||
## Validation Status
|
||||
|
||||
All reference documents have been validated against authoritative sources including:
|
||||
- IWA Scientific and Technical Reports (ASM models)
|
||||
- Peer-reviewed publications (Takacs 1991, Vesilind, Burger-Diehl)
|
||||
- Engineering Toolbox (pump affinity laws)
|
||||
- ISA publications (Astrom & Hagglund PID control)
|
||||
- IEC standards (61298, 62443)
|
||||
- EU Directive 91/271/EEC (wastewater compliance)
|
||||
- InfluxDB official documentation (schema design)
|
||||
127
third_party/docs/asm-models.md
vendored
Normal file
127
third_party/docs/asm-models.md
vendored
Normal file
@@ -0,0 +1,127 @@
|
||||
# Activated Sludge Models (ASM1, ASM2d, ASM3)
|
||||
|
||||
> **Used by**: `biological-process-engineer` agent, `reactor` node, `monster` node
|
||||
> **Validation**: Verified against IWA publications, WaterTAP documentation, and peer-reviewed literature
|
||||
|
||||
## ASM1 — Activated Sludge Model No. 1
|
||||
|
||||
**Source**: Henze, M., Grady, C.P.L., Gujer, W., Marais, G.v.R., Matsuo, T. (1987). IAWPRC Task Group on Mathematical Modelling for Design and Operation of Biological Wastewater Treatment.
|
||||
|
||||
**Published**: IWA Scientific and Technical Report No. 1
|
||||
|
||||
### 13 Components (State Variables)
|
||||
|
||||
| Symbol | Description | Type |
|
||||
|--------|-------------|------|
|
||||
| S_I | Soluble inert organic matter | Soluble |
|
||||
| S_S | Readily biodegradable substrate | Soluble |
|
||||
| X_I | Particulate inert organic matter | Particulate |
|
||||
| X_S | Slowly biodegradable substrate | Particulate |
|
||||
| X_B,H | Active heterotrophic biomass | Particulate |
|
||||
| X_B,A | Active autotrophic biomass | Particulate |
|
||||
| X_P | Particulate products from biomass decay | Particulate |
|
||||
| S_O | Dissolved oxygen | Soluble |
|
||||
| S_NO | Nitrate and nitrite nitrogen | Soluble |
|
||||
| S_NH | Ammonium nitrogen (NH₄⁺-N) | Soluble |
|
||||
| S_ND | Soluble biodegradable organic nitrogen | Soluble |
|
||||
| X_ND | Particulate biodegradable organic nitrogen | Particulate |
|
||||
| S_ALK | Alkalinity (molar units) | Soluble |
|
||||
|
||||
### 8 Processes
|
||||
|
||||
1. **Aerobic growth of heterotrophs**: S_S + S_O + S_NH → X_B,H (Monod kinetics)
|
||||
2. **Anoxic growth of heterotrophs**: S_S + S_NO + S_NH → X_B,H (denitrification)
|
||||
3. **Aerobic growth of autotrophs**: S_NH + S_O → X_B,A + S_NO (nitrification)
|
||||
4. **Decay of heterotrophs**: X_B,H → X_P + X_S (death-regeneration concept)
|
||||
5. **Decay of autotrophs**: X_B,A → X_P + X_S
|
||||
6. **Ammonification of soluble organic nitrogen**: S_ND → S_NH
|
||||
7. **Hydrolysis of entrapped organics**: X_S → S_S
|
||||
8. **Hydrolysis of entrapped organic nitrogen**: X_ND → S_ND
|
||||
|
||||
### Key Kinetic Parameters (default values at 20°C)
|
||||
|
||||
| Parameter | Symbol | Default | Unit | Description |
|
||||
|-----------|--------|---------|------|-------------|
|
||||
| Max. heterotrophic growth rate | μ_H | 6.0 | d⁻¹ | |
|
||||
| Half-saturation (substrate) | K_S | 20.0 | g COD/m³ | |
|
||||
| Half-saturation (oxygen, het.) | K_O,H | 0.2 | g O₂/m³ | |
|
||||
| Half-saturation (nitrate) | K_NO | 0.5 | g NO₃-N/m³ | |
|
||||
| Heterotrophic decay rate | b_H | 0.62 | d⁻¹ | |
|
||||
| Max. autotrophic growth rate | μ_A | 0.8 | d⁻¹ | |
|
||||
| Half-saturation (ammonia) | K_NH | 1.0 | g NH₃-N/m³ | |
|
||||
| Half-saturation (oxygen, aut.) | K_O,A | 0.4 | g O₂/m³ | |
|
||||
| Autotrophic decay rate | b_A | 0.05 | d⁻¹ | |
|
||||
| Anoxic reduction factor | η_g | 0.8 | — | |
|
||||
| Hydrolysis rate | k_h | 3.0 | g X_S/(g X_B,H · d) | |
|
||||
| Yield (heterotrophic) | Y_H | 0.67 | g COD/g COD | |
|
||||
| Yield (autotrophic) | Y_A | 0.24 | g COD/g N | |
|
||||
| Fraction to X_P | f_P | 0.08 | — | |
|
||||
|
||||
### Temperature Correction
|
||||
|
||||
Arrhenius-type: k(T) = k(20) · θ^(T-20)
|
||||
|
||||
Common θ values:
|
||||
- Heterotrophic growth: θ = 1.072
|
||||
- Autotrophic growth: θ = 1.103 (nitrifiers are very temperature-sensitive)
|
||||
- Decay: θ = 1.04
|
||||
|
||||
### Presentation Format
|
||||
|
||||
The model is presented in the **Petersen matrix** (also called Gujer matrix) format, where rows are processes and columns are components. Each cell contains the stoichiometric coefficient for how a process affects a component.
|
||||
|
||||
## ASM2d — Activated Sludge Model No. 2d
|
||||
|
||||
**Source**: Henze, M., Gujer, W., Mino, T., Matsuo, T., Wentzel, M.C., Marais, G.v.R., van Loosdrecht, M.C.M. (1999)
|
||||
|
||||
**Published**: IWA Scientific and Technical Report No. 3; also Water Science & Technology 39(1), 165-182
|
||||
|
||||
### Key Extensions over ASM1
|
||||
- Adds **biological phosphorus removal** by phosphorus accumulating organisms (PAOs)
|
||||
- Includes **denitrifying PAOs** (simultaneous P-removal and denitrification)
|
||||
- 19 components, 21 processes
|
||||
- Models storage of poly-hydroxy-alkanoates (PHA) and polyphosphate (poly-P) by PAOs
|
||||
- Includes fermentation of readily biodegradable substrate
|
||||
|
||||
### Additional Components (beyond ASM1)
|
||||
- S_F: Fermentable, readily biodegradable substrate
|
||||
- S_A: Fermentation products (acetate)
|
||||
- S_PO4: Soluble ortho-phosphate
|
||||
- X_PAO: Phosphorus accumulating organisms
|
||||
- X_PP: Poly-phosphate stored by PAOs
|
||||
- X_PHA: Poly-hydroxy-alkanoates stored by PAOs
|
||||
|
||||
## ASM3 — Activated Sludge Model No. 3
|
||||
|
||||
**Source**: Gujer, W., Henze, M., Mino, T., van Loosdrecht, M.C.M. (1999); updated in Henze et al. (2001)
|
||||
|
||||
**Published**: IWA Scientific and Technical Report No. 9
|
||||
|
||||
### Key Differences from ASM1
|
||||
- **Replaces death-regeneration** with endogenous respiration (conceptually simpler)
|
||||
- **Introduces storage polymers** (X_STO) for heterotrophic biomass — substrate is first stored, then used for growth
|
||||
- 13 state variables, 12 reactions
|
||||
- More suitable for dynamic simulation and control applications
|
||||
- Eliminates the problematic simultaneous storage/growth ambiguity in ASM1
|
||||
|
||||
### Storage-Based Metabolism
|
||||
In ASM3, heterotrophs first store readily biodegradable substrate as internal storage products (X_STO), then grow on these stored products. This two-step process better reflects observed biological behavior.
|
||||
|
||||
## Choosing Between Models
|
||||
|
||||
| Criterion | ASM1 | ASM2d | ASM3 |
|
||||
|-----------|------|-------|------|
|
||||
| Carbon & nitrogen | Yes | Yes | Yes |
|
||||
| Phosphorus removal | No | Yes | Via separate Bio-P module |
|
||||
| Computational cost | Low | High | Medium |
|
||||
| Calibration effort | Low | High | Medium |
|
||||
| Best for | Carbon/N only WWTPs | Bio-P plants | Dynamic control |
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. Henze, M. et al. (1987). "Activated Sludge Model No. 1" — IAWPRC Scientific and Technical Report No. 1
|
||||
2. Henze, M. et al. (1995). "Activated Sludge Model No. 2" — IAWQ Scientific and Technical Report No. 3
|
||||
3. Henze, M. et al. (1999). "Activated Sludge Model No. 2d" — Water Sci. Technol. 39(1), 165-182
|
||||
4. Gujer, W. et al. (1999). "Activated Sludge Model No. 3" — Water Sci. Technol. 39(1), 183-193
|
||||
5. Henze, M. et al. (2000). "Activated Sludge Models ASM1, ASM2, ASM2d and ASM3" — IWA Publishing, ISBN 9781900222242
|
||||
6. Jeppsson, U. (1996). "Modelling Aspects of Wastewater Treatment Processes" — Lund University PhD thesis (comprehensive ASM1 parameter listing)
|
||||
122
third_party/docs/influxdb-schema-design.md
vendored
Normal file
122
third_party/docs/influxdb-schema-design.md
vendored
Normal file
@@ -0,0 +1,122 @@
|
||||
# InfluxDB Time-Series Best Practices
|
||||
|
||||
> **Used by**: `telemetry-database` agent, `dashboardAPI` node
|
||||
> **Validation**: Verified against InfluxDB official documentation (v1, v2, v3)
|
||||
|
||||
## Tag vs. Field Decision Framework
|
||||
|
||||
| Criterion | Use Tag | Use Field |
|
||||
|-----------|---------|-----------|
|
||||
| Queried in WHERE clause frequently | Yes | No |
|
||||
| Used in GROUP BY | Yes | No |
|
||||
| Low cardinality (< 100 distinct values) | Yes | Acceptable |
|
||||
| High cardinality (IDs, timestamps, free text) | **Never** | Yes |
|
||||
| Numeric measurement values | No | Yes |
|
||||
| Needs aggregation (mean, sum, etc.) | No | Yes |
|
||||
| Node/station/machine identifier | Yes | No |
|
||||
| Actual sensor reading | No | Yes |
|
||||
| Setpoint value | No | Yes |
|
||||
| Quality flag | Depends* | Yes |
|
||||
|
||||
*Quality flags: If you have ≤5 quality levels (good/uncertain/bad), a tag is acceptable. If quality is a numeric score, use a field.
|
||||
|
||||
## EVOLV Tag/Field Convention
|
||||
|
||||
### Standard Tags (low cardinality, indexed)
|
||||
```
|
||||
locationId — Site identifier (e.g., "wwtp-brabant-01")
|
||||
nodeType — Node type (e.g., "rotatingMachine", "reactor")
|
||||
nodeName — Instance name (e.g., "pump-01", "reactor-A")
|
||||
machineType — Equipment type (e.g., "pump", "blower", "valve")
|
||||
stationId — Parent station identifier
|
||||
measurementType — Sensor type (e.g., "flow", "pressure", "temperature")
|
||||
```
|
||||
|
||||
### Standard Fields (not indexed, high cardinality)
|
||||
```
|
||||
value — Primary measurement value
|
||||
setpoint — Control setpoint
|
||||
quality — Data quality score (0.0-1.0)
|
||||
state — Machine state (numeric code)
|
||||
power — Power consumption (W)
|
||||
efficiency — Current efficiency (0.0-1.0)
|
||||
speed — Rotational speed (RPM or fraction)
|
||||
position — Valve position (0.0-1.0)
|
||||
```
|
||||
|
||||
## Cardinality Management
|
||||
|
||||
### What Is Cardinality?
|
||||
Series cardinality = unique combinations of (measurement_name × tag_key_1 × tag_key_2 × ... × tag_key_n)
|
||||
|
||||
### Cardinality Limits
|
||||
- **InfluxDB v1/v2 (TSM engine)**: High cardinality degrades query performance and increases memory usage. Keep below ~1M series per database.
|
||||
- **InfluxDB v3**: Supports infinite series cardinality (new storage engine), but keeping cardinality low still improves query speed.
|
||||
|
||||
### Anti-Patterns (NEVER do these)
|
||||
- Encoding timestamps in tag values
|
||||
- Using UUIDs or session IDs as tags
|
||||
- Free-text strings as tags
|
||||
- Unbounded enum values as tags
|
||||
- One measurement per sensor (use tags to differentiate instead)
|
||||
|
||||
### Good Patterns
|
||||
- Use a single measurement name per data category
|
||||
- Differentiate by tags, not by measurement name
|
||||
- Keep tag value sets bounded and predictable
|
||||
- Document all tag values in a schema registry
|
||||
|
||||
## Retention Policies
|
||||
|
||||
### Three-Tier Strategy
|
||||
|
||||
| Tier | Retention | Resolution | Purpose |
|
||||
|------|-----------|------------|---------|
|
||||
| Hot | 7-30 days | Full resolution (1s-10s) | Real-time dashboards, control loops |
|
||||
| Warm | 90-365 days | Downsampled (1min-5min) | Trending, troubleshooting |
|
||||
| Cold | 2-10 years | Heavily aggregated (1h-24h) | Compliance reporting, long-term trends |
|
||||
|
||||
### EVOLV Recommended Defaults
|
||||
- Port 1 data at full resolution: 30 days
|
||||
- 1-minute aggregates: 1 year
|
||||
- 1-hour aggregates: 5 years (matches regulatory retention requirements)
|
||||
|
||||
## Continuous Queries / Tasks (Downsampling)
|
||||
|
||||
### InfluxDB v1: Continuous Queries
|
||||
```sql
|
||||
CREATE CONTINUOUS QUERY "downsample_1m" ON "evolv"
|
||||
BEGIN
|
||||
SELECT mean("value") AS "value", max("value") AS "max", min("value") AS "min"
|
||||
INTO "rp_warm"."downsampled_1m"
|
||||
FROM "telemetry"
|
||||
GROUP BY time(1m), *
|
||||
END
|
||||
```
|
||||
|
||||
### InfluxDB v2: Tasks
|
||||
```flux
|
||||
option task = {name: "downsample_1m", every: 1m}
|
||||
|
||||
from(bucket: "telemetry")
|
||||
|> range(start: -task.every)
|
||||
|> aggregateWindow(every: 1m, fn: mean, createEmpty: false)
|
||||
|> to(bucket: "telemetry-warm")
|
||||
```
|
||||
|
||||
## Query Performance Tips
|
||||
|
||||
1. **Always filter by time range first** — time is the primary index
|
||||
2. **Use tag filters in WHERE** — tags are indexed, fields are not
|
||||
3. **Avoid regex on tag values** — use exact matches when possible
|
||||
4. **Limit series scanned** — filter by specific nodeType/nodeName
|
||||
5. **Use aggregation** — let the database aggregate rather than fetching raw points
|
||||
6. **Batch writes** — write in batches of 5,000-10,000 points for optimal throughput
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. InfluxDB Documentation — "Schema Design and Data Layout" (https://docs.influxdata.com/influxdb/v1/concepts/schema_and_data_layout/)
|
||||
2. InfluxDB Documentation — "Schema Design Recommendations and Best Practices" (v2/v3)
|
||||
3. InfluxData Blog — "Time Series Data, Cardinality, and InfluxDB"
|
||||
4. InfluxDB Documentation — "Resolve High Series Cardinality" (https://docs.influxdata.com/influxdb/v2/write-data/best-practices/resolve-high-cardinality/)
|
||||
5. InfluxData (2023). "InfluxDB Best Practices" — Official technical guides
|
||||
149
third_party/docs/ot-security-iec62443.md
vendored
Normal file
149
third_party/docs/ot-security-iec62443.md
vendored
Normal file
@@ -0,0 +1,149 @@
|
||||
# OT Security Standards — IEC 62443 & NIST SP 800-82
|
||||
|
||||
> **Used by**: `ot-security-integration` agent
|
||||
> **Validation**: Verified against IEC 62443 series, NIST SP 800-82, Dragos, and Rockwell Automation publications
|
||||
|
||||
## IEC 62443 Framework Overview
|
||||
|
||||
IEC 62443 "Industrial communication networks — IT security for networks and systems" is the primary international standard series for Industrial Automation and Control System (IACS) cybersecurity.
|
||||
|
||||
### Standard Structure
|
||||
|
||||
| Part | Title | Scope |
|
||||
|------|-------|-------|
|
||||
| 62443-1-x | General | Concepts, vocabulary, use cases |
|
||||
| 62443-2-x | Policies & Procedures | Security management system, patch management |
|
||||
| 62443-3-x | System | System security requirements, zones & conduits |
|
||||
| 62443-4-x | Component | Secure development lifecycle, component requirements |
|
||||
|
||||
### Key Parts for EVOLV
|
||||
- **62443-3-2**: Security risk assessment and system design
|
||||
- **62443-3-3**: System security requirements and security levels
|
||||
- **62443-4-2**: Technical security requirements for IACS components
|
||||
|
||||
## Zones and Conduits
|
||||
|
||||
### Security Zone
|
||||
A **zone** is a logical or physical grouping of assets that share common security requirements. Assets within a zone have the same security level (SL) target.
|
||||
|
||||
### Conduit
|
||||
A **conduit** is a logical or physical grouping of communication channels connecting two or more zones. Conduits require security controls appropriate for the traffic they carry.
|
||||
|
||||
### EVOLV Zone Architecture (typical WWTP deployment)
|
||||
|
||||
```
|
||||
Zone 0: Enterprise IT Network (SL 1-2)
|
||||
↕ [Conduit: Firewall/DMZ]
|
||||
Zone 1: SCADA/Historian Network (SL 2-3)
|
||||
↕ [Conduit: Data diode or filtered bridge]
|
||||
Zone 2: Process Control Network (SL 3)
|
||||
↕ [Conduit: Managed switch with ACLs]
|
||||
Zone 3: Field Device Network (SL 2-3)
|
||||
- PLCs, RTUs, I/O modules
|
||||
- Node-RED edge runtime (EVOLV)
|
||||
- Sensors and actuators
|
||||
```
|
||||
|
||||
### Zone Design Rules
|
||||
- A zone can contain sub-zones
|
||||
- A conduit cannot contain sub-conduits
|
||||
- A zone can have more than one conduit
|
||||
- Every device must belong to exactly one zone
|
||||
- Communication between zones must pass through a conduit
|
||||
|
||||
## Security Levels (SL)
|
||||
|
||||
| Level | Protection Against | Typical Measures |
|
||||
|-------|-------------------|------------------|
|
||||
| SL 0 | No specific requirements | — |
|
||||
| SL 1 | Casual or coincidental violation | Basic authentication, logging |
|
||||
| SL 2 | Intentional attack with low motivation, generic skills | Role-based access, encrypted communications |
|
||||
| SL 3 | Intentional attack with moderate motivation, IACS-specific skills | Strong authentication, intrusion detection, hardened systems |
|
||||
| SL 4 | Intentional attack with high motivation, IACS-specific skills, extended resources | Dedicated security team, continuous monitoring, zero-trust |
|
||||
|
||||
### SL Types
|
||||
- **SL-T (Target)**: Required security level for the zone
|
||||
- **SL-A (Achieved)**: Actual security level implemented
|
||||
- **SL-C (Capability)**: Maximum security level a component can support
|
||||
|
||||
## NIST SP 800-82 — Guide to ICS Security
|
||||
|
||||
**Source**: NIST Special Publication 800-82 Revision 3 (2023). "Guide to Operational Technology (OT) Security"
|
||||
|
||||
### Key Recommendations
|
||||
1. Develop and maintain an OT-specific security program
|
||||
2. Segment OT networks from IT networks (defense in depth)
|
||||
3. Apply least privilege access control
|
||||
4. Monitor OT network traffic for anomalies
|
||||
5. Maintain an accurate OT asset inventory
|
||||
6. Implement secure remote access with MFA
|
||||
7. Develop OT-specific incident response plans
|
||||
8. Regularly assess and manage OT security risks
|
||||
|
||||
## OPC UA Security Model
|
||||
|
||||
### Authentication
|
||||
- X.509 certificates for server and client authentication
|
||||
- Username/password as fallback (less secure)
|
||||
- Anonymous access (should be disabled in production)
|
||||
|
||||
### Encryption
|
||||
- Security policies define algorithm suites:
|
||||
- `None` — No security (testing only)
|
||||
- `Basic128Rsa15` — Deprecated, avoid
|
||||
- `Basic256` — Deprecated, avoid
|
||||
- `Basic256Sha256` — Minimum recommended
|
||||
- `Aes128_Sha256_RsaOaep` — Preferred
|
||||
- `Aes256_Sha256_RsaPss` — Strongest
|
||||
|
||||
### Message Security Modes
|
||||
- `None` — No signing or encryption
|
||||
- `Sign` — Messages signed but not encrypted
|
||||
- `SignAndEncrypt` — Full protection (recommended)
|
||||
|
||||
## Modbus Security Considerations
|
||||
|
||||
### Vulnerabilities (standard Modbus TCP)
|
||||
- No authentication — any network client can read/write registers
|
||||
- No encryption — all traffic is plaintext
|
||||
- No integrity protection — commands can be modified in transit
|
||||
- Predictable function codes — easy to craft malicious packets
|
||||
|
||||
### Mitigations
|
||||
1. Network segmentation — isolate Modbus devices in dedicated VLANs
|
||||
2. Firewall rules — whitelist only authorized master IP addresses
|
||||
3. Application-layer filtering — deep packet inspection for Modbus function codes
|
||||
4. Monitoring — detect unusual register access patterns
|
||||
5. Modbus/TCP Security (TLS) — available in newer implementations (RFC 7878-based)
|
||||
|
||||
## EVOLV-Specific Security Considerations
|
||||
|
||||
### Node-RED Admin Endpoints
|
||||
- `GET /<nodeName>/menu.js` — Serves configuration data to editor
|
||||
- `GET /<nodeName>/configData.js` — Serves runtime config to editor
|
||||
- **Risk**: Information disclosure if exposed beyond editor network
|
||||
- **Mitigation**: Bind Node-RED to localhost or trusted network only
|
||||
|
||||
### msg.topic Input Validation
|
||||
- All `msg.topic` handlers must validate topic format before processing
|
||||
- Prevent topic injection: reject topics containing path separators, special characters
|
||||
- Validate payload types and ranges before applying to control logic
|
||||
|
||||
### Dynamic Configuration
|
||||
- Configuration loaded from files or received via MQTT
|
||||
- Must validate schema, types, and value ranges before applying
|
||||
- Reject configurations that would violate safety envelopes
|
||||
|
||||
### Control Message Safety
|
||||
- Validate actuator commands against physical limits before sending
|
||||
- Rate-limit control output changes (prevent rapid cycling)
|
||||
- Log all control actions with timestamp, source, and reason
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. IEC 62443 series (2018-2024). "Industrial communication networks — IT security for networks and systems"
|
||||
2. NIST SP 800-82 Rev 3 (2023). "Guide to Operational Technology (OT) Security"
|
||||
3. Dragos Inc. — "Understanding ISA/IEC 62443: A Guide for OT Security Teams" (https://www.dragos.com/blog/isa-iec-62443-concepts)
|
||||
4. ISA/IEC 62443-3-3 — "System Security Requirements and Security Levels"
|
||||
5. OPC Foundation — "OPC UA Security Model" specification
|
||||
6. Modbus Organization — "MODBUS/TCP Security" specification
|
||||
168
third_party/docs/pid-control-theory.md
vendored
Normal file
168
third_party/docs/pid-control-theory.md
vendored
Normal file
@@ -0,0 +1,168 @@
|
||||
# PID Control for Process Applications
|
||||
|
||||
> **Used by**: `mechanical-process-engineer` agent, `node-red-runtime` agent, `generalFunctions/src/pid/`
|
||||
> **Validation**: Verified against Astrom & Hagglund (ISA, 2006) and MATLAB/Simulink documentation
|
||||
|
||||
## Continuous PID Controller
|
||||
|
||||
### Standard (ISA/Ideal) Form
|
||||
|
||||
```
|
||||
u(t) = K_p · [e(t) + (1/T_i) · ∫e(τ)dτ + T_d · de(t)/dt]
|
||||
```
|
||||
|
||||
Where:
|
||||
- u(t) = controller output
|
||||
- e(t) = error = setpoint - process variable (SP - PV)
|
||||
- K_p = proportional gain
|
||||
- T_i = integral time (seconds)
|
||||
- T_d = derivative time (seconds)
|
||||
|
||||
### Parallel Form
|
||||
|
||||
```
|
||||
u(t) = K_p · e(t) + K_i · ∫e(τ)dτ + K_d · de(t)/dt
|
||||
```
|
||||
|
||||
Where K_i = K_p/T_i and K_d = K_p · T_d
|
||||
|
||||
## Discrete PID Implementation
|
||||
|
||||
### Positional Form (absolute output)
|
||||
|
||||
```
|
||||
u[k] = K_p · e[k] + K_i · Δt · Σe[j] + K_d · (e[k] - e[k-1]) / Δt
|
||||
```
|
||||
|
||||
- Computes the absolute output value each cycle
|
||||
- Requires tracking of the integral sum
|
||||
- Subject to integral windup
|
||||
- Used when the controller output directly sets an actuator position
|
||||
|
||||
### Velocity Form (incremental output)
|
||||
|
||||
```
|
||||
Δu[k] = K_p · (e[k] - e[k-1]) + K_i · Δt · e[k] + K_d · (e[k] - 2·e[k-1] + e[k-2]) / Δt
|
||||
u[k] = u[k-1] + Δu[k]
|
||||
```
|
||||
|
||||
- Computes the **change** in output each cycle
|
||||
- Inherently bumpless on mode transfers (auto→manual→auto)
|
||||
- Less prone to windup (but still needs anti-windup for output limits)
|
||||
- Preferred for incremental actuators (VFDs, variable valves)
|
||||
|
||||
## Anti-Windup Strategies
|
||||
|
||||
Integral windup occurs when the controller output saturates (hits actuator limits) but the integral term continues to accumulate, causing large overshoot when the error changes sign.
|
||||
|
||||
### 1. Clamping (Conditional Integration)
|
||||
|
||||
Stop integrating when the output is saturated **and** the error has the same sign as the integral term:
|
||||
|
||||
```
|
||||
if (u_raw > u_max || u_raw < u_min) && sign(e) == sign(integral):
|
||||
freeze integral (do not accumulate)
|
||||
else:
|
||||
integral += e * Δt
|
||||
```
|
||||
|
||||
- Simple to implement
|
||||
- Effective for most process control applications
|
||||
- The approach used in the EVOLV generalFunctions PID implementation
|
||||
|
||||
### 2. Back-Calculation
|
||||
|
||||
When the output saturates, feed back the difference between the saturated and unsaturated output to "unwind" the integrator:
|
||||
|
||||
```
|
||||
integral += (K_i · e + K_b · (u_saturated - u_raw)) · Δt
|
||||
```
|
||||
|
||||
Where K_b = 1/T_t (tracking time constant, typically T_t = √(T_i · T_d) or T_t = T_d).
|
||||
|
||||
- More sophisticated than clamping
|
||||
- Better performance for systems with large disturbances
|
||||
- Recommended by Astrom & Hagglund for demanding applications
|
||||
|
||||
### 3. Integrator Reset
|
||||
|
||||
Reset the integrator to a value that would produce the saturated output:
|
||||
|
||||
```
|
||||
if u_raw > u_max:
|
||||
integral = (u_max - K_p · e) / K_i
|
||||
```
|
||||
|
||||
- Simple but can be aggressive
|
||||
- May cause discontinuities
|
||||
|
||||
## Derivative Filtering
|
||||
|
||||
The derivative term amplifies high-frequency noise. Always filter it:
|
||||
|
||||
### First-Order Low-Pass Filter on D-Term
|
||||
|
||||
```
|
||||
D_filtered[k] = α · D_filtered[k-1] + (1-α) · D_raw[k]
|
||||
```
|
||||
|
||||
Where α = T_f / (T_f + Δt) and T_f = T_d / N (N typically 5-20, default 10).
|
||||
|
||||
### Derivative on PV (not Error)
|
||||
|
||||
To avoid derivative kick on setpoint changes:
|
||||
```
|
||||
D = -K_d · (PV[k] - PV[k-1]) / Δt (instead of using error)
|
||||
```
|
||||
|
||||
This is the standard approach for process control.
|
||||
|
||||
## Cascade PID
|
||||
|
||||
Two nested loops where the outer (master) loop's output is the setpoint for the inner (slave) loop.
|
||||
|
||||
```
|
||||
[SP_outer] → [PID_outer] → [SP_inner] → [PID_inner] → [Actuator] → [Process]
|
||||
↑ ↑
|
||||
[PV_outer] ←──── [Process] ←── [PV_inner]
|
||||
```
|
||||
|
||||
### Design Rules
|
||||
- Inner loop must be **5-10x faster** than outer loop
|
||||
- Tune inner loop first (with outer loop in manual)
|
||||
- Then tune outer loop
|
||||
- Anti-windup on outer loop essential (its output is bounded by inner loop's SP limits)
|
||||
|
||||
### EVOLV Application
|
||||
- Outer: Level controller → outputs flow setpoint
|
||||
- Inner: Flow controller → outputs pump speed
|
||||
- pumpingStation node coordinates this cascade
|
||||
|
||||
## Tuning Methods
|
||||
|
||||
### Ziegler-Nichols (Ultimate Gain Method)
|
||||
1. Set I and D to zero, increase K_p until sustained oscillation
|
||||
2. Record ultimate gain K_u and ultimate period T_u
|
||||
3. Apply: K_p = 0.6·K_u, T_i = T_u/2, T_d = T_u/8
|
||||
|
||||
### Cohen-Coon (Process Reaction Curve)
|
||||
1. Apply step change, record process reaction curve
|
||||
2. Identify: gain K, dead time L, time constant τ
|
||||
3. Apply formulas based on K, L, τ
|
||||
|
||||
### Lambda Tuning (IMC-based)
|
||||
1. Identify process as FOPDT: K, L, τ
|
||||
2. Choose closed-loop time constant λ (typically λ = max(3·L, τ))
|
||||
3. K_p = τ/(K·λ), T_i = τ
|
||||
|
||||
- **Preferred for process control** — gives non-oscillatory response
|
||||
- Directly specifies desired closed-loop speed
|
||||
- Robust to model uncertainty when λ is chosen conservatively
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. Astrom, K.J. & Hagglund, T. (2006). "Advanced PID Control." ISA — The Instrumentation, Systems, and Automation Society.
|
||||
2. Astrom, K.J. & Hagglund, T. (1995). "PID Controllers: Theory, Design, and Tuning." 2nd ed., ISA.
|
||||
3. Seborg, D.E. et al. (2011). "Process Dynamics and Control." 3rd ed., Wiley. (Chapter on PID control)
|
||||
4. MATLAB/Simulink documentation — "Anti-Windup Control Using PID Controller Block"
|
||||
5. Smith, C.A. & Corripio, A.B. (2005). "Principles and Practices of Automatic Process Control." 3rd ed., Wiley.
|
||||
154
third_party/docs/pump-affinity-laws.md
vendored
Normal file
154
third_party/docs/pump-affinity-laws.md
vendored
Normal file
@@ -0,0 +1,154 @@
|
||||
# Pump Affinity Laws & Curve Theory
|
||||
|
||||
> **Used by**: `mechanical-process-engineer` agent, `rotatingMachine` node, `pumpingStation` node
|
||||
> **Validation**: Verified against Engineering Toolbox, Hydraulic Institute standards, and ScienceDirect
|
||||
|
||||
## Affinity Laws
|
||||
|
||||
The affinity laws describe how centrifugal pump performance scales with changes in rotational speed (N) or impeller diameter (D). They are derived from dimensional analysis under the assumption of geometric similarity (velocity triangles at the impeller remain geometrically similar).
|
||||
|
||||
### Speed Variation (constant diameter)
|
||||
|
||||
```
|
||||
Q₂/Q₁ = N₂/N₁
|
||||
H₂/H₁ = (N₂/N₁)²
|
||||
P₂/P₁ = (N₂/N₁)³
|
||||
```
|
||||
|
||||
### Diameter Variation (constant speed)
|
||||
|
||||
```
|
||||
Q₂/Q₁ = D₂/D₁
|
||||
H₂/H₁ = (D₂/D₁)²
|
||||
P₂/P₁ = (D₂/D₁)³
|
||||
```
|
||||
|
||||
### Combined Variation
|
||||
|
||||
```
|
||||
Q₂/Q₁ = (N₂/N₁) · (D₂/D₁)
|
||||
H₂/H₁ = (N₂/N₁)² · (D₂/D₁)²
|
||||
P₂/P₁ = (N₂/N₁)³ · (D₂/D₁)³
|
||||
```
|
||||
|
||||
### Practical Example (validated against Engineering Toolbox)
|
||||
A pump at 1750 rpm delivering 100 gpm at 100 ft head using 5 bhp:
|
||||
- At 3500 rpm: **200 gpm** flow, **400 ft** head, **40 bhp** power
|
||||
- A 10% speed increase yields: +10% flow, +21% head, +33% power
|
||||
|
||||
### Accuracy Limitations
|
||||
- Affinity laws are **approximate** — accuracy decreases with large speed changes (>±30%)
|
||||
- Efficiency shifts slightly with speed change (not captured by basic affinity laws)
|
||||
- Trimming impeller diameter >15-20% significantly reduces accuracy
|
||||
- Laws assume no significant change in Reynolds number effects
|
||||
|
||||
## Pump Curve Types
|
||||
|
||||
### Q-H Curve (Flow vs. Head)
|
||||
- Primary performance curve
|
||||
- Head decreases as flow increases (for centrifugal pumps)
|
||||
- Shape depends on specific speed (Ns): flat, steep, or drooping
|
||||
- **Monotonicity**: Should be monotonically decreasing for stable operation. Non-monotonic (drooping) curves can cause instability in parallel operation.
|
||||
|
||||
### Q-P Curve (Flow vs. Power)
|
||||
- Power consumption as function of flow
|
||||
- Shape varies by pump type:
|
||||
- Radial: power increases with flow (non-overloading possible at shutoff)
|
||||
- Mixed flow: relatively flat
|
||||
- Axial: power **decreases** with flow (overload risk at low flow)
|
||||
|
||||
### Q-η Curve (Flow vs. Efficiency)
|
||||
- Efficiency peaks at Best Efficiency Point (BEP)
|
||||
- Falls off on both sides of BEP
|
||||
- Operating far from BEP causes excessive vibration, cavitation risk, and energy waste
|
||||
|
||||
## Best Efficiency Point (BEP)
|
||||
|
||||
The BEP is the operating point where the pump converts the maximum fraction of input power to useful hydraulic work.
|
||||
|
||||
### BEP Tracking Under VFD Control
|
||||
When speed changes via VFD, the BEP shifts along a **parabolic path** in the Q-H plane:
|
||||
```
|
||||
H_BEP ∝ Q_BEP²
|
||||
```
|
||||
This is because both Q and H scale with speed, but H scales as the square of Q's scaling factor.
|
||||
|
||||
### Preferred Operating Region
|
||||
- Continuous operation: 80% – 110% of BEP flow
|
||||
- Allowable range: 70% – 120% of BEP flow
|
||||
- Outside this range: increased bearing loads, seal wear, cavitation risk
|
||||
|
||||
## System Curve Theory
|
||||
|
||||
The system curve describes the head required by the piping system as a function of flow:
|
||||
|
||||
```
|
||||
H_system = H_static + k · Q²
|
||||
```
|
||||
|
||||
Where:
|
||||
- H_static = static head (elevation difference + tank pressure difference)
|
||||
- k = system resistance coefficient (Pa·s²/m⁶ in SI)
|
||||
- Q = volumetric flow rate
|
||||
|
||||
### Duty Point
|
||||
The **duty point** (operating point) is the intersection of the pump curve and system curve:
|
||||
```
|
||||
H_pump(Q) = H_system(Q)
|
||||
```
|
||||
This is solved numerically — find Q where pump curve equals system curve.
|
||||
|
||||
## Parallel Pump Operation
|
||||
|
||||
### Flow Summation Rule
|
||||
For pumps in parallel at equal head:
|
||||
```
|
||||
Q_total = Q₁ + Q₂ + ... + Qₙ (at each head value)
|
||||
```
|
||||
The combined curve is constructed by **horizontally adding** individual pump curves.
|
||||
|
||||
### Key Considerations
|
||||
- Each pump must overcome the same system head
|
||||
- Adding a pump shifts the combined curve right, moving the operating point
|
||||
- Diminishing returns: each additional pump adds less incremental flow
|
||||
- Risk of back-flow through stopped pumps (check valves required)
|
||||
- Unstable operation if pump curves have a drooping characteristic
|
||||
|
||||
## Series Pump Operation
|
||||
|
||||
For pumps in series at equal flow:
|
||||
```
|
||||
H_total = H₁ + H₂ + ... + Hₙ (at each flow value)
|
||||
```
|
||||
The combined curve is constructed by **vertically adding** individual pump curves.
|
||||
|
||||
## Specific Energy
|
||||
|
||||
The key energy KPI for pumping systems:
|
||||
|
||||
```
|
||||
SE = P / Q [W / (m³/s) = J/m³]
|
||||
```
|
||||
|
||||
More commonly expressed as:
|
||||
```
|
||||
SE = P / Q [kWh/m³] (with appropriate unit conversion)
|
||||
```
|
||||
|
||||
Where:
|
||||
- P = electrical power input (kW)
|
||||
- Q = volumetric flow rate (m³/h)
|
||||
- SE = P / Q · (1/1000) for kWh/m³ when P in W and Q in m³/s
|
||||
|
||||
### Wire-to-Water Efficiency
|
||||
```
|
||||
η_total = η_motor · η_VFD · η_pump = (ρ · g · Q · H) / P_electrical
|
||||
```
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. Karassik, I.J. et al. "Pump Handbook" 4th ed. — McGraw-Hill (comprehensive pump engineering reference)
|
||||
2. Europump/Hydraulic Institute (2001). "Pump Life Cycle Costs: A Guide to LCC Analysis for Pumping Systems"
|
||||
3. Engineering Toolbox — "Affinity Laws for Pumps" (https://www.engineeringtoolbox.com/affinity-laws-d_408.html)
|
||||
4. Hydraulic Institute Standards (HI 9.6.1 — Rotodynamic Pumps Guideline for NPSH Margin)
|
||||
5. Gülich, J.F. (2014). "Centrifugal Pumps" 3rd ed. — Springer (theoretical foundation)
|
||||
130
third_party/docs/settling-models.md
vendored
Normal file
130
third_party/docs/settling-models.md
vendored
Normal file
@@ -0,0 +1,130 @@
|
||||
# Sludge Settling & Clarifier Models
|
||||
|
||||
> **Used by**: `biological-process-engineer` agent, `settler` node
|
||||
> **Validation**: Verified against Takacs et al. (1991), Vesilind (1968), and Burger-Diehl framework publications
|
||||
|
||||
## Vesilind Model — Zone Settling Velocity
|
||||
|
||||
**Source**: Vesilind, P.A. (1968). "Design of Prototype Thickeners from Batch Settling Tests." Water Sewage Works, 115, 302-307.
|
||||
|
||||
### Equation
|
||||
|
||||
```
|
||||
v_s = v_0 · exp(-k · X)
|
||||
```
|
||||
|
||||
Where:
|
||||
- v_s = settling velocity (m/h)
|
||||
- v_0 = maximum initial settling velocity (m/h)
|
||||
- k = settling parameter (m³/kg or L/g)
|
||||
- X = suspended solids concentration (kg/m³ or g/L)
|
||||
|
||||
### Typical Parameter Ranges for Municipal Wastewater
|
||||
|
||||
| Parameter | Typical Range | Unit | Notes |
|
||||
|-----------|---------------|------|-------|
|
||||
| v_0 | 4 – 12 | m/h | ~7.8 m/h is a commonly observed average |
|
||||
| k | 0.3 – 0.8 | m³/kg | Correlates with SVI; higher SVI → higher k |
|
||||
|
||||
### SVI Correlation
|
||||
|
||||
The settling parameter k can be estimated from Sludge Volume Index:
|
||||
- k ≈ 0.16 + 0.003 · SVI (for SVI in mL/g, k in m³/kg)
|
||||
- Better correlations use SSVI (Stirred SVI) or DSVI (Diluted SVI)
|
||||
|
||||
### Limitations
|
||||
- Only describes **zone settling** (hindered settling of a blanket)
|
||||
- Does not capture compression settling at high concentrations
|
||||
- Does not model the clarification zone (low-concentration region above blanket)
|
||||
|
||||
## Takacs Model — Double-Exponential Settling
|
||||
|
||||
**Source**: Takacs, I., Patry, G.G., Nolasco, D. (1991). "A dynamic model of the clarification-thickening process." Water Research, 25(10), 1263-1271.
|
||||
|
||||
### Equation
|
||||
|
||||
```
|
||||
v_s = v_0 · (exp(-r_h · (X - X_min)) - exp(-r_p · (X - X_min)))
|
||||
```
|
||||
|
||||
Where:
|
||||
- v_s = settling velocity (m/h)
|
||||
- v_0 = maximum Vesilind settling velocity (m/h)
|
||||
- r_h = hindered settling parameter (m³/kg)
|
||||
- r_p = flocculent settling parameter (m³/kg)
|
||||
- X = suspended solids concentration (kg/m³)
|
||||
- X_min = non-settleable fraction (kg/m³)
|
||||
|
||||
### Key Innovation
|
||||
The double-exponential form captures **both** the clarification zone (low concentrations, dominated by the r_p term) and the thickening zone (high concentrations, dominated by the r_h term). This allows simulation of the complete solids profile from effluent to underflow.
|
||||
|
||||
### Typical Parameter Values
|
||||
|
||||
| Parameter | Typical Range | Default | Unit |
|
||||
|-----------|---------------|---------|------|
|
||||
| v_0 | 4 – 12 | 7.5 | m/h |
|
||||
| r_h | 0.3 – 0.8 | 0.576 | m³/kg |
|
||||
| r_p | 2.0 – 6.0 | 2.86 | m³/kg |
|
||||
| X_min | 0 – 0.1 | 0.01 | kg/m³ |
|
||||
|
||||
### Sensitivity
|
||||
- **r_p** is the most sensitive parameter — it governs effluent suspended solids
|
||||
- Takacs et al. recommend finding r_p by simulation/calibration
|
||||
- v_0 and r_h primarily affect the sludge blanket position and underflow concentration
|
||||
|
||||
### 1D Layer Model Implementation
|
||||
The settler is divided into N horizontal layers (typically 10-30). For each layer:
|
||||
1. Calculate settling velocity from local concentration
|
||||
2. Apply solids flux theory (gravity flux + bulk flux)
|
||||
3. Update concentration via mass balance
|
||||
4. Handle feed layer, overflow, and underflow boundary conditions
|
||||
|
||||
## Burger-Diehl Framework — PDE-Based 1D Settler
|
||||
|
||||
**Source**: Burger, R., Diehl, S. and various co-authors (2011-present). Multiple publications developing the framework.
|
||||
|
||||
### Key Characteristics
|
||||
- Based on rigorous **partial differential equation** theory (hyperbolic-elliptic PDE)
|
||||
- Accounts for hindered settling, compression settling, and inlet dispersion
|
||||
- Every implementation detail is consistent with PDE theory (unlike ad-hoc layer models)
|
||||
- More realistic prediction of underflow sludge concentration
|
||||
- Essential for accurate wet-weather modelling
|
||||
|
||||
### Advantages Over Takacs Layer Model
|
||||
- Proper handling of compression settling (important at high MLSS)
|
||||
- Mathematically rigorous — convergence guaranteed
|
||||
- Better sludge blanket dynamics during storm events
|
||||
- Can be extended with reactive terms (ASM1 biokinetics inside settler)
|
||||
|
||||
### When to Use Which Model
|
||||
|
||||
| Scenario | Recommended Model |
|
||||
|----------|-------------------|
|
||||
| Steady-state design | Vesilind + flux theory |
|
||||
| Dynamic simulation (standard) | Takacs 1D layer model |
|
||||
| Wet-weather / high-MLSS dynamics | Burger-Diehl PDE model |
|
||||
| Quick estimation | Vesilind with SVI correlation |
|
||||
|
||||
## Flux Theory for Clarifier Design
|
||||
|
||||
The solids flux approach combines the gravity settling flux with the bulk (underflow) flux:
|
||||
|
||||
```
|
||||
J_total = J_gravity + J_bulk = v_s(X) · X + Q_u/A · X
|
||||
```
|
||||
|
||||
Where:
|
||||
- J_total = total solids flux (kg/m²/h)
|
||||
- v_s(X) = settling velocity at concentration X (from Vesilind or Takacs)
|
||||
- Q_u = underflow rate (m³/h)
|
||||
- A = clarifier surface area (m²)
|
||||
|
||||
The **limiting flux** determines the maximum solids loading rate — operating above this causes blanket rise and eventual washout.
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. Vesilind, P.A. (1968). "Design of Prototype Thickeners from Batch Settling Tests." Water Sewage Works, 115, 302-307.
|
||||
2. Takacs, I., Patry, G.G., Nolasco, D. (1991). "A dynamic model of the clarification-thickening process." Water Res. 25(10), 1263-1271.
|
||||
3. Burger, R., Diehl, S., Nopens, I. (2011). "A consistent modelling methodology for secondary settling tanks in wastewater treatment." Water Res. 45(6), 2247-2260.
|
||||
4. Torfs, E. (2015). "Different settling regimes in secondary settling tanks." PhD thesis, Ghent University.
|
||||
5. Daigger, G.T. (1995). "Development of refined clarifier operating diagrams using an updated settling characteristics database." Water Environment Research, 67(1), 95-100.
|
||||
157
third_party/docs/signal-processing-sensors.md
vendored
Normal file
157
third_party/docs/signal-processing-sensors.md
vendored
Normal file
@@ -0,0 +1,157 @@
|
||||
# Sensor Signal Conditioning & Data Quality
|
||||
|
||||
> **Used by**: `instrumentation-measurement` agent, `measurement` node
|
||||
> **Validation**: Verified against IEC 61298, sensor manufacturer literature, and signal processing references
|
||||
|
||||
## Signal Conditioning Pipeline
|
||||
|
||||
```
|
||||
Raw Signal → Scaling → Filtering → Outlier Rejection → Quality Flagging → Output
|
||||
```
|
||||
|
||||
## Scaling: Engineering Unit Conversion
|
||||
|
||||
### 4-20 mA Standard
|
||||
```
|
||||
value = range_min + (I - 4) / (20 - 4) · (range_max - range_min)
|
||||
```
|
||||
|
||||
Where I is the measured current in mA.
|
||||
|
||||
### Key Rules
|
||||
- 0 mA = wire break (fault condition)
|
||||
- < 4 mA = under-range or fault
|
||||
- 4 mA = range minimum (0%)
|
||||
- 20 mA = range maximum (100%)
|
||||
- > 20 mA = over-range or fault
|
||||
- NAMUR NE43 recommends 3.8 mA and 20.5 mA as fault thresholds
|
||||
|
||||
## Filtering Methods
|
||||
|
||||
### Moving Average
|
||||
```
|
||||
y[k] = (1/N) · Σ x[k-i] for i = 0 to N-1
|
||||
```
|
||||
- Simple, effective for white noise
|
||||
- Introduces phase lag proportional to (N-1)/2 samples
|
||||
- Good for steady-state signals, poor for fast transients
|
||||
|
||||
### Exponential Moving Average (EMA)
|
||||
```
|
||||
y[k] = α · x[k] + (1-α) · y[k-1]
|
||||
```
|
||||
Where α = 2/(N+1) or α = Δt/(τ + Δt) for time-constant-based tuning.
|
||||
- Less memory than moving average
|
||||
- Equivalent to first-order low-pass filter
|
||||
- τ (time constant) sets the cutoff frequency: f_c = 1/(2π·τ)
|
||||
|
||||
### Savitzky-Golay Filter
|
||||
- Fits a polynomial to a window of data points, uses the polynomial value as the filtered output
|
||||
- Preserves higher-order moments (peaks, edges) better than moving average
|
||||
- Configurable by window size and polynomial order
|
||||
- Typical: window = 5-11 points, order = 2-3
|
||||
|
||||
## Outlier Detection
|
||||
|
||||
### Z-Score Method
|
||||
```
|
||||
z = |x - μ| / σ
|
||||
outlier if z > threshold (typically 3.0)
|
||||
```
|
||||
- Assumes normal distribution
|
||||
- Sensitive to the outliers themselves (they inflate σ)
|
||||
|
||||
### Modified Z-Score (MAD-based)
|
||||
```
|
||||
MAD = median(|x_i - median(x)|)
|
||||
modified_z = 0.6745 · (x - median(x)) / MAD
|
||||
outlier if |modified_z| > threshold (typically 3.5)
|
||||
```
|
||||
- Robust to outliers (uses median instead of mean)
|
||||
- **Recommended for process measurements** where occasional spikes are common
|
||||
- 0.6745 is the 75th percentile of the standard normal distribution
|
||||
|
||||
### IQR Method
|
||||
```
|
||||
Q1 = 25th percentile, Q3 = 75th percentile
|
||||
IQR = Q3 - Q1
|
||||
outlier if x < Q1 - 1.5·IQR or x > Q3 + 1.5·IQR
|
||||
```
|
||||
- Non-parametric, no distribution assumption
|
||||
- Common in exploratory data analysis
|
||||
- Less suitable for real-time streaming (needs window of data)
|
||||
|
||||
## NRMSE for Drift Detection
|
||||
|
||||
Normalized Root Mean Square Error compares a recent measurement window against a reference window to detect sensor drift.
|
||||
|
||||
### Calculation
|
||||
```
|
||||
RMSE = √(Σ(x_i - x_ref_i)² / N)
|
||||
NRMSE = RMSE / (x_max - x_min) or RMSE / x_mean
|
||||
```
|
||||
|
||||
### Thresholds (typical for process sensors)
|
||||
| NRMSE Range | Quality | Action |
|
||||
|-------------|---------|--------|
|
||||
| 0 – 0.05 | Good | Normal operation |
|
||||
| 0.05 – 0.15 | Uncertain | Flag for review, increase monitoring |
|
||||
| 0.15 – 0.30 | Poor | Alarm, reduce weight in control loops |
|
||||
| > 0.30 | Bad | Remove from control, maintenance required |
|
||||
|
||||
### Reference Window Selection
|
||||
- Calibration data (gold standard)
|
||||
- Post-maintenance baseline
|
||||
- Rolling reference from a known-good period
|
||||
- Multi-sensor cross-validation
|
||||
|
||||
## Sensor Accuracy Classes
|
||||
|
||||
### IEC 61298 Framework
|
||||
IEC 61298 "Process measurement and control devices — General methods and procedures for evaluating performance" defines standardized test methods for evaluating sensor accuracy under reference and influence conditions.
|
||||
|
||||
### Key Performance Metrics
|
||||
- **Accuracy**: Closeness of measured value to true value (includes systematic and random errors)
|
||||
- **Repeatability**: Closeness of successive measurements under identical conditions
|
||||
- **Hysteresis**: Maximum difference between upscale and downscale readings
|
||||
- **Linearity**: Maximum deviation from a straight line between zero and span
|
||||
- **Deadband**: Smallest change in input that produces a detectable output change
|
||||
|
||||
### Common Accuracy Specifications
|
||||
| Sensor Type | Typical Accuracy | Response Time |
|
||||
|-------------|-----------------|---------------|
|
||||
| Pressure transmitter | ±0.04 – 0.1% FS | < 100 ms |
|
||||
| Flow meter (electromagnetic) | ±0.2 – 0.5% of reading | 1-3 s |
|
||||
| Temperature (RTD/Pt100) | ±0.1 – 0.3°C | 5-30 s (depends on housing) |
|
||||
| Level (ultrasonic) | ±0.25% FS | 1-5 s |
|
||||
| pH | ±0.02 – 0.1 pH | 10-60 s |
|
||||
| Dissolved oxygen | ±1-2% of reading | 30-90 s (membrane) |
|
||||
| Turbidity (nephelometric) | ±2% of reading | 5-15 s |
|
||||
| Ammonia (ion-selective) | ±5-10% of reading | 60-180 s |
|
||||
|
||||
## Sensor States and Warmup
|
||||
|
||||
### State Machine
|
||||
```
|
||||
Maintenance → Warmup → Active → Cooldown → Maintenance
|
||||
```
|
||||
|
||||
### Warmup Behavior
|
||||
- **Duration**: Varies by sensor type (seconds for pressure, minutes for pH, hours for dissolved oxygen)
|
||||
- **During warmup**: Measurements flagged as "uncertain" quality
|
||||
- **Completion criterion**: Readings stabilize within defined tolerance for a minimum duration
|
||||
- **EVOLV convention**: Warmup state prevents measurements from propagating to control loops
|
||||
|
||||
### Stabilization Detection
|
||||
```
|
||||
stable if std_dev(last_N_readings) < threshold for T_stable seconds
|
||||
```
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. IEC 61298 series (2008). "Process measurement and control devices — General methods and procedures for evaluating performance"
|
||||
2. IEC 61326-2-3. "Electrical equipment for measurement, control and laboratory use — EMC requirements — Part 2-3: Particular requirements — Transducers with integrated or remote signal conditioning"
|
||||
3. NAMUR NE43 (2003). "Standardization of the Signal Level for the Failure Information of Digital Transmitters"
|
||||
4. Bently Nevada / Baker Hughes. "Fundamentals of Rotating Machinery Diagnostics"
|
||||
5. Oppenheim, A.V. & Willsky, A.S. (1997). "Signals & Systems" 2nd ed., Prentice Hall
|
||||
6. Press, W.H. et al. (2007). "Numerical Recipes" 3rd ed., Chapter 14 (Savitzky-Golay filters)
|
||||
0
third_party/docs/sources/.gitkeep
vendored
Normal file
0
third_party/docs/sources/.gitkeep
vendored
Normal file
21
third_party/docs/sources/README.md
vendored
Normal file
21
third_party/docs/sources/README.md
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Source Documents
|
||||
|
||||
Place actual scientific papers, standards, and technical manuals here. Reference them from the summary files in the parent directory.
|
||||
|
||||
## Suggested Sources to Add
|
||||
|
||||
- IWA Scientific and Technical Report No. 1 — ASM1 (Henze et al., 1987)
|
||||
- IWA Scientific and Technical Report No. 3 — ASM2d (Henze et al., 1999)
|
||||
- IWA Scientific and Technical Report No. 9 — ASM3 (Gujer et al., 1999)
|
||||
- Takacs et al. (1991) "A dynamic model of the clarification-thickening process" Water Res. 25(10), 1263-1271
|
||||
- Astrom & Hagglund (2006) "Advanced PID Control" ISA
|
||||
- Karassik et al. "Pump Handbook" McGraw-Hill
|
||||
- Europump/Hydraulic Institute "Pump Life Cycle Costs"
|
||||
- IEC 62443 series (OT security)
|
||||
- IEC 61298 series (process measurement)
|
||||
- EU Directive 91/271/EEC (Urban Waste Water Treatment)
|
||||
- NIST SP 800-82 Rev 3 (Guide to ICS Security)
|
||||
|
||||
## File Naming Convention
|
||||
|
||||
`<author-year>-<short-title>.pdf` — e.g., `takacs-1991-clarification-thickening.pdf`
|
||||
117
third_party/docs/wastewater-compliance-nl.md
vendored
Normal file
117
third_party/docs/wastewater-compliance-nl.md
vendored
Normal file
@@ -0,0 +1,117 @@
|
||||
# Dutch Wastewater Regulations & Compliance
|
||||
|
||||
> **Used by**: `commissioning-compliance` agent, `biological-process-engineer` agent
|
||||
> **Validation**: Verified against EU Directive 91/271/EEC, Activiteitenbesluit milieubeheer, and Dutch water authority publications
|
||||
|
||||
## Regulatory Framework
|
||||
|
||||
### European Level
|
||||
- **EU Urban Waste Water Treatment Directive 91/271/EEC** — Primary directive governing collection, treatment, and discharge of urban wastewater across all EU member states
|
||||
- **Water Framework Directive 2000/60/EC** — Establishes river basin management and environmental quality standards
|
||||
- **Revised UWWTD (2024)** — Updated directive with stricter nutrient limits and energy neutrality targets
|
||||
|
||||
### Dutch National Level
|
||||
- **Waterwet** (Water Act) — National water management framework
|
||||
- **Waterschapswet** (Water Authority Act) — Governance of regional water authorities
|
||||
- **Activiteitenbesluit milieubeheer** — General rules for environmental activities including wastewater discharge
|
||||
- **Besluit lozing afvalwater huishoudens** — Rules for domestic wastewater discharge
|
||||
|
||||
### Regional Level
|
||||
- **Waterschap Brabantse Delta** — Regional water authority managing the target WWTP
|
||||
- Operates under national framework with site-specific discharge permits (watervergunning)
|
||||
|
||||
## EU UWWTD Effluent Standards (Annex I)
|
||||
|
||||
### Table 1: Secondary Treatment Requirements
|
||||
|
||||
| Parameter | Concentration | Min. Reduction |
|
||||
|-----------|--------------|----------------|
|
||||
| BOD₅ (at 20°C, without nitrification) | 25 mg/L O₂ | 70-90% |
|
||||
| COD | 125 mg/L O₂ | 75% |
|
||||
| TSS (Total Suspended Solids) | 35 mg/L | 90% (>10,000 p.e.) |
|
||||
| TSS (Total Suspended Solids) | 60 mg/L | 70% (2,000-10,000 p.e.) |
|
||||
|
||||
### Table 2: Nutrient Requirements for Sensitive Areas
|
||||
|
||||
| Parameter | Concentration | Min. Reduction |
|
||||
|-----------|--------------|----------------|
|
||||
| Total Phosphorus (10,000-100,000 p.e.) | 2 mg/L P | 80% |
|
||||
| Total Phosphorus (>100,000 p.e.) | 1 mg/L P | 80% |
|
||||
| Total Nitrogen (10,000-100,000 p.e.) | 15 mg/L N | 70-80% |
|
||||
| Total Nitrogen (>100,000 p.e.) | 10 mg/L N | 70-80% |
|
||||
|
||||
*Note: The Netherlands designated its entire territory as a "sensitive area" under the UWWTD, meaning nutrient requirements (Table 2) apply to all significant WWTPs.*
|
||||
|
||||
### Dutch Practice (Often Stricter)
|
||||
Dutch water authorities commonly set stricter limits than the EU minimum:
|
||||
|
||||
| Parameter | Typical Dutch Permit Limit | EU Minimum |
|
||||
|-----------|---------------------------|------------|
|
||||
| N-total | 5-10 mg/L | 10-15 mg/L |
|
||||
| P-total | 0.3-1.0 mg/L | 1-2 mg/L |
|
||||
| BOD₅ | 5-10 mg/L | 25 mg/L |
|
||||
| COD | 50-100 mg/L | 125 mg/L |
|
||||
| TSS | 10-20 mg/L | 35 mg/L |
|
||||
| NH₄-N | 1-2 mg/L | Not specified in UWWTD |
|
||||
|
||||
*Note: Actual permit limits are site-specific. The values above represent common ranges for Dutch WWTPs.*
|
||||
|
||||
## Monitoring and Reporting Obligations
|
||||
|
||||
### Sampling Requirements (UWWTD Annex I, Table 3)
|
||||
| Plant Size | Min. Annual Samples |
|
||||
|------------|-------------------|
|
||||
| 2,000-9,999 p.e. | 12 |
|
||||
| 10,000-49,999 p.e. | 12 |
|
||||
| ≥50,000 p.e. | 24 |
|
||||
|
||||
### Compliance Assessment
|
||||
- Based on **annual averages** for most parameters
|
||||
- A defined number of samples may fail while still meeting compliance (concentration limits)
|
||||
- Percentage reduction assessed against influent loading
|
||||
|
||||
### Reporting Chain
|
||||
1. WWTP operator monitors and reports to water authority
|
||||
2. Water authority (Waterschap) reports to Province
|
||||
3. Province reports to national government (IenW)
|
||||
4. National government reports to European Commission
|
||||
|
||||
## Waterschap Brabantse Delta Context
|
||||
|
||||
### Service Area
|
||||
- Province of Noord-Brabant (western part)
|
||||
- Multiple WWTPs of varying sizes
|
||||
- Mixed urban/agricultural/industrial catchment
|
||||
|
||||
### Key Challenges
|
||||
- Agricultural runoff contributing nutrient loading
|
||||
- Seasonal variations in flow and temperature
|
||||
- Emerging contaminants (pharmaceuticals, microplastics)
|
||||
- Energy efficiency targets (energy-neutral WWTP goal)
|
||||
|
||||
### EVOLV Relevance
|
||||
- Process automation targets: optimizing energy use while maintaining effluent quality
|
||||
- Real-time monitoring: continuous measurement of key parameters (NH₄, NO₃, PO₄, DO, TSS)
|
||||
- Predictive control: using ASM models to anticipate process changes
|
||||
- Reporting support: automated telemetry data for compliance reporting
|
||||
|
||||
## Key Compliance Parameters for EVOLV
|
||||
|
||||
| Parameter | Measurement Method | EVOLV Node | Typical Control Strategy |
|
||||
|-----------|-------------------|------------|-------------------------|
|
||||
| NH₄-N | Ion-selective electrode | measurement | Aeration control (DO setpoint cascade) |
|
||||
| NO₃-N | UV absorption / ISE | measurement | Anoxic zone recirculation control |
|
||||
| PO₄-P | Colorimetric / ISE | measurement | Chemical dosing or Bio-P optimization |
|
||||
| DO | Amperometric / optical | measurement | Blower/diffuser control |
|
||||
| TSS/MLSS | Optical (turbidity) | measurement | Sludge wasting control |
|
||||
| Flow | Electromagnetic | measurement | Hydraulic load monitoring |
|
||||
| Temperature | RTD/Pt100 | measurement | Process rate compensation |
|
||||
|
||||
## Authoritative References
|
||||
|
||||
1. Council Directive 91/271/EEC (1991). "Concerning urban waste water treatment" — Official Journal L 135/40
|
||||
2. Directive (EU) 2024/3019. "Concerning urban wastewater treatment (recast)"
|
||||
3. Activiteitenbesluit milieubeheer — Dutch Activities Decree (environmental management)
|
||||
4. Waterschapswet — Dutch Water Authority Act
|
||||
5. STOWA (Stichting Toegepast Onderzoek Waterbeheer) — Dutch Foundation for Applied Water Research (various publications on WWTP optimization)
|
||||
6. Waterschap Brabantse Delta — Regional water management plans and discharge permits
|
||||
Reference in New Issue
Block a user