Case study 02 / 21Data Science
A test bench that scores pipeline leak detectors against known leaks
Writes synthetic leaks with a known answer into a lab OPC UA server and scores three detectors per run: the improved model led 83.9 to 69.3.
Stack
- n8n
- TypeScript
- Node.js
- OPC UA
- InfluxDB
- PowerShell
01 · How it works
- TodaytoChanged in this work
- Data flow
What it does
A node-based flow app built on n8n, with n8n's code untouched: my own node reads an industrial OPC UA server read-only, and chained calculation nodes turn its signals into results such as sensor scores and leak alarms. On top of it I built a test bench that writes synthetic pipeline cases, each with a known answer, into a lab OPC UA server read by three detectors at once: the existing detector as the baseline, a faithful 1:1 port of it in nodes, and an improved model.
Read more
Every run is scored 0–100 and drawn as a timeline, so the team can see which detector to trust and whether a model change helped.
8 connections· select a part to trace it
- Case generator to Lab OPC UA server: one row per second
- Case generator to Scorer (0–100): the known answer
- Lab OPC UA server to Existing detector: reads every 0.5 s
- Lab OPC UA server to 1:1 port in n8n nodes: read-only, every 10 s
- Lab OPC UA server to Improved model: read-only, every 2 s
- Existing detector to Scorer (0–100): alarm state and place
- 1:1 port in n8n nodes to Scorer (0–100): one line per tick
- Improved model to Scorer (0–100): state, place and rate
02Results
- R01Higher is better
Mean score out of 100, same 48 runs, baseline vs improved
Before 69.3, after 83.9, +21%. Higher is better.BeforeAfter - R02Lower is better
Median time to confirm a leak, baseline vs improved
Before 458 s, after 165 s, 2.8× lower. Lower is better.BeforeAfter - R03Higher is better
Leaks detected out of 33, baseline vs improved
Before 27, after 30, +11%. Higher is better.BeforeAfter
What it achieved
- Test cases: none with a known answer → a 30-template catalog with edge cases and 51 scored runs, each with a second-by-second timeline
- Mean score out of 100 on the 48 runs with a baseline reading: 69.3 for the baseline → 83.9 for the improved model, which also saw 30 of 33 leaks to its 27 and raised no false alarm
- Median time to confirm a leak: 458 s → 165 s, with a median location error about a third of the baseline's
- Existing detection logic: only inside a desktop application → also as a 1:1 port in 57 flow nodes, which matched its leak state on every recorded minute and on 88 % of bench minutes
Read more
- Model changes: tried on one test leak → 11 changes, each checked by replaying the runs it touched; the latest full replay of all 51 runs, for both node models, left 3 scores better, 99 equal and 0 worse
- CPU per pipeline: 163 % of a core for the desktop application → 49 % for the whole flow app with both node models and their pages, 20 % for the models alone, measured on different machines
The challenge
A leak detector can only be judged against leaks whose place, size and start time are known, and there were none to judge with. The development signals were frozen: 0 of 225 moved in 55 minutes. The last replay of a recorded leak scenario wrote one row every 14 s instead of every second, and no output of the existing detector reached the server in any of its 3,607 rows, so not even a baseline could be read.
Read more
That detector runs live only, so a run could not be replayed afterwards.
Any claim that a new model was better would have rested on opinion, and the port of the existing logic had only been checked on three recordings, none with an event at a known time.
What I learned
A bench can mislead as easily as it measures. My first synthetic leak moved only the flow meters, instantly, and then a pressure signature drawn backwards put it at a section end for all three detectors; once pressures followed friction and the wave speed, location became testable. I score the median estimate after detection, not the first guess, and count only complete runs; a run without a value reruns on its own.
Read more
Replay dropped ideas too: a location rule that fixed a rupture case broke four minutes later, so I replaced it with a step fitted across all the sections between two flow meters. Synthetic data is not the plant: a controlled real leak comes before any decision.
Stack
- n8n
- TypeScript
- Node.js
- OPC UA
- InfluxDB
- PowerShell
Skills
- Pipeline leak detection
- Test bench and scoring design
- Synthetic data from a real schema
- Sequential change detection
- Regression testing by replay
