Case study 07 / 17Data Engineering
Mass-conservation auditing for a pipeline and channel spill model
An auditor that follows the mass from the leak model to every drawn frame: three error layers fixed, four invariants enforcing balance.
Stack
- TypeScript
- Node.js
- CesiumJS
- InfluxDB
- MongoDB
- Puppeteer
01 · How it works
- TodaytoChanged in this work
- Data flow
What it does
In the monitoring platform, a pipeline system and a channel each run a simulation that estimates how much fluid escapes and renders where it goes on a 3D map for operators. I built an auditor that traces the mass from the volume fed to the spill engine to every drawn frame, measured on the geometry the map actually renders.
Read more
Every frame now reconciles with the computation, and four invariants keep it that way.
6 connections· select a part to trace it
- Leak detection model to Event aggregator: flow, volume, start time
- Event aggregator to Spill engine: volume released while open
- Spill engine to Render package: cells, widths per frame
- Render package to 3D map service: draws each frame
- Event aggregator to Balance auditor: attributed vs released
- Render package to Balance auditor: drawn vs computed volume
02Results
- R01Lower is better
Drawn volume ÷ released volume
Before 4.3×, after 1×, 4.3× lower. Lower is better.BeforeAfter - R02Lower is better
Worst frame error, drawn vs computed
Before 538%, after 1.1%, 489× lower. Lower is better.BeforeAfter - R03Lower is better
Overflow release ÷ channel flow
Before 1.8×, after 1×, −44%. Lower is better.BeforeAfter
What it achieved
- Drawn spill volume: up to 4.3× the actual release → equal to it
- Frames that reconcile with the computation: 22% → 100%
- Channel overflow: up to 1.8× the channel's flow → bounded by the flow it carries
Read more
- Dilute runoff: a sheet about 34× too wide → a path computed from slope and roughness
- Regression protection: none → four mass-balance invariants, in the test suite and in the browser
The challenge
Nothing raised an error, which made this a verification problem rather than a debugging one: the deposit oscillated instead of growing, because mass was created and destroyed in three separate layers.
The leak model reports volume as flow multiplied by the time since it believes the leak began, so events after a data gap inherited the whole counter and a dip in flow was counted twice; on one line the map drew 4.3 times the volume that had left the pipe.
Read more
Frames reused stale widths, the water split lost about 5%, and fluid leaving the modeled terrain was never accounted for, so 78% of frames could not be checked at all. On the channel, a single overflow could release 1.8 times the channel's own flow.
What I learned
Audit on the geometry the user actually sees: following the mass all the way to the pixel exposed errors that no single component showed on its own. I left the leak model's volume untouched and displayed it beside the engine's, so operators compare two independent estimates instead of trusting one.
Read more
The same instrument surfaced limits I deliberately left for later, such as a velocity floor and sensitivity to mesh resolution, because changing them would shift results already in use. Thresholds deserve the rigor of equations: one slope cut-off made the runoff oscillate, and a continuous ramp fixed it.
Stack
- TypeScript
- Node.js
- CesiumJS
- InfluxDB
- MongoDB
- Puppeteer
Skills
- Simulation auditing
- Numerical modeling
- Invariant-based testing
- Open-channel hydraulics
- Data pipeline debugging
