The dashboard that watches the pipeline. Each ingestion process reports its last heartbeat, its cadence measured against what's expected, how many files it's behind on, and the exact error that stopped it.
A dashboard that stops receiving data and doesn't say so is worse than having no dashboard at all: it still looks fine, and people keep deciding based on it. This is the one that says so.
The whole suite runs on resident processes that watch folders, clean up files, and push them into the integration layer. They're invisible while they work and catastrophic when they fail silently. This dashboard puts all of them on one screen with a readable status:
A Python agent walks the machine where the watchers live and reports facts, not judgments: how many files are waiting and since when, when it last archived something, when it last wrote to its log, what cadence was measured for it and with how many samples, whether the process is alive, and what its last error was.
The traffic-light status isn't decided by the agent: it's decided by the dashboard. That's a design rule of the system, not an aesthetic choice — it means adjusting a threshold is a one-line change in the dashboard, not logging into a production machine to touch a process that's running.
Same production file. The fixture delivers the raw facts just like the real agent does, and the dashboard still calculates the traffic-light status: changing one last-processed date is enough for it to recompute the status on its own, which is exactly the property this is meant to show.
What's described here is the real system in production. This page is a portfolio replica: same interface and same logic, with the data layer swapped for a made-up data generator. It contains no client data, credentials, or internal addresses. · Back to the portfolio