I spent a good chunk of the past week on a deceptively simple question for a client project: across a large multi-site network with more than seventy virtual network segments provisioned, which ones are actually being used?
It sounds like the kind of thing you should be able to answer in an afternoon. Pull the data, count the hosts, done. It turned out to be a great case study in why one-off answers to network questions are usually wrong, and why the real deliverable is almost never a report — it’s a reporting system.
The first answer was wrong
The obvious approach is to take a snapshot of which devices are talking on each segment and count. I did that. The first pass said dozens of segments were completely dead.
Except they weren’t. Two things were lying to me.
First, the tables I was reading from age out. Devices that were active a few hours ago simply vanish from the data. A snapshot taken at a quiet time of day isn’t the truth — it’s a low-water mark. If you make decommissioning decisions off a single quiet-hour snapshot, you’re going to break things for real users.
Second, the opposite problem: some segments looked alive when they weren’t. The firewalls on this network respond on nearly every segment as part of normal plumbing, so a naive count sees “activity” everywhere. I had to teach the tooling to recognize infrastructure noise and filter it out, so the numbers reflected actual users rather than the network talking to itself.
Once both corrections were in, the picture got crisp: roughly forty segments with thousands of real users on them, about thirty segments provisioned but sitting empty, and one that was genuinely dead. That’s an actionable answer instead of a misleading one.
One report became a pipeline
Because a single snapshot can’t be trusted, the answer had to come from trends. So instead of handing over a document, I built an automated pipeline: new data drops into a folder, and a self-contained interactive report regenerates itself — latest picture, full history with trend lines for every segment, and per-site drill-downs.
The part I like most is the callouts. The report automatically flags which segments newly came alive since the last collection, which ones went quiet, and which moved the most. Some segments turned out to have a daily rhythm — busy during working hours, silent overnight — which is exactly the kind of thing a single snapshot can never show you and a trend line makes obvious.
The feature nobody asks for: refusing to report garbage
Midway through, one data collection came back silently broken. Stray formatting in the input had truncated things upstream, and the raw output looked plausible while containing almost nothing. An earlier version of the tooling would have happily rendered a beautiful report announcing that the entire network was unused.
That’s the failure mode that scares me most in automated reporting — not crashes, but confident nonsense. So I built validation in front of everything: the pipeline now counts what it actually received, checks coverage against what it expected, and if the input is broken it says so with a big warning instead of producing a plausible-looking page of zeros. Later in the week that guard earned its keep a second time, catching an incomplete collection before it could contaminate the historical trends.
Why this matters beyond one network
The pattern here isn’t really about networks. It’s about any recurring operational question: the first honest answer is usually “it depends on when you ask,” the raw data lies in at least two directions, and the valuable thing isn’t the analysis — it’s an automated system that repeats the analysis, tracks change over time, and knows when its own inputs can’t be trusted.
That last part is where most homegrown reporting falls down. Anyone can generate a dashboard. Building one that tells you when it shouldn’t be believed is what makes it safe to act on.
If you’ve got operational questions like this — data you’re sitting on but can’t quite trust, or reports someone regenerates by hand every week — this is exactly the kind of thing I build. Take a look at what I’m doing with AI-assisted automation, or get in touch and tell me about the question you keep having to answer manually.