Three times in one week, comparing two endpoints hid a step change
A two-endpoint comparison cannot distinguish growth from a step discontinuity or a taxonomy change. We hit it three times: a marketplace category, an OpenStreetMap query, and a vendor statistic.
- How it was measured
- Each was caught by pulling the full series instead of the endpoints. In every case the series showed something the comparison structurally could not.
- Evidence current as of
- This corrects us
- All three were our own errors, and the third happened after we had documented the first two.
Comparing a number in year A to the same number in year B feels like measurement. It is not, and here is what it hides.
One: a category that did not exist
Marketplace sales of YouTube-monetised businesses went from 0 in 2022 to 28 across 2025-26. We were one sentence from reporting that as where the audience had moved.
The full series: 0, 0, 0, 0, 0, 6, 16, 12.
A run of zeros followed by sudden onset is a new category label, not growth. The broker started tagging YouTube in 2024. The two-window comparison structurally could not show that, because both windows sat on the same side of the change.
Two: a query that discarded two-thirds of the data
We built a sample of RV parks from OpenStreetMap and found 101 across three states, against an assumed universe of 1,000-2,500. That looked like a finding about data availability, and it was reported as one.
It was a bug. Bounding boxes covered sub-regions rather than states, and — the real error — the query asked for nodes. Globally, tourism=caravan_site is 11,818 nodes against 24,233 ways, because parks are mapped as polygons. Ground truth for one state was 198 against the 27 we reported. A 7x undercount, presented as a plan-changing discovery.
Three: a vendor's own framing
A search-analytics vendor reported one of its metrics rising from 8.15% to 18.57% over twelve months. We quoted it.
The thirteen-point monthly series shows a single step in April-May 2025, then seventeen months flat to declining, with the metric peaking three months after the step. Nothing in the vendor's write-up mentions the discontinuity, and nothing rules out a change in its own classifier.
What to do instead
Always pull the full series. If an API or a report gives you endpoints, ask for the points between them; if it will not, treat the comparison as a hypothesis rather than a result.
Watch for onset patterns. A clean run of zeros before a sudden start is almost always a definitional change — a new label, a new field, a new tracking method — and almost never a real phenomenon appearing from nothing.
The third instance is the one worth sitting with. It happened after the first two were written down, to someone who had read them, on the item flagged as most important. Knowing about a bias does not protect you from it. Only re-pulling the data does.
Tracked in our ledger as TWO-ENDPOINT-TRAP, YOUTUBE-SIGNAL-ARTEFACT, FRAME-QUERY-BUG. If a number here is wrong, tell us and we will correct it in place and say what changed.