Half a step early

The finding arrived politely, in a code review: every timestamp vidqa had ever reported was wrong. Not scattered wrong — systematically wrong, every label early by half a sampling step. Earlier that same day, a different review had read the same lines and passed them. And sitting in my own test suite was a test whose docstring proudly said it had pinned the current behavior as the specification.

vidqa is a small open-source tool with one job: answer questions about a test recording, deterministically, from the command line. When did the payment error appear on screen. Did this run diverge from the last one, and at what second. Gate this CI run on the confirmation showing before a deadline. Every answer is a timestamp, or a pass/fail verdict derived from one. If the timestamps run early by half a step, every deadline gate is quietly more lenient than its rules file says — and a short-lived error can slip between samples entirely.

The bug had armor because I built the armor. vidqa samples a video by running it through ffmpeg's fps filter, which picks one frame per interval. Weeks earlier I had noticed odd behavior at window boundaries, explained it to myself with a plausible mechanism — the filter emits the frame just before each instant — and wrote a test that locked that explanation in. I measured a symptom, invented a mechanism, and pinned the invention.

Then the reviews split. I run automated review passes over changes like this one — the first, a quick pass with Claude Sonnet, read the sampling code and found nothing to flag. A deeper pass with the family's strongest model at maximum reasoning effort claimed the opposite of my test: the fps filter's default rounding emits the source frame from roughly (i + 0.5) × step, so every label at i × step runs half a step early. Two confident, contradictory readings of the same eight-line filter chain, and the worst available move was to pick whichever sounded more sure of itself.

There is a cheaper arbiter than argument: a video that tells you which frame you are looking at. I generated an eight-second probe clip, thirty frames per second, in which the brightness of every pixel equals the frame's own index — frame zero is black, and each frame after it is one shade brighter. Decode a sample, read its brightness, and the frame confesses exactly where in the source it came from.

The first version of the probe lied to me. Raw brightness came back rescaled — video pipelines store luma in a compressed range by convention — and the naive readout produced impossible answers, including a frame index past the end of the video. The probe needed the same skepticism I was aiming at the filter. So the final version calibrates itself: it first decodes every frame through the identical path and builds a lookup table of what each index actually decodes to, and only then matches sampled frames against that table instead of reading them directly.

With the probe calibrated, the verdict took one run — reproduced again this morning on ffmpeg 8.0.1 before writing this post. At a sampling step of one and a half seconds, every sample the default filter produced came from well after its label. The deeper review was right; the earlier review and my pinned test were both wrong.

The figure is the whole bug. The sample labeled 1.5 seconds was actually the frame from 2.23 seconds. In the regression suite that number has teeth: a fixture flashes an error message during a one-second window, and a scan at this step sampled just past the window — so the tool reported the error never appeared at all. Not late — missed.

The fix is almost insultingly small. The fps filter accepts a rounding mode, and round=up changes the selection rule to: emit the last frame at or before each instant — the frame a viewer was actually seeing at that moment. One shared helper now builds the filter clause, eight modules call it, and every command that samples — text search, failure locate, CI gates, load timing, run-to-run diff — inherited the correction at once.

The satisfying part was rewriting the test that had protected the bug. Same file, same fixture, opposite allegiance: it now asserts that a scan finds the flashed error at exactly the labeled instant, and that a coarser scan is honestly blind to a window that fits between its samples — with a docstring telling the next reader to pick a step no larger than the shortest state they need to catch. The test that once enshrined folklore now enforces a measurement.

The lesson I keep relearning: a test suite is a record of your beliefs, not a record of the world. Writing a test to pin current behavior feels rigorous — it is standard advice, and I still endorse it — but the pin is only as true as the observation behind it, and mine was an invented mechanism wearing a measured symptom. A wrong belief with a green test is worse than no test, because every future reader inherits your confidence along with your bug.

It also settled how I treat disagreeing reviewers, human or machine. When two competent readings of the same code contradict each other, the move is not to adjudicate the argument but to make the system testify. The probe is forty lines of Python and cost less time than carefully re-reading either review — and it did not have an opinion.

The correction is tagged and released as vidqa 0.2.0, alongside a suite of 196 tests built so that no verdict path can change again without a test flipping. The half step is gone; the habit of measuring before pinning is the part I get to keep.