From pixels to kilometres

A camera sees the road at a slant: a metre far away covers far fewer pixels than a metre up close. Speed in pixels is therefore meaningless; the same car seems to slow down as it heads for the horizon. But the road is a flat plane, and the relation between two images of a plane is captured exactly by a single 3×3 matrix, a homography. To find it you only need to mark the four corners of a rectangle of known size on the road.

With a correct calibration the estimated speeds sit within a few percent of the truth. Moving a handle by only a few pixels already inflates the errors, because distant pixels carry many more metres. That is why panoptes does not leave calibration to hand clicks alone; it cross-checks against known measures like lane width and reports with a confidence range.

The detector produces boxes and confidence scores every frame. The tracker’s job is to link those boxes to the same car across frames. The hard moment is when a car passes behind a pole or another vehicle: the detector either misses it or sees it with low confidence. A naive tracker throws low-confidence boxes away, loses the track and hands the car a new identity when it reappears.

ByteTrack’s idea is simple and strong: first match high-confidence boxes to existing tracks, then try matching the leftover tracks to the low-confidence boxes too. Meanwhile tracks are carried forward by a motion model even while invisible. panoptes implements this from scratch, with a clean licence.

Detection and tracking are just raw material. The real question is “what happened?”: who broke the speed limit, which truck is in the left lane, which car stopped in a no-stopping zone. panoptes has these rules written in a declarative language instead of buried in code; changing a rule needs no rebuild. The editor below is a miniature of the same idea: every line is a rule and the events change as you type.