Corvus ISR, known for its wide-area motion imagery (WAMI) exploitation solutions, has recently published a detailed public tracker benchmark comparing two distinct models on an identical synthetic scene. This approach leverages perfect ground truth data from synthetic environments, ensuring measurement accuracy and repeatability. Such benchmarks are invaluable in the scientific community for understanding tracker performance under controlled, reproducible conditions.
The benchmark pits the v1 model, a simple, two-pass greedy association tracker, against the more sophisticated v2, featuring a three-tier auction association, velocity-consistency gating, and other enhancements. Both models operate on the same seed (seed 1337), with identical detection generation, sensor models, and metric definitions, differing only in their tracking algorithms. This isolation of variables allows for a clear comparison of tracking methodologies.
In the published results, the v2 model demonstrates significant improvements over v1. For instance, in a scenario with 150 movers at 2 fps, ID switches per minute dropped from 2,042 to 1,183 — a 42.1% reduction. Similar reductions were observed in dense scenes with 400 movers, decreasing from 14,032 to 8,040 (42.7% reduction). These figures highlight how advanced association strategies can mitigate identity errors even under challenging conditions.
Corvus ISR emphasizes the importance of transparency in evaluation. The published metrics include not only improvements but also the residual errors, given that both models still commit thousands of identity errors per minute under stress. The strict ID switch metric used counts every change in track identity, including fragmentations and re-acquisitions, providing a rigorous measure that exceeds typical challenge benchmarks like MOT. This transparency fosters trust and pushes the field toward more robust solutions.
Engineering-wise, the v2 tracker runs at approximately 1.2ms per sensor tick at 400 density, well within real-time constraints. The benchmark is fully reproducible in the live demo, where anyone can run the same test, compare results, and validate the findings without signup or NDA. This openness underpins the scientific value of the published data, demonstrating that even the most advanced tracker still faces challenges, as evidenced by the thousands of errors recorded in synthetic scenes with perfect ground truth.
All data and results are generated within a fully synthetic environment, with no real persons, vehicles, or locations involved—every pixel is simulated. The methodology exemplifies how synthetic scenes enable precise, measurement-based evaluation of tracking algorithms, removing ambiguity and bias inherent in real-world data.
By publishing these failure numbers openly, Corvus ISR advocates for a measurement-driven approach rather than marketing-only success stories. Future tracker developers are encouraged to land their models against this same benchmark, fostering a transparent and rigorous advancement of the field. You can explore the benchmark and see the results firsthand by running it yourself — simply visit the demo and press ‘Run benchmark’.
Bouncie GPS Tracker for Vehicles with Real-Time Location, Route History, Speed Monitoring, Geo-Fence & Accident Notifications – for Fleets & Family – Cancel Anytime
- Real-Time Location Updates: Up-to-the-minute vehicle location tracking
- Unlimited Trip History: Detailed route and trip records
- Speed and Driving Behavior Monitoring: Tracks speed, acceleration, and braking
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
