<- Back to Datasets

TAO

A Large-Scale Benchmark for Tracking Any Object

TAO

TAO is a federated dataset for Tracking Any Object, containing 2,907 high resolution videos, captured in diverse environments, which are half a minute long on average. We adopt a bottom-up approach for discovering a large vocabulary of 833 categories, an order of magnitude more than prior tracking benchmarks. To this end, we ask annotators to label tracks for objects that move at any point in the video, and give names to them post factum. Our vocabulary is both significantly larger and qualitatively different from existing tracking datasets. To ensure scalability of annotation, we employ a federated approach that focuses manual effort on labeling tracks for those relevant objects in a video (e.g., those that move). We perform an extensive evaluation of state-of-the-art tracking methods and make a number of important discoveries regarding large-vocabulary tracking in an open-world. In particular, we show that existing single- and multi-object trackers struggle when applied to this scenario, and that detection-based, multi-object trackers are in fact competitive with user-initialized ones. We hope that our dataset and analysis will boost further progress in the tracking community.

View this Dataset
->
View author website
Task
Video Object Tracking
Annotation Types
2907
Items
833
Classes
2907
Labels
Models using this dataset
Last updated on 
January 20, 2022
Licensed under 
Research Only
Gain control of your training data
15,000+ ML engineers can’t be wrong