a-TRECVID: A dataset of images with multiple attributes

 

Summary

To facilitate multi-attribute based image retrieval [1] and other attribute related research, we compile the largest multi-attribute image dateset to date, including 126 fully labeled attributes of 0.26 million images. This dataset is compiled from the TRECVID 2011 Semantic Indexing (SIN) track common annotation set by:

1.     Discarding attributes with too few positive images;

2.     Discarding images with too few local feature detection regions.

The original dataset includes about 0.3 million video frames extracted from videos with durations ranging from 10s to just longer than 3.5 minutes. The total length of the videos is about 200 hours. Originally, there are 346 fully labeled, unique query attributes for the video frames. The attributes are mostly from the concepts defined in the LSCOM multimedia ontology. The annotation of the dataset was organized by LIG (Laboratoire d'Informatique de Grenoble) and LIF (Laboratoire d'Informatique Fondamentale de Marseille). Details can be found in their website http://mrim.imag.fr/tvca/.


The attributes

126 query attributes of a-TRECVID, selected from a pool of 346 concepts defined in TRECVID 2011 SIN task, by discarding attributes with too few positive images.


Performance evaluation using weak attributes [1][2]

Retrieval performance on a-TRECVID dataset, with the varying training size. From left to right: performance of single, double and triple attribute queries.


Download

Download the description for a-TRECVID here (in Matlab format). You will need TRECVID 2011 SIN dataset in order to get image data of a-TRECVID: http://www-nlpir.nist.gov/projects/tv2011/tv2011.html#sin. Please email yuxinnan@ee.columbia.edu for data and further information.

You will also need to download the TRECVID 2011 collaborative annotation (*) http://mrim.imag.fr/tvca/ in order to get the attribute labels. Please email yuxinnan@ee.columbia.edu for further information.

* S. Ayache and G. Quenot. Video corpus annotation using active learning. In European Conference of Information Retrieval (ECIR), 2008


Project Page and Software Download

 http://www.ee.columbia.edu/ln/dvmm/weak/


Publications

 [1] Felix X. Yu; Rongrong Ji; Ming-Hen Tsai; Guangnan Ye; Shih-Fu Chang.

Weak attributes for large-scale image retrieval, CVPR 2012 [PDF] [Supplementary Material]

[2] Felix X. Yu; Rongrong Ji; Ming-Hen Tsai; Guangnan Ye; Shih-Fu Chang.

Experiments of image retrieval using weak attributes, Technical Report # CUCS 005-12 [PDF]