The « Photo annotation » task consists in automatically annotating a huge number of images from social photo sharing websites with high-level semantic concepts that can be present in images. In 2012, 94 concepts were defined, classified into 5 different groups: natural elements (day, night, sunrise, etc.), environment (desert, coast, landscape, etc.), people (baby, child, teenager, etc.), image elements (in focus, city life, active, etc.), human elements (rail vehicle, water vehicle, air vehicle, etc.). This is an extremely difficult task in the computer vision and pattern recognition fields because of the numerous and large intra-class variations (different types of landscapes for instance), inter-concept similarities (moto vs bicycle for instance) and the semantic gap between the concepts to identify in images and the low-level features extracted from these images that aim at carrying the information used by the algorithm to automatically identify the concepts.

The Imagine team from LIRIS laboratory has made several innovations in its submissions. New visual features have been proposed (OC-LBP, dynamism and color harmony, …), as well as new textual features (Histogram of Textual Concepts) that allow to take into account the semantic similarity between concepts. Moreover, a new fusion scheme, SWLF (Selective Weighted Late Fusion), has been elaborated to characterize each high-level concept with a specific set of features and to weight, through a learning step, the influence of these features according to their ability to characterize each of the concepts to identify. The paper describing this approach is available on the competition website at the following address:

http://www.clef-initiative.eu/documents/71612/60637cac-7c0e-4f59-a45d-37637f0f37cd

 A detailed analysis of the results is provided in the synthesis document written by the organizers of the competition, available at the following address:

http://www.clef-initiative.eu/documents/71612/ec10fe5c-92e7-4217-b6fa-24ad439df1ba

 18 international research teams have participated in 2012 with 80 submissions (each team can make at most 5 different submissions). Three criteria have been used for comparing the methods, such as the mean average precision (MiAP). The LIRIS multimodal submission that associates visual and textual features with the SWLF fusion scheme obtained the best performance among the 80 submissions according to the three evaluation criteria (see section 6.2, table 3). Among purely textual approaches, LIRIS also obtained the first rank for the three criteria (table 4) and among purely visual approaches, LIRIS obtained the first rank for two out of the three evaluation criteria (table 5).

 The LIRIS members involved in this competition are Ningning Liu, Emmanuel Dellandréa, Liming Chen, Aliaksandr Trus, Chao Zhu, Yu Zhang, Charles-Edmond Bichot, Stéphane Brès and Bruno Tellez. Technical and administrative supports were Colette Vial-Buil, Isabelle Dominique and Aliaksandr Paradzinets (Ghanni). The LIRIS participations at ImageCLEF in 2011 and 2012 were partly funded by the VideoSense ANR project (2009 CORD 02602).