Tagging over time: Real-world image annotation by lightweight meta-learning

Ritendra Datta, Dhiraj Joshi, Jia Li, James Z. Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations


Automatic image annotation has been a hot-pursuit among multimedia researchers of late. Modest performance guarantees and limited adaptability often restrict its applicability to real-world settings. We propose tagging over time (T/T) to push the technology toward real-world applicability. Of particular interest are online systems that receive user-provided images and feedback over time, with user focus possibly changing and evolving. The T/T framework consists of a principled probabilistic approach to meta-learning, which acts as a go-between for a 'black-box' annotation system and the users. Inspired by inductive transfer, the approach attempts to harness available information, including the black-box model's performance, the image representations, and the WordNet ontology. Being computationally 'lightweight', this meta-learner efficiently re-trains over time, to improve and/or adapt to changes. The black-box annotation model is not required to be re-trained, allowing computationally intensive algorithms to be used. We experiment with standard image datasets and real-world data streams, using two existing annotation systems as black-boxes. Both batch and online annotation settings are experimented with. It is observed that the addition of this meta-learning layer produces much improved results that outperform best-known results. For the online setting, the T/T approach produces progressively better annotation with time, significantly outperforming the black-box as well as the static form of the meta-learner, on real-world data.

Original languageEnglish (US)
Title of host publicationProceedings of the Fifteenth ACM International Conference on Multimedia, MM'07
Number of pages10
StatePublished - 2007
Event15th ACM International Conference on Multimedia, MM'07 - Augsburg, Bavaria, Germany
Duration: Sep 24 2007Sep 29 2007

Publication series

NameProceedings of the ACM International Multimedia Conference and Exhibition


Other15th ACM International Conference on Multimedia, MM'07
CityAugsburg, Bavaria

All Science Journal Classification (ASJC) codes

  • Computer Science(all)


Dive into the research topics of 'Tagging over time: Real-world image annotation by lightweight meta-learning'. Together they form a unique fingerprint.

Cite this