TINA: Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation

Yan Hua, Shuhui Wang, Siyuan Liu, Qingming Huang, Anni Cai

Research output: Contribution to journalConference articlepeer-review

10 Scopus citations


With the explosive growth of web data, effective and efficient technologies are in urgent needs for retrieving semantically relevant contents of heterogeneous modalities. Previous studies construct global transformations to project the heterogeneous data into a measurable subspace. However, global projections cannot appropriately adapt to diverse contents, and the naturally existing multi-level semantic relation in web data is ignored. We study the problem of semantic coherent retrieval, where documents from different modalities should be ranked by the semantic relevance to the queries. Accordingly, we propose TINA, a correlation learning method by Adaptive Hierarchical Semantic Aggregation. First, by joint modeling of content and ontology similarities, we build a semantic hierarchy to measure multi-level semantic relevance. Second, with a set of local linear projections aggregated by gating functions, we optimize the structure risk objective function that involves semantic coherence measurement, local projection consistency and the complexity penalty of local projections. Therefore, semantic coherence and a better bias-variance trade-off can be achieved by TINA. Extensive experiments on widely used NUS-WIDE and ICML-Challenge datasets demonstrate that TINA outperforms state-of-the-art, and achieves better adaptation to the multi-level semantic relation and content divergence.

Original languageEnglish (US)
Article number7023336
Pages (from-to)190-199
Number of pages10
JournalProceedings - IEEE International Conference on Data Mining, ICDM
Issue numberJanuary
StatePublished - Jan 26 2015
Event14th IEEE International Conference on Data Mining, ICDM 2014 - Shenzhen, China
Duration: Dec 14 2014Dec 17 2014

All Science Journal Classification (ASJC) codes

  • Engineering(all)


Dive into the research topics of 'TINA: Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation'. Together they form a unique fingerprint.

Cite this