Fusing heterogeneous data: A case for remote sensing and social media

Han Wang, Erik Skau, Hamid Krim, Guido Cervone

Research output: Contribution to journalArticlepeer-review

34 Scopus citations


Data heterogeneity can pose a great challenge to process and systematically fuse low-level data from different modalities with no recourse to heuristics and manual adjustments and refinements. In this paper, a new methodology is introduced for the fusion of measured data for detecting and predicting weather-driven natural hazards. The proposed research introduces a robust theoretical and algorithmic framework for the fusion of heterogeneous data in near real time. We establish a flexible information-based fusion framework with a target optimality criterion of choice, which for illustration, is specialized to a maximum entropy principle and a least effort principle for semisupervised learning with noisy labels. We develop a methodology to account for multimodality data and a solution for addressing inherent sensor limitations. In our case study of interest, namely, that of flood density estimation, we further show that by fusing remote sensing and social media data, we can develop well founded and actionable flood maps. This capability is valuable in situations where environmental hazards, such as hurricanes or severe weather, affect very large areas. Relative to the state of the art working with such data, our proposed information-theoretic solution is principled and systematic, while offering a joint exploitation of any set of heterogeneous sensor modalities with minimally assuming priors. This flexibility is coupled with the ability to quantitatively and clearly state the fusion principles with very reasonable computational costs. The proposed method is tested and substantiated with the multimodality data of a 2013 Boulder Colorado flood event.

Original languageEnglish (US)
Article number8412269
Pages (from-to)6956-6968
Number of pages13
JournalIEEE Transactions on Geoscience and Remote Sensing
Issue number12
StatePublished - Dec 2018

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • General Earth and Planetary Sciences


Dive into the research topics of 'Fusing heterogeneous data: A case for remote sensing and social media'. Together they form a unique fingerprint.

Cite this