Advances in data representation for hard/soft information fusion

Jeffrey C. Rimland, Dan Coughlin, David L. Hall, Jacob L. Graham

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


Information fusion is becoming increasingly human-centric. While past systems typically relegated humans to the role of analyzing a finished fusion product, current systems are exploring the role of humans as integral elements in a modular and extensible distributed framework where many tasks can be accomplished by either human or machine performers. For example, "participatory sensing" campaigns give humans the role of "soft sensors" by uploading their direct observations or as "soft sensor platforms" by using mobile devices to record human-annotated, GPS-encoded high quality photographs, video, or audio. Additionally, the role of "human-in-the-loop", in which individuals or teams using advanced human computer interface (HCI) tools such as stereoscopic 3D visualization, haptic interfaces, or aural "sonification" interfaces can help to effectively engage the innate human capability to perform pattern matching, anomaly identification, and semantic-based contextual reasoning to interpret an evolving situation. The Pennsylvania State University is participating in a Multi-disciplinary University Research Initiative (MURI) program funded by the U.S. Army Research Office to investigate fusion of hard and soft data in counterinsurgency (COIN) situations. In addition to the importance of this research for Intelligence Preparation of the Battlefield (IPB), many of the same challenges and techniques apply to health and medical informatics, crisis management, crowd-sourced "citizen science", and monitoring environmental concerns. One of the key challenges that we have encountered is the development of data formats, protocols, and methodologies to establish an information architecture and framework for the effective capture, representation, transmission, and storage of the vastly heterogeneous data and accompanying metadata -- including capabilities and characteristics of human observers, uncertainty of human observations, "soft" contextual data, and information pedigree. This paper describes our findings and offers insights into the role of data representation in hard/soft fusion.

Original languageEnglish (US)
Title of host publicationMultisensor, Multisource Information Fusion
Subtitle of host publicationArchitectures, Algorithms, and Applications 2012
ISBN (Print)9780819490858
StatePublished - 2012
EventMultisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2012 - Baltimore, MD, United States
Duration: Apr 25 2012Apr 26 2012

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X


OtherMultisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2012
Country/TerritoryUnited States
CityBaltimore, MD

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Advances in data representation for hard/soft information fusion'. Together they form a unique fingerprint.

Cite this