Semantic data fusion through visually-enabled analytical reasoning

Investigating terrorist activity patterns and predicting threats involve collecting and analyzing data from both hard sensors and humans as part of analysts' reasoning process (evidence building, hypothesis creation and testing and decision making). Although automated data fusion methods have been proposed in previous studies, they tend to operate on low-level linguistic features of events and fail to connect to high-level conceptual categories that analysts need to make judgment. This paper argues for extending data fusion models and architecture with an explicit component of visual analytics that integrates human and machine analytical capability through interactive visual analysis. We motivate this argument by the need for human-driven analytical reasoning in counter-intelligence investigation domain. Our extended data fusion architecture follows the sensemaking theory of Pirolli and Card, which provides a framework for understanding specific details on how investigative analysis weave computation, visualization and human reasoning to support coherent analytics. The feasibility of this data fusion architecture is demonstrated through an Analysts' Workbench that allows analysts to construct intelligence reports through discovering, assessing, and associating evidences.