Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion

Noah S. Wood, Benjamin P. Lewis, Ram M. Narayanan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Within the field of target recognition, significant attention is given to data fusion techniques to optimize decision making in systems of multiple sensors. The challenge of fusing synthetic aperture radar (SAR) and electrooptical (EO) imagery is of particular interest to the defense community due to those sensors' prevalence in target recognition systems. In this paper, the performances of two network architectures (a simple CNN and a ResNet) are compared, each implemented with multiple fusion methods to classify SAR and EO imagery of military targets. The Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset is used, an expansion of the MSTAR dataset, using both original measured SAR data and synthetic EO data. The classification performance of both networks is compared using the data modalities individually, using feature level fusion, using decision level fusion, and using a novel fusion method based on the three RGB-input channels of the ResNet (or other CNN for color image processing). In the input channel fusion method proposed, SAR imagery is fed to one of the three input channels, and the grayscale EO data is passed to a second of the three input channels. Despite its simplicity and off-the-shelf implementation, the input channel fusion method provides strong results, indicating it is worthy of further study.

Original languageEnglish (US)
Title of host publicationRadar Sensor Technology XXVI
EditorsKenneth I. Ranney, Ann M. Raynal
PublisherSPIE
ISBN (Electronic)9781510650923
DOIs
StatePublished - 2022
EventRadar Sensor Technology XXVI 2022 - Virtual, Online
Duration: Jun 6 2022Jun 12 2022

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume12108
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceRadar Sensor Technology XXVI 2022
CityVirtual, Online
Period6/6/226/12/22

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion'. Together they form a unique fingerprint.

Cite this