Multiobjective Reinforcement Learning for Cognitive Satellite Communications Using Deep Neural Network Ensembles

Paulo Victor Rodrigues Ferreira, Randy Paffenroth, Alexander M. Wyglinski, Timothy M. Hackett, Sven G. Bilen, Richard C. Reinhart, Dale J. Mortensen

Research output: Contribution to journalArticlepeer-review

96 Scopus citations

Abstract

Future spacecraft communication subsystems will potentially benefit from software-defined radios controlled by artificial intelligence algorithms. In this paper, we propose a novel radio resource allocation algorithm leveraging multiobjective reinforcement learning and artificial neural network ensembles able to manage available resources and conflicting mission-based goals. The uncertainty in the performance of thousands of possible radio parameter combinations and the dynamic behavior of the radio channel over time producing a continuous multidimensional state-action space requires a fixed-size memory continuous state-action mapping instead of the traditional discrete mapping. In addition, actions need to be decoupled from states in order to allow for online learning, performance monitoring, and resource allocation prediction. The proposed approach leverages the authors' previous research on constraining decisions predicted to have poor performance through 'virtual environment exploration.' The simulation results show the performance for different communication mission profiles, and accuracy benchmarks are provided for the future research reference. The proposed approach constitutes part of the core cognitive engine proof-of-concept delivered to the NASA John H. Glenn Research Center's SCaN Testbed radios on-board the International Space Station.

Original languageEnglish (US)
Article number8353861
Pages (from-to)1030-1041
Number of pages12
JournalIEEE Journal on Selected Areas in Communications
Volume36
Issue number5
DOIs
StatePublished - May 2018

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Multiobjective Reinforcement Learning for Cognitive Satellite Communications Using Deep Neural Network Ensembles'. Together they form a unique fingerprint.

Cite this