TY - JOUR
T1 - Multiobjective Reinforcement Learning for Cognitive Satellite Communications Using Deep Neural Network Ensembles
AU - Ferreira, Paulo Victor Rodrigues
AU - Paffenroth, Randy
AU - Wyglinski, Alexander M.
AU - Hackett, Timothy M.
AU - Bilen, Sven G.
AU - Reinhart, Richard C.
AU - Mortensen, Dale J.
N1 - Funding Information:
Manuscript received June 23, 2017; revised December 30, 2017; accepted April 4, 2018. Date of publication May 3, 2018; date of current version July 23, 2018. This work was supported in part by the NASA John H. Glenn Research Center under Grant NNC14AA01A, in part by the NASA Space Technology Research Fellowship under Grant NNX15AQ41H, and in part by CAPES Science Without Borders Scholarship under Grant BEX 18701/12-4. (Corresponding author: Paulo Victor Rodrigues Ferreira.) P. V. R. Ferreira and A. M. Wyglinski are with the Department of Electrical and Computer Engineering, Worcester Polytechnic Institute, Worcester, MA 01609 USA (e-mail: prferreira@wpi.edu; alexw@wpi.edu).
Funding Information:
of Electrical and Computer Engineering at Worces-ter Polytechnic Institute, Worcester, MA, USA and Director of the Wireless Innovation Laboratory. Dr. Wyglinski received his B.Eng. and Ph.D. degrees in 1999 and 2005 from McGill University, and his M.Sc.(Eng.) degree from Queen’s University in Kingston in 2000, all in Electrical Engineer-ing. During his academic career, Dr. Wyglinski has published over 40 journal papers, over 80 confer-ence papers, 9 book chapters, and two textbooks. Dr. Wyglinsk’s current research activities include wireless communications, cognitive radio, software-defined radio, dynamic spectrum access, spectrum measurement and characterization, electromagnetic security, wireless system optimization and adaptation, and cyber-physical systems. He is currently being or has been sponsored by organizations such as the Defense Advanced Research Projects Agency (DARPA), the Naval Research Laboratory (NRL), the Office of Naval Research (ONR), the Air Force Research Laboratory (AFRL) - Space Vehicles Directorate, The MathWorks, Toyota InfoTechnol-ogy Center U.S.A., Raytheon, the MITRE Corporation, National Aeronautics and Space Administration (NASA) and the National Science Foundation (NSF). Dr. Wyglinski is a Senior Member of the IEEE, as well as a member of Sigma Xi, Eta Kappa Nu, and the ASEE. Furthermore, Dr. Wyglinski is currently the President of the IEEE Vehicular Technology Society.
Publisher Copyright:
© 1983-2012 IEEE.
PY - 2018/5
Y1 - 2018/5
N2 - Future spacecraft communication subsystems will potentially benefit from software-defined radios controlled by artificial intelligence algorithms. In this paper, we propose a novel radio resource allocation algorithm leveraging multiobjective reinforcement learning and artificial neural network ensembles able to manage available resources and conflicting mission-based goals. The uncertainty in the performance of thousands of possible radio parameter combinations and the dynamic behavior of the radio channel over time producing a continuous multidimensional state-action space requires a fixed-size memory continuous state-action mapping instead of the traditional discrete mapping. In addition, actions need to be decoupled from states in order to allow for online learning, performance monitoring, and resource allocation prediction. The proposed approach leverages the authors' previous research on constraining decisions predicted to have poor performance through 'virtual environment exploration.' The simulation results show the performance for different communication mission profiles, and accuracy benchmarks are provided for the future research reference. The proposed approach constitutes part of the core cognitive engine proof-of-concept delivered to the NASA John H. Glenn Research Center's SCaN Testbed radios on-board the International Space Station.
AB - Future spacecraft communication subsystems will potentially benefit from software-defined radios controlled by artificial intelligence algorithms. In this paper, we propose a novel radio resource allocation algorithm leveraging multiobjective reinforcement learning and artificial neural network ensembles able to manage available resources and conflicting mission-based goals. The uncertainty in the performance of thousands of possible radio parameter combinations and the dynamic behavior of the radio channel over time producing a continuous multidimensional state-action space requires a fixed-size memory continuous state-action mapping instead of the traditional discrete mapping. In addition, actions need to be decoupled from states in order to allow for online learning, performance monitoring, and resource allocation prediction. The proposed approach leverages the authors' previous research on constraining decisions predicted to have poor performance through 'virtual environment exploration.' The simulation results show the performance for different communication mission profiles, and accuracy benchmarks are provided for the future research reference. The proposed approach constitutes part of the core cognitive engine proof-of-concept delivered to the NASA John H. Glenn Research Center's SCaN Testbed radios on-board the International Space Station.
UR - http://www.scopus.com/inward/record.url?scp=85046431536&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85046431536&partnerID=8YFLogxK
U2 - 10.1109/JSAC.2018.2832820
DO - 10.1109/JSAC.2018.2832820
M3 - Article
AN - SCOPUS:85046431536
SN - 0733-8716
VL - 36
SP - 1030
EP - 1041
JO - IEEE Journal on Selected Areas in Communications
JF - IEEE Journal on Selected Areas in Communications
IS - 5
M1 - 8353861
ER -