TY - JOUR
T1 - Scalable massively parallel artificial neural networks
AU - Long, Lyie N.
AU - Gupta, Ankur
N1 - Funding Information:
The authors would like to thank the NASA Advanced Supercomputing Division (NAS) for the use of the Columbia supercomputer,21 and Argonne National Labs for the use of the IBM Bluegene/L supercomputer.22 We’d also like to thank the Office of Naval Research for funding this work (Grant No. N00014-05-1-0844).
PY - 2008/1
Y1 - 2008/1
N2 - Artificial Neural Networks (ANN) can be very effective for pattern recognition, function approximation, scientific classification, control, and the analysis of time series data; however they can require very large training times for large networks. Once the network is trained for a particular problem, however, it can produce results in a very short time. Traditional ANNs using back-propagation algorithm do not scale well as each neuron in one level is fully connected to each neuron in the previous level. In the present work only the neurons at the edges of the domains were involved in communication, in order to reduce the communication costs and maintain scalability. Ghost neurons were created at these processor boundaries for information communication. An object-oriented, massively-parallel ANN software package SPANN (Scalable Parallel Artificial Neural Network) has been developed and is described here. MPI was used to parallelize the C++ code. The back-propagation algorithm was used to train the network. In preliminary tests, the software was used to identify character sets consisting of 48 characters and with increasing resolutions. The code correctly identified all the characters when adequate training was used in the network. The training of a problem size with 2 billion neuron weights on an IBM BlueGene/L computer using 1000 dual PowerPC 440 processors required less than 30 minutes. Various comparisons in training time, forward propagation time, and error reduction were also made.
AB - Artificial Neural Networks (ANN) can be very effective for pattern recognition, function approximation, scientific classification, control, and the analysis of time series data; however they can require very large training times for large networks. Once the network is trained for a particular problem, however, it can produce results in a very short time. Traditional ANNs using back-propagation algorithm do not scale well as each neuron in one level is fully connected to each neuron in the previous level. In the present work only the neurons at the edges of the domains were involved in communication, in order to reduce the communication costs and maintain scalability. Ghost neurons were created at these processor boundaries for information communication. An object-oriented, massively-parallel ANN software package SPANN (Scalable Parallel Artificial Neural Network) has been developed and is described here. MPI was used to parallelize the C++ code. The back-propagation algorithm was used to train the network. In preliminary tests, the software was used to identify character sets consisting of 48 characters and with increasing resolutions. The code correctly identified all the characters when adequate training was used in the network. The training of a problem size with 2 billion neuron weights on an IBM BlueGene/L computer using 1000 dual PowerPC 440 processors required less than 30 minutes. Various comparisons in training time, forward propagation time, and error reduction were also made.
UR - http://www.scopus.com/inward/record.url?scp=42949108112&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=42949108112&partnerID=8YFLogxK
U2 - 10.2514/1.31026
DO - 10.2514/1.31026
M3 - Article
AN - SCOPUS:42949108112
SN - 1542-9423
VL - 5
SP - 3
EP - 15
JO - Journal of Aerospace Computing, Information and Communication
JF - Journal of Aerospace Computing, Information and Communication
IS - 1
ER -