TY - GEN
T1 - Empirical analysis of the expected source values rule
AU - Spartz, Richard
AU - Honavar, Vasant
PY - 1993/12/1
Y1 - 1993/12/1
N2 - Despite its notoriously slow learning time, back-propagation (BP) is one of the most widely used neural network training algorithms. Two major reasons for this slow convergence are the step size problem and the flat spot problem [Fahlman, 1988]. In [Samad, 1991] a simple modification, the expected source values (ESV) rule, is proposed for speeding up the BP algorithm. We have extended the ESV rule by coupling it with a flat-spot removal strategy presented in [Fahlman, 1988], as well as incorporating a momentum term to combat the step size problem. The resulting rule has shown dramatically improved learning time over standard BP, measured in training epochs. Two versions of the ESV modification are mentioned in [Samad. 1991], on-demand and up-front, but simulation results are given mostly for the on-demand case. Our results indicate that the up-front version works somewhat better than the on-demand version in terms of learning speed. We have also analyzed the interactions between the three modifications as they are used in various combinations.
AB - Despite its notoriously slow learning time, back-propagation (BP) is one of the most widely used neural network training algorithms. Two major reasons for this slow convergence are the step size problem and the flat spot problem [Fahlman, 1988]. In [Samad, 1991] a simple modification, the expected source values (ESV) rule, is proposed for speeding up the BP algorithm. We have extended the ESV rule by coupling it with a flat-spot removal strategy presented in [Fahlman, 1988], as well as incorporating a momentum term to combat the step size problem. The resulting rule has shown dramatically improved learning time over standard BP, measured in training epochs. Two versions of the ESV modification are mentioned in [Samad. 1991], on-demand and up-front, but simulation results are given mostly for the on-demand case. Our results indicate that the up-front version works somewhat better than the on-demand version in terms of learning speed. We have also analyzed the interactions between the three modifications as they are used in various combinations.
UR - http://www.scopus.com/inward/record.url?scp=0027871691&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0027871691&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:0027871691
SN - 1565550072
T3 - Proceedings of SPIE - The International Society for Optical Engineering
SP - 95
EP - 100
BT - Proceedings of SPIE - The International Society for Optical Engineering
A2 - Padgett, Marry Lou
PB - Publ by Society of Photo-Optical Instrumentation Engineers
T2 - Proceedings of the 3rd Workshop on Neural Networks: Academic/Industrial/NASA/Defense
Y2 - 10 February 1992 through 12 February 1992
ER -