Learning with Product Units

Laurens R. Leerink; C. Lee Giles; Bill G. Home; Marwan A. Jabri

Learning with Product Units

Laurens R. Leerink, C. Lee Giles, Bill G. Home, Marwan A. Jabri

Research output: Contribution to conference › Paper › peer-review

Abstract

Product units provide a method of automatically learning the higher-order input combinations required for efficient learning in neural networks. However, we show that problems are encountered when using backpropagation to train networks containing these units. This paper examines these problems, and proposes some atypical heuristics to improve learning. Using these heuristics a constructive method is introduced which solves well-researched problems with significantly less neurons than previously reported. Secondly, product units are implemented as candidate units in the Cascade Correlation (Fahlman & Lebiere, 1990) system. This resulted in smaller networks which trained faster than when using sigmoidal or Gaussian units.

Original language	English (US)
Pages	537-544
Number of pages	8
State	Published - 1994
Event	7th International Conference on Neural Information Processing Systems, NIPS 1994 - Denver, United States Duration: Jan 1 1994 → Jan 1 1994

Conference

Conference	7th International Conference on Neural Information Processing Systems, NIPS 1994
Country/Territory	United States
City	Denver
Period	1/1/94 → 1/1/94

All Science Journal Classification (ASJC) codes

Information Systems
Signal Processing
Computer Networks and Communications

Cite this

@conference{784802c0823a47418ccf6598034b9e29,

title = "Learning with Product Units",

abstract = "Product units provide a method of automatically learning the higher-order input combinations required for efficient learning in neural networks. However, we show that problems are encountered when using backpropagation to train networks containing these units. This paper examines these problems, and proposes some atypical heuristics to improve learning. Using these heuristics a constructive method is introduced which solves well-researched problems with significantly less neurons than previously reported. Secondly, product units are implemented as candidate units in the Cascade Correlation (Fahlman & Lebiere, 1990) system. This resulted in smaller networks which trained faster than when using sigmoidal or Gaussian units.",

author = "Leerink, {Laurens R.} and {Lee Giles}, C. and Home, {Bill G.} and Jabri, {Marwan A.}",

note = "Publisher Copyright: {\textcopyright} NIPS 1994.All rights reserved.; 7th International Conference on Neural Information Processing Systems, NIPS 1994 ; Conference date: 01-01-1994 Through 01-01-1994",

year = "1994",

language = "English (US)",

pages = "537--544",

}

TY - CONF

T1 - Learning with Product Units

AU - Leerink, Laurens R.

AU - Lee Giles, C.

AU - Home, Bill G.

AU - Jabri, Marwan A.

PY - 1994

Y1 - 1994

N2 - Product units provide a method of automatically learning the higher-order input combinations required for efficient learning in neural networks. However, we show that problems are encountered when using backpropagation to train networks containing these units. This paper examines these problems, and proposes some atypical heuristics to improve learning. Using these heuristics a constructive method is introduced which solves well-researched problems with significantly less neurons than previously reported. Secondly, product units are implemented as candidate units in the Cascade Correlation (Fahlman & Lebiere, 1990) system. This resulted in smaller networks which trained faster than when using sigmoidal or Gaussian units.

AB - Product units provide a method of automatically learning the higher-order input combinations required for efficient learning in neural networks. However, we show that problems are encountered when using backpropagation to train networks containing these units. This paper examines these problems, and proposes some atypical heuristics to improve learning. Using these heuristics a constructive method is introduced which solves well-researched problems with significantly less neurons than previously reported. Secondly, product units are implemented as candidate units in the Cascade Correlation (Fahlman & Lebiere, 1990) system. This resulted in smaller networks which trained faster than when using sigmoidal or Gaussian units.

UR - http://www.scopus.com/inward/record.url?scp=85153930587&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85153930587&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85153930587

SP - 537

EP - 544

T2 - 7th International Conference on Neural Information Processing Systems, NIPS 1994

Y2 - 1 January 1994 through 1 January 1994

ER -

Learning with Product Units

Abstract

Conference

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this