Training recurrent neural networks with temporal input encodings

C. W. Omlin; C. L. Giles; B. G. Horne; L. R. Leerink; T. Lin

Training recurrent neural networks with temporal input encodings

C. W. Omlin
, C. L. Giles
, B. G. Horne
, L. R. Leerink
, T. Lin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about 'good' temporal encoding, i.e. encodings which do not significantly increase training time compared to training of networks with multiple input neurons.

Original language	English (US)
Title of host publication	IEEE International Conference on Neural Networks - Conference Proceedings
Publisher	IEEE
Pages	1267-1272
Number of pages	6
Volume	2
State	Published - 1994
Event	Proceedings of the 1994 IEEE International Conference on Neural Networks. Part 1 (of 7) - Orlando, FL, USA Duration: Jun 27 1994 → Jun 29 1994

Other

Other	Proceedings of the 1994 IEEE International Conference on Neural Networks. Part 1 (of 7)
City	Orlando, FL, USA
Period	6/27/94 → 6/29/94

All Science Journal Classification (ASJC) codes

Software

Cite this

@inproceedings{1dc81f455d3542828d70f3002133697d,

title = "Training recurrent neural networks with temporal input encodings",

abstract = "We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about 'good' temporal encoding, i.e. encodings which do not significantly increase training time compared to training of networks with multiple input neurons.",

author = "Omlin, \{C. W.\} and Giles, \{C. L.\} and Horne, \{B. G.\} and Leerink, \{L. R.\} and T. Lin",

year = "1994",

language = "English (US)",

volume = "2",

pages = "1267--1272",

booktitle = "IEEE International Conference on Neural Networks - Conference Proceedings",

publisher = "IEEE",

note = "Proceedings of the 1994 IEEE International Conference on Neural Networks. Part 1 (of 7) ; Conference date: 27-06-1994 Through 29-06-1994",

}

TY - GEN

T1 - Training recurrent neural networks with temporal input encodings

AU - Omlin, C. W.

AU - Giles, C. L.

AU - Horne, B. G.

AU - Leerink, L. R.

AU - Lin, T.

PY - 1994

Y1 - 1994

N2 - We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about 'good' temporal encoding, i.e. encodings which do not significantly increase training time compared to training of networks with multiple input neurons.

AB - We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about 'good' temporal encoding, i.e. encodings which do not significantly increase training time compared to training of networks with multiple input neurons.

UR - https://www.scopus.com/pages/publications/0028756069

UR - https://www.scopus.com/pages/publications/0028756069#tab=citedBy

M3 - Conference contribution

AN - SCOPUS:0028756069

VL - 2

SP - 1267

EP - 1272

BT - IEEE International Conference on Neural Networks - Conference Proceedings

PB - IEEE

T2 - Proceedings of the 1994 IEEE International Conference on Neural Networks. Part 1 (of 7)

Y2 - 27 June 1994 through 29 June 1994

ER -

Training recurrent neural networks with temporal input encodings

Abstract

Other

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this