Recurrent neural networks, hidden Markov models and stochastic grammars

G. Z. Sun; H. H. Chen; Y. C. Lee; C. L. Giles

Recurrent neural networks, hidden Markov models and stochastic grammars

G. Z. Sun
, H. H. Chen
, Y. C. Lee
, C. L. Giles

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

Original language	English (US)
Title of host publication	90 Int Jt Conf Neural Networks IJCNN 90
Publisher	Publ by IEEE
Pages	729-734
Number of pages	6
State	Published - 1990
Event	1990 International Joint Conference on Neural Networks - IJCNN 90 - San Diego, CA, USA Duration: Jun 17 1990 → Jun 21 1990

Other

Other	1990 International Joint Conference on Neural Networks - IJCNN 90
City	San Diego, CA, USA
Period	6/17/90 → 6/21/90

All Science Journal Classification (ASJC) codes

General Engineering

Cite this

@inproceedings{acb1271a06524f5dafe80c6eda10d18b,

title = "Recurrent neural networks, hidden Markov models and stochastic grammars",

abstract = "A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.",

author = "Sun, \{G. Z.\} and Chen, \{H. H.\} and Lee, \{Y. C.\} and Giles, \{C. L.\}",

year = "1990",

language = "English (US)",

pages = "729--734",

booktitle = "90 Int Jt Conf Neural Networks IJCNN 90",

publisher = "Publ by IEEE",

note = "1990 International Joint Conference on Neural Networks - IJCNN 90 ; Conference date: 17-06-1990 Through 21-06-1990",

}

TY - GEN

T1 - Recurrent neural networks, hidden Markov models and stochastic grammars

AU - Sun, G. Z.

AU - Chen, H. H.

AU - Lee, Y. C.

AU - Giles, C. L.

PY - 1990

Y1 - 1990

N2 - A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

AB - A discussion is presented of the advantage of using a linear recurrent network to encode and recognize sequential data. The hidden Markov model (HMM) is shown to be a special case of such linear recurrent second-order neural networks. The Baum-Welch reestimation formula, which has proved very useful in training HMM, can also be used to learn a linear recurrent network. As an example, a network has successfully learned the stochastic Reber grammar with only a few hundred sample strings in about 14 iterations. The relative merits and limitations of the Baum-Welch optimal ascent algorithm in comparison with the error correction-gradient descent-learning algorithm are discussed.

UR - https://www.scopus.com/pages/publications/0025547722

UR - https://www.scopus.com/pages/publications/0025547722#tab=citedBy

M3 - Conference contribution

AN - SCOPUS:0025547722

SP - 729

EP - 734

BT - 90 Int Jt Conf Neural Networks IJCNN 90

PB - Publ by IEEE

T2 - 1990 International Joint Conference on Neural Networks - IJCNN 90

Y2 - 17 June 1990 through 21 June 1990

ER -

Recurrent neural networks, hidden Markov models and stochastic grammars

Abstract

Other

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this