Nonparametric entropy estimation for stationary processesand random fields, with applications to english text

I. Kontoyiannis; P. H. Algoet; Yu M. Suhov; A. J. Wyner

doi:10.1109/18.669425

Nonparametric entropy estimation for stationary processesand random fields, with applications to english text

I. Kontoyiannis
, P. H. Algoet
, Yu M. Suhov
, A. J. Wyner

Mathematics

Research output: Contribution to journal › Article › peer-review

195 Scopus citations

Abstract

We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesàro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker. We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.

Original language	English (US)
Pages (from-to)	1319-1327
Number of pages	9
Journal	IEEE Transactions on Information Theory
Volume	44
Issue number	3
DOIs	https://doi.org/10.1109/18.669425
State	Published - 1998

All Science Journal Classification (ASJC) codes

Information Systems
Computer Science Applications
Library and Information Sciences

Access to Document

10.1109/18.669425

Cite this

@article{8b953c2d17d446999948f9b51be81fdb,

title = "Nonparametric entropy estimation for stationary processesand random fields, with applications to english text",

abstract = "We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Ces{\`a}ro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker. We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.",

author = "I. Kontoyiannis and Algoet, \{P. H.\} and Suhov, \{Yu M.\} and Wyner, \{A. J.\}",

year = "1998",

doi = "10.1109/18.669425",

language = "English (US)",

volume = "44",

pages = "1319--1327",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - Nonparametric entropy estimation for stationary processesand random fields, with applications to english text

AU - Kontoyiannis, I.

AU - Algoet, P. H.

AU - Suhov, Yu M.

AU - Wyner, A. J.

PY - 1998

Y1 - 1998

N2 - We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesàro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker. We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.

AB - We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesàro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker. We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.

UR - https://www.scopus.com/pages/publications/0032073870

UR - https://www.scopus.com/pages/publications/0032073870#tab=citedBy

U2 - 10.1109/18.669425

DO - 10.1109/18.669425

M3 - Article

AN - SCOPUS:0032073870

SN - 0018-9448

VL - 44

SP - 1319

EP - 1327

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 3

ER -

Nonparametric entropy estimation for stationary processesand random fields, with applications to english text

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this