Abstract
This paper presents a hybrid model for part-of-speech (POS) guessing of Chinese unknown words. Most previous studies on this task have developed a unified statistical model for all Chinese unknown words and have rejected rulebased models without testing. We argue that models that use different sources of information about unknown words, both structural and contextual, can be effective for handling different types of unknown words. We propose a rulebased model that uses information about the type, length, and internal structure of unknown words and combine it with two existing statistical models that use information about the POS context and component characters of unknown words respectively for this task. By combining the complementary strengths of the three models that use different sources of information, the hybrid model achieves an accuracy of 89%, a significant improvement over the best result reported in previous studies.
Original language | English (US) |
---|---|
Pages (from-to) | 169-193 |
Number of pages | 25 |
Journal | International Journal of Corpus Linguistics |
Volume | 13 |
Issue number | 2 |
DOIs | |
State | Published - 2008 |
All Science Journal Classification (ASJC) codes
- Language and Linguistics
- Linguistics and Language