Enhancing cross document co reference of web documents with context similarity and very large scale text categorization

Cross Document Co reference (CDC) is the task of constructing the co reference chain for mentions of a person across a set of documents. This work offers a holistic view of using document-level categories, sub-document level context and extracted entities and relations for the CDC task. We train a categorization component with an efficient flat algorithm using thousands of ODP categories and over a million web documents. We propose to use ranked categories as co reference information, particularly suitable for web documents that are widely different in style and content. An ensemble composite co reference function, amenable to inactive features, combines these three levels of evidence for disambiguation. A thorough feature importance study is conducted to analyze how these three components contribute to the co reference results. The overall solution is evaluated using the WePS benchmark data and demonstrate superior performance.