Assessing the performance of macromolecular sequence classifiers

Cornelia Caragea, Jivko Sinapov, Vasant Honavar, Drena Dobbs

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

Machine learning approaches offer some of the most cost-effective approaches to building predictive models (e.g., classifiers) in a broad range of applications in computational biology. Comparing the effectiveness of different algorithms requires reliable procedures for accurately assessing the performance (e.g., accuracy, sensitivity, and specificity) of the resulting predictive classifiers. The difficulty of this task is compounded by the use of different data selection and evaluation procedures and in some cases, even different definitions for the same performance measures. We explore the problem of assessing the performance of predictive classifiers trained on macromolecular sequence data, with an emphasis on cross-validation and data selection methods. Specifically, we compare sequence-based and window-based cross-validation procedures on three sequence-based prediction tasks: identification of glycosylation sites, RNA-Protein interface residues, and Protein-Protein interface residues from amino acid sequence. Our experiments with two representative classifiers (Naive Bayes and Support Vector Machine) show that sequence-based and windows-based cross-validation procedures and data selection methods can yield different estimates of commonly used performance measures such as accuracy, Matthews correlation coefficient and area under the Receiver Operating Characteristic curve. We argue that the performance estimates obtained using sequence-based cross-validation provide more realistic estimates of performance than those obtained using window-based cross-validation.

Original languageEnglish (US)
Title of host publicationProceedings of the 7th IEEE International Conference on Bioinformatics and Bioengineering, BIBE
Pages320-326
Number of pages7
DOIs
StatePublished - Dec 1 2007
Event7th IEEE International Conference on Bioinformatics and Bioengineering, BIBE - Boston, MA, United States
Duration: Jan 14 2007Jan 17 2007

Publication series

NameProceedings of the 7th IEEE International Conference on Bioinformatics and Bioengineering, BIBE

Other

Other7th IEEE International Conference on Bioinformatics and Bioengineering, BIBE
Country/TerritoryUnited States
CityBoston, MA
Period1/14/071/17/07

All Science Journal Classification (ASJC) codes

  • Biotechnology
  • Genetics
  • Bioengineering

Fingerprint

Dive into the research topics of 'Assessing the performance of macromolecular sequence classifiers'. Together they form a unique fingerprint.

Cite this