Properties of consensus methods for inferring species trees from gene trees

Consensus methods provide a useful strategy for summarizing information from, a collection of gene trees. An important application of consensus methods is to combine gene trees to estimate a species tree. To investigate the theoretical properties of consensus trees that would be obtained from large numbers of loci evolving according to a basic evolutionary model, we construct consensus trees from rooted, gene trees that occur in proportion, to gene-tree probabilities derived from, coalescent theory. We consider majority-rule, rooted triple (R*), and greedy consensus trees obtained from known, rooted gene trees, both in the asymptotic case as numbers of gene trees approach infinity and for finite numbers of genes. Our results show that for some combinations of species-tree branch lengths, increasing the number of independent loci can make the rooted majority-rule consensus tree more likely to be at least partially unresolved. However, the probability that the R* consensus tree has the species-tree topology approaches 1 as the number of gene trees approaches oo. Although the greedy consensus algorithm can be the quickest to converge on the correct species-tree topology when increasing the number of gene trees, it can also be positively misleading. The majority-rule consensus tree is not a misleading estimator of the species-tree topology, and the R* consensus tree is a statistically consistent estimator of the species-tree topology. Our results therefore suggest a method for using multiple loci to infer the species-tree topology, even when it is discordant with the most likely gene tree. [Anomalous gene tree; coalescence; discordance; lineage sorting; phylogeneties; statistical, consistency]