Nonparametric Regularized Regression for Phenotype-Associated Taxa Selection and Network Construction with Metagenomic Count Data

We use a metagenomic approach and network analysis to investigate the relationships between phenotypes across taxa under different environmental conditions. The network structure of taxa can be affected by the disease-associated environmental conditions. In addition, taxa abundance is differentiated under conditions. Therefore, knowing how the correlation or relative abundance changes with these factors would be of great interest to researchers. We develop a nonparametric regularized regression method to construct taxa association networks under different clinical conditions. We let the coefficients be unknown functions of the environmental variable. The varying coefficients are estimated by using regression splines. The proposed method is regularized with concave penalties, and an efficient group descent algorithm is developed for computation. We also apply the varying coefficient model to estimate taxa abundance to see how it changes across different environmental conditions. Moreover, for conducting inference, we propose a bootstrap method to construct the simultaneous confidence bands for the corresponding coefficients. We use different simulated designs and a real data set to demonstrate that our method can identify the network structures successfully under different environmental conditions. As such, the proposed method has potential applications for researchers to construct differential networks and identify taxa.