TY - JOUR
T1 - Random-forest-inspired neural networks
AU - Wang, Suhang
AU - Aggarwal, Charu
AU - Liu, Huan
N1 - Funding Information:
This material is based on work supported by, or in part by, the National Science Foundation (NSF) grants #1614576 and IIS-1217466 and the Office of Naval Research (ONR) grant N00014-16-1-2257. This study is an extension of Reference [38], which appears in the Proceedings of the 17th SIAM International Conference on Data Mining. Authors’ addresses: S. Wang, Arizona State University, Tempe, AZ 85281; email: [email protected]; C. Aggarwal, IBM T. J. Watson Research Center, Yorktown Heights, NY 10598; email: [email protected]; H. Liu, Arizona State University, Tempe, AZ 85281; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2018 ACM 2157-6904/2018/10-ART69 $15.00 https://doi.org/10.1145/3232230
Publisher Copyright:
© 2018 ACM.
PY - 2018/10
Y1 - 2018/10
N2 - Neural networks have become very popular in recent years, because of the astonishing success of deep learning in various domains such as image and speech recognition. In many of these domains, specifc architectures of neural networks, such as convolutional networks, seem to ft the particular structure of the problem domain very well and can therefore perform in an astonishingly effective way. However, the success of neural networks is not universal across all domains. Indeed, for learning problems without any special structure, or in cases where the data are somewhat limited, neural networks are known not to perform well with respect to traditional machine-learning methods such as random forests. In this article, we show that a carefully designed neural network with random forest structure can have better generalization ability. In fact, this architecture is more powerful than random forests, because the back-propagation algorithm reduces to a more powerful and generalized way of constructing a decision tree. Furthermore, the approach is efcient to train and requires a small constant factor of the number of training examples. This efciency allows the training of multiple neural networks to improve the generalization accuracy. Experimental results on real-world benchmark datasets demonstrate the effectiveness of the proposed enhancements for classifcation and regression.
AB - Neural networks have become very popular in recent years, because of the astonishing success of deep learning in various domains such as image and speech recognition. In many of these domains, specifc architectures of neural networks, such as convolutional networks, seem to ft the particular structure of the problem domain very well and can therefore perform in an astonishingly effective way. However, the success of neural networks is not universal across all domains. Indeed, for learning problems without any special structure, or in cases where the data are somewhat limited, neural networks are known not to perform well with respect to traditional machine-learning methods such as random forests. In this article, we show that a carefully designed neural network with random forest structure can have better generalization ability. In fact, this architecture is more powerful than random forests, because the back-propagation algorithm reduces to a more powerful and generalized way of constructing a decision tree. Furthermore, the approach is efcient to train and requires a small constant factor of the number of training examples. This efciency allows the training of multiple neural networks to improve the generalization accuracy. Experimental results on real-world benchmark datasets demonstrate the effectiveness of the proposed enhancements for classifcation and regression.
UR - http://www.scopus.com/inward/record.url?scp=85056450206&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85056450206&partnerID=8YFLogxK
U2 - 10.1145/3232230
DO - 10.1145/3232230
M3 - Article
AN - SCOPUS:85056450206
SN - 2157-6904
VL - 9
JO - ACM Transactions on Intelligent Systems and Technology
JF - ACM Transactions on Intelligent Systems and Technology
IS - 6
M1 - a69
ER -