Neural Rule Ensembles: Encoding Feature Interactions into Neural Networks
Artificial Neural Networks form the basis of very powerful learning methods. It has been observed that a naive application of fully connected neural networks often leads to overfitting. In an attempt to circumvent this issue, a prior knowledge pertaining to feature interactions can be encoded into t...
| Other Authors: | |
|---|---|
| Format: | Others |
| Language: | English English |
| Published: |
Florida State University
|
| Subjects: | |
| Online Access: | http://purl.flvc.org/fsu/fd/2018_Su_Dawer_fsu_0071E_14670 |
| Summary: | Artificial Neural Networks form the basis of very powerful learning methods. It has been observed that a naive application of fully connected neural networks often leads to overfitting. In an attempt to circumvent this issue, a prior knowledge pertaining to feature interactions can be encoded into these networks. This defines a task-specific structure on an underlying representation and helps in reducing the number of learnable parameters. Convolutional Neural Network is such an adaptation of artificial neural networks for image datasets which exploits the spatial relationship among the features and explicitly encodes the translational equivariance. Similarly, Recurrent Neural Networks are designed to exploit the temporal relationship inherent in sequential data. However, for tabular datasets, any prior structure on feature relationships is not apparent. In this work, we use decision trees to capture such feature interactions for this kind of datasets and define a mapping to encode extracted relationships into a neural network. This addresses the initialization related concerns of fully connected neural networks and enables learning of compact representations compared to state of the art tree-based approaches. Empirical evaluations and simulation studies show the superiority of such an approach over fully connected neural networks and tree-based approaches. === A Dissertation submitted to the Department of Mathematics in partial fulfillment of the requirements for the degree of Doctor of Philosophy. === Summer Semester 2018. === June 20, 2018. === Decision Trees, Ensemble Methods, Neural Networks, Neural Rule Ensembles, Rule Learning, Supervised Learning === Includes bibliographical references. === Adrian Barbu, Professor Co-Directing Dissertation; Kyle Gallivan, Professor Co-Directing Dissertation; Gordon Erlebacher, University Representative; Giray Okten, Committee Member; Mark Sussman, Committee Member. |
|---|