Performance prediction based on neural architecture features

• Main Authors: Duo Long, Shizhou Zhang, Yanning Zhang
• Format: Article
• Language: English
• Published: Wiley 2020-04-01
• Series: Cognitive Computation and Systems
• Subjects: neural nets; regression analysis; learning (artificial intelligence); n-grams model; architecture textual description; extracted neural architecture features; appropriate regression model; modelling learning curve; performance prediction method; neural architecture search; candidate neural networks; high-performance network architecture; optimising hyperparameters; training steps; sequence features; chain-structured neural architecture
• Online Access: https://digital-library.theiet.org/content/journals/10.1049/ccs.2019.0024

Neural Architecture Search (NAS) usually requires to train quantities of candidate neural networks on a dataset for choosing a high-performance network architecture and optimising hyperparameters, which is very time consuming and computationally expensive. In order to resolve the issue, the authors try to use a performance prediction method to predict the model performance with little or even no training steps. They assume that the performance is determined once the architecture and other hyperparameters are chosen. So they first extract the sequence features of the chain-structured neural architecture by introducing the N-grams model to process architecture textual description. Subsequently, based on the extracted neural architecture features, they use the appropriate regression model to predict validation accuracies for a modelling learning curve. Through a series of experimental comparisons, they verify the effectiveness of the authors’ proposed performance prediction method and the effective acceleration of the NAS process.