Empirical analysis of neural networks training optimisation

• Main Author: Kayembe, Mutamba Tonton
• Format: Others
• Language: en
• Published: 2017
• Subjects: Neural networks (Computer science); Training
• Online Access: Kayembe, Mutamba Tonton (2016) Empirical analysis of neural networks training optimisation, University of Witwatersrand, Johannesburg, <http://wiredspace.wits.ac.za/handle/10539/21679>; http://hdl.handle.net/10539/21679

A Dissertation submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Master of Science in Mathematical Statistics,School of Statistics and Actuarial Science. October 2016. === Neural networks (NNs) may be characterised by complex error functions with attributes such as saddle-points, local minima, even-spots and plateaus. This complicates the associated training process in terms of efficiency, convergence and accuracy given that it is done by minimising such complex error functions. This study empirically investigates the performance of two NNs training algorithms which are based on unconstrained and global optimisation theories, i.e. the Resilient propagation (Rprop) and the Conjugate Gradient with Polak-Ribière updates (CGP). It also shows how the network structure plays a role in the training optimisation of NNs. In this regard, various training scenarios are used to classify two protein data, i.e. the Escherichia coli and Yeast data. These training scenarios use varying numbers of hidden nodes and training iterations. The results show that Rprop outperforms CGP. Moreover, it appears that the performance of classifiers varies under various training scenarios. === LG2017