Divide-and-Conquer Learning and Modular Perceptron Network

• Main Authors: Yen-Po Lee, 李衍博
• Other Authors: Prof. Hsin-Chia Fu
• Format: Others
• Language: zh-TW
• Published: 2002
• Online Access: http://ndltd.ncl.edu.tw/handle/59652302687861525600

博士 === 國立交通大學 === 資訊工程系 === 90 === A novel Modular Perceptron Network (MPN) and Divide-and-Conquer Learning (DCL) schemes with Weight Estimation for the design of modular neural networks are proposed. Finally the perceptron networks is performed on Dimensionality reduction with obvious effectiveness for solution in Curse of Dimensionality. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region (e.g., a hard to be learned training set) into two easier (hopely) to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We have evaluated and compared the proposed MPN with several representative neural networks on the Two-spirals problem and real-world database. On learning the the Two-spirals problem, the MPN achieves better weight learning performance by requiring much less data presentations ( less 99.01%~ 87.86%) during the network training phases, and better generalization performance (4.0% better), and less processing time (less 2.0% ~ 81.3%) during the retrieving phase. On learning the real-world data, the MPNs show the same performance as above problem learning results and less overfitting compared to single MLP. Otherwise, for solution in curse of dimensionality, induced from high dimensional input data during learning process. An evaluation algorithm is established in this paper to overcome this problem. It shows that from the experimental results the dimension of the input data can be largely decreased (less 75% ~ 88%) and the data presentations are also reduced (less 70% ~ 90%). So a small size MPNs can be procured with learning and testing performance maintained as the good level as before. In addition, due to its self-growing and fast local learning characteristics, the modular network MPN can easily adapt to on-line and/or incremental learning requirements for a rapid changing environment.