| Summary: | 碩士 === 國立中山大學 === 電機工程研究所 === 82 === Neural networks may overcome the difficulties related to noise
and uncertainty. Conventionally, a trial-and-error method must
be used to find the proper neural network architecture for a
given problem when one is using back-propagation algorithms.
Also, the conventional method initializes all weights and
thresholds to zero in a neural network, likely resulting in a
long training time and poor classification accuracy. We propose
an idea of constructing neural networks by making use of
decision trees and threshold logic. Decision trees are obtained
by an ID3 algorithm. Such a tree is represented by a logic
expression for constructing a threshold network which forms the
basis of the architecture of the desired neural network. The
number of layers and the number of nodes in each layer of the
neural network are determined. Initial values for weights and
thresholds are also determined. Experiments have shown that a
neural network constructed in this manner learns fast and
performs efficiently. Determining proper neural network
architectures is an issue when one is using back-propagation
algorithms. We make use of information entropy to overcome this
difficulty . Minimal numbers of hidden layers and nodes in each
hidden layer are determined by minimizing the information
entropy function associated with the partition induced by
hyperplanes. A learning process combining the delta rule and a
Monte-Carlo based simulated annealing method is developed to
obtain the coefficients of optimal hyperplanes. Cios and Liu
proposed an entropy based method to generate neural network
architectures for two-class discretization. For multi-class
discretization, independent two-class subnetworks are combined,
resulting in large networks since hidden nodes and layers
cannot be shared among different classes. We propose a
generation procedure, also based on entropy measure, to produce
a network in which hidden nodes and layers are shared among
classes.
|
|---|
