Learning Generative Models for Density Estimation and Function Approximation

• Main Authors: Zheng-Han Lin, 林政漢
• Other Authors: Jiann-Ming Wu
• Format: Others
• Language: en_US
• Published: 2005
• Online Access: http://ndltd.ncl.edu.tw/handle/26599859445399880913

博士 === 國立東華大學 === 應用數學系 === 93 === This thesis systematically explores generative models for unsupervised and supervised pattern analysis. By designing and fitting the proposed generative models for unsupervised and supervised pattern analysis, we devise new approaches to solving complex tasks in the field of neural networks, including density estimation, independent component analysis and function approximation. The generative models are synchronous and feed-forward networks of generators and primitive operations. Following the proposed design, either the probability density function (pdf) of the output of a generative model for unsupervised pattern analysis or the conditional pdf of the model output to input for supervised pattern analysis is well expressed in terms of the model parameters.　Learning a generative model is equivalent to fitting the correspondent pdf to training data. Since the resulted mathematical framework consists of continuous and discrete variables, it is resolved by the annealed expectation and maximization method or neural relaxation based on a hybrid of the mean field annealing and (natural) gradient descent methods. Learning generative models induces novel algorithms for solving independent component analysis, density estimation and function approximation. Numerical simulations show that novel algorithms possess outstanding performance relative to existing methods for related tasks.