Multi-class Iterative Minimum-Squared-Error Discriminant Classifier

• Main Authors: Hau-Ju Tang, 湯皓如
• Other Authors: 陳正剛
• Format: Others
• Language: en_US
• Published: 2007
• Online Access: http://ndltd.ncl.edu.tw/handle/06752037310233686648

碩士 === 國立臺灣大學 === 工業工程學研究所 === 95 === Discriminant classifier is a type of supervised machine learning technique. There are two approaches to it. One is the Fisher’s discriminant; the other is the Minimum-Squared-Error (MSE) discriminant. The MSE discriminant is usually used to deal with two-class problems. The multi-class MSE approach extends the MSE discriminant to allow problems with more than two classes by providing a set of orthonormal class-label vectors through the Gram-Schmidt process. The resulting class-label vectors are made orthonormal so that the discriminants can be orthogonal as well. However, by giving different linearly independent vectors to the Gram-Schmidt process, the resulting class-label vectors will be different and so do the corresponding discriminants. That is, the solution of multi-class MSE is not unique and may not be the optimal. This research develops an iterative algorithm to obtain the class-label vectors and the discriminant loadings simultaneously while the objective is achieved. The objective is to make the discriminant scores as close to its corresponding class labels as possible. The iterative process is proven to be converged by the power method. The multi-class discriminants found through this iterative algorithm is called multi-class iterative MSE discriminants (IMSED). Through discriminant approaches, we will obtain the discriminant score for each instance. To allocate the instances to classes, there are mainly two types of classification rules. One is distance based; the other is probability based. Four classification rules will be discussed in this research and a probabilistic classification rule will be developed. Iris dataset will be used to illustrate the iterative algorithm and the classification rules. Finally, two real-world data sets with multiple classes are used to compare the IMSED classifier with the Fisher’s discriminant classifier and the multi-class MSE discriminant classifier.