Statistical methods for comparing the performance of two classification algorithms on imbalanced data sets

• Main Authors: Che-HsuanLin, 林哲玄
• Other Authors: Tzu-Tsung Wong
• Format: Others
• Language: zh-TW
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/39398602168263993420

碩士 === 國立成功大學 === 資訊管理研究所 === 104 === The performance of classification algorithms are generally evaluated by accuracy. However, when the numbers of instances or the misclassification costs for various class values are largely different, accuracy is no longer an appropriate measure for performance evaluation. Some other measures such as recall and precision will be better choices for imbalance data sets. This study presents parametric methods for comparing the performance of two classification algorithms on multiple imbalance data sets when the evaluation measure is recall, precision, or their arithmetic mean. When the testing results satisfy the large-sample conditions, the sampling distributions of both recall and precision can be assumed to be normally distributed. Since recall and precision for the same data set are dependent, their arithmetic mean is assumed to follow a bivariate normal distribution for deriving its sampling distribution. There are four classification algorithms considered in this study. The experimental results on seven imbalance data sets demonstrate that the parametric methods proposed in this study can effectively compare the performance of two classification algorithms on multiple imbalance data sets.