A Linear Approximation Approach of F-Measure for Evaluating the Performance of Classification Algorithms on Imbalanced Data Sets

• Main Authors: Wen-JingChen, 陳玟靜
• Other Authors: Tzu-Tsung Wong
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/w3262v

碩士 === 國立成功大學 === 資訊管理研究所 === 106 === The performance of classification algorithms are generally evaluated by accuracy with huge amounts of data. Accuracy is one of the most convenient and direct indicators. However, classification algorithms will tend to predict most of data as the majority of 　the category values on imbalanced data sets, accuracy is no longer an appropriate measure for performance evaluation. F-measure is the harmonic mean of precision and recall, and these two indicators are dependent of each other. So, there is no appropriate parametric method to compare the F-measures of different classification algorithms. This study presents parametric methods for comparing the performance of two classification algorithms on one or multiple imbalance data sets when the evaluation measure is bivariate normal distribution by recall and precision. Then hypothesis testing is used to compare whether there is significant difference between two classification algorithms. The main purpose is to use F-measures as performance evaluation on imbalanced data. There are four classification algorithms considered in this study. The experimental results show that Naive Bayes method performs poorly under imbalanced data sets. After we compare with nonparametric Wilcoxon signed- test, we find that the parametric method proposed in this study can effectively compare the performance of two classification algorithms.