HUMAN DECISION VS BAYESIAN BASED TOOL PREDICTION – AN ANALYSIS OF MISCLASSIFIED CASES IN A MACHINE LEARNING APPLICATION TO TASK ANALYSIS

• Main Author: Danu Koeswara
• Other Authors: Shu-Chiang Lin
• Format: Others
• Language: en_US
• Published: 2012
• Online Access: http://ndltd.ncl.edu.tw/handle/49189093164766014229

碩士 === 國立臺灣科技大學 === 工業管理系 === 100 === A new task analysis methodology that combines a statistical approach had been adapted to develop a semi-automated task analysis tool based on machine learning application to help call canter agent during remote troubleshooting process (Lin and Lehto, 2009). Three Bayesian models had been chosen as statistical approach for identifying subtasks, such as the classic Bayes, fuzzy Bayes, and hybrid Bayes. By examining the preliminary results of Bayesian based tool's performance, it had been proven that the Bayesian based tool is able to learn subtask categories from the agent or customer narrative telephone conversations and to predict them as well. However, there is still necessary to prove the Bayesian based tool is better applied than human expert on many occasions. One of the key issue on previous research is the misclassified cases of discrepancies between human decision and Bayesian based tool's prediction when determines the subtask categories based on conversations or narratives. This issue will be appointed in this study to be a starting point for more in depth investigation of the application of task analysis tools to the naturalistic decision making environment. The discrepancies had pointed out the bad result of Bayesian based tool when predict the subtask categories. Based on the signal detection theory, the error can happened in "false alarm" cases and "miss" cases. By learning the preliminary results of misclassified revealed that the tool not only identifies the hard to classify dialogs, but also actually helps uncover the misclassified pre-assigned subtask categories by human experts. This information denotes the misclassified cases can be caused by human decision or tool's prediction or even both of them. This study focuses on further analysis of misclassified cases with emphasis to discrepancies of human decision and tool's prediction, especially in "false alarm" condition and "miss" condition. Through statistical proving will be done as approach in this study with exploit the abundant data and results obtained by Lin and Lehto's research to reveals from time to time that the tool is more accurate than human experts on many occasions. Furthermore, the phases of this study are arranged as follows: data collection, data identification, data preparation, data analysis, and the advanced investigation of misclassified cases. In these results of this study point out there are influence of discrepancies caused by some treatments in the preliminary stage of Bayesian based tool development when the measuring of tool's performance, but not significant. Due to the bad results of the Bayesian based tool is very low, so the Bayesian based tool’s prediction can be proven better than human decision to identify the subtask categories. In more deep investigation, the discrepancies between human decision and tool’s prediction have been found the misclassified by the human is greater than the tool. These findings can confirm the Bayesian based tool had been proven to be able to uncover the misclassified pre-assigned by the human expert.