Guided Interactive Machine Learning

• Main Author: Pace, Aaron J.
• Format: Others
• Published: BYU ScholarsArchive 2006
• Subjects: Active Learning; Crayons; Confusion; Iterative Committee; Iterative Committee Correction; ICC; Distribution; Guidance; Thermometer Widget; Accuracy Estimation; Incremental Consistency; Image Classifier; Computer Sciences
• Online Access: https://scholarsarchive.byu.edu/etd/477; https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=1476&context=etd

This thesis describes a combination of two current areas of research: the Crayons image classifier system and active learning. Currently Crayons provides no guidance to the user in what pixels should be labeled or when the task is complete. This work focuses on two main areas: 1) active learning for user guidance, and 2) accuracy estimation as a measure of completion. First, I provide a study through simulation and user experiments of seven active learning techniques as they relate to Crayons. Three of these techniques were specifically designed for use in Crayons. These three perform comparably to the others and are much less computationally intensive. A new widget is proposed for use in the Crayons environment giving an overview of the system "confusion". Second, I give a comparison of four accuracy estimation techniques relating to true accuracy and for use as a completion estimate. I show how three traditional accuracy estimation techniques are ineffective when placed in the Crayons environment. The fourth technique uses the same computation as the three new active learning techniques proposed in this work and thus requires little extra computation and outstrips the other three as a completion estimate both in simulation and user experiments.