Bayesian Student Modelling and Decision-Theoretic Selection of Tutorial Actions in Intelligent Tutoring Systems

• Main Author: Mayo, Michael John
• Language: en
• Published: University of Canterbury. Computer Science 2009
• Online Access: http://hdl.handle.net/10092/2565

This thesis proposes, demonstrates, and evaluates, the concept of the normative Intelligent Tutoring System (ITS). Normative theories are ideal, optimal theories of rational behaviour. Two normative theories suitable for reasoning under conditions of uncertainty are Bayesian probability theory, which allows one to update one’s beliefs about the world given previous beliefs and new observations, and decision theory, which shows how to fuse one’s preferences with one’s beliefs in order to rationally decide how to behave. A normative ITS is a tutoring system in which beliefs about the student (the student model) are represented with a Bayesian network, and teaching actions are selected using decision-theoretic principles. The main advantage of a normative ITS is that the normative theories provide an optimal framework for implementing learning theories. In other words, the particular learning theory underlying the ITS is guaranteed to be optimally applied to the student if it is defined as a set of normative representations (probability distributions and utility functions). In contrast, the more traditional type of ITS with an ad-hoc implementation of a learning theory is not guaranteed to be optimal. A general methodology for building normative ITSs is proposed and demonstrated. The methodology advocates building an adaptive, generalised Bayesian network student model using machine learning techniques from student performance data collected in the classroom. The Bayesian network is then used as the basis for the decision-theoretic selection of tutorial actions. The methodology is demonstrated with two implementations. Both implementations were evaluated in a classroom, rather than a lab, setting. The first implementation is an extension to an existing ITS called SQL-Tutor. A Bayesian network-based student model was added to SQL-Tutor, and this was applied to select the next problem for students. Although this system only partly implemented the normative methodology, the evaluation results were promising enough to continue in this direction. The second evaluation was more comprehensive. An entirely new ITS called CAPIT was implemented by application of the methodology. CAPIT teaches the basics of English capitalisation and punctuation to 8-10 year old school children, and it uses constraint-based modelling to represent domain knowledge. The system models the child’s long-term mastery of the domain constraints using an adaptive Bayesian network, and it selects the next problem and best error message (when a student makes more than one error following a solution attempt) using the decision-theoretic principle of expected utility maximisation. Learning theories define both the semantics of the Bayesian network and the form of the utility functions. The evaluation of CAPIT was a success. Three groups of children, A, B, and C, were enlisted and given a pre-test. Group B then used a randomised (non-normative) version of CAPIT for a four week period, while Group C used the full normative version of the tutor. All groups were then administered a post-test. The results show that while both Groups B and C gradually mastered the domain constraints, Group C mastered the constraints at a faster rate than group B. Group A, who did not have access to an ITS in the domain, actually regressed on the post-test.