Statistical analysis and algorithms for online change detection in real-time psychophysiological data

• Main Author: Cannon, Jordan
• Other Authors: Krokhmal, Pavlo
• Format: Others
• Language: English
• Published: University of Iowa 2009
• Subjects: Adaptive Aiding; Cognitive Load; EEG; Index; Operator Functional State; Real Time Detection; Industrial Engineering
• Online Access: https://ir.uiowa.edu/etd/342; https://ir.uiowa.edu/cgi/viewcontent.cgi?article=1527&context=etd

Modern systems produce a great amount of information and cues from which human operators must take action. On one hand, these complex systems can place a high demand on an operator's cognitive load, potentially overwhelming them and causing poor performance. On the other hand, some systems utilize extensive automation to accommodate their complexity; this can cause an operator to become complacent and inattentive, which again leads to deteriorated performance (Wilson, Russell, 2003a; Wilson, Russell, 2003b). An ideal human-machine interface would be one that optimizes the functional state of the operator, preventing overload while not permitting complacency, thus resulting in improved system performance. An operator's functional state (OFS) is the momentary ability of an operator to meet task demands with their cognitive resources. A high OFS indicates that an operator is vigilant and aware, with ample cognitive resources to achieve satisfactory performance. A low OFS, however, indicates a non-optimal cognitive load, either too much or too little, resulting in sub-par system performance (Wilson, Russell, 1999). With the ability to measure and detect changes in OFS in real-time, a closed-loop system between the operator and machine could optimize OFS through the dynamic allocation of tasks. For instance, if the system detects the operator is in cognitive overload, it can automate certain tasks allowing them to better focus on salient information. Conversely, if the system detects under-vigilance, it can allocate tasks back to the manual control of the operator. In essence, this system operates to "dynamically match task demands to [an] operator's momentary cognitive state", thereby achieving optimal OFS (Wilson, Russell, 2007). This concept is termed adaptive aiding and has been the subject of much research, with recent emphasis on accurately assessing OFS in real-time. OFS is commonly measured indirectly, like using overt performance metrics on tasks; if performance is declining, a low OFS is assumed. Another indirect measure is the subjective estimate of mental workload, where an operator narrates his/her perceived functional state while performing tasks (Wilson, Russell, 2007). Unfortunately, indirect measures of OFS are often infeasible in operational settings; performance metrics are difficult to construct for highly-automated complex systems, and subjective workload estimates are often inaccurate and intrusive (Wilson, Russell, 2007; Prinzel et al., 2000; Smith et al., 2001). OFS can be more directly measured via psychophysiological signals such as electroencephalogram (EEG) and electrooculography (EOG). Current research has demonstrated these signals' ability to respond to changing cognitive load and to measure OFS (Wilson, Fisher, 1991; Wilson, Fisher, 1995; Gevins et al., 1997; Gevins et al., 1998; Byrne, Parasuraman, 1996). Moreover, psychophysiological signals are continuously available and can be obtained in a non-intrusive manner, pre-requisite for their use in operational environments. The objective of this study is to advance schemes which detect change in OFS by monitoring psychophysiological signals in real-time. Reviews on similar methods can be found in, e.g., Wilson and Russell (2003a) and Wilson and Russell (2007). Many of these methods employ pattern recognition to classify mental workload into one of several discrete categories. For instance, given an experiment with easy, medium and hard tasks, and assuming the tasks induce varying degrees of mental workload on a subject, these methods classify which task is being performed for each epoch of psychophysiological data. The most common classifiers are artificial neural networks (ANN) and multivariate statistical techniques such as stepwise discriminant analysis (SWDA). ANNs have proved especially effective at classifying OFS as they account for the non-linear and higher order relationships often present in EEG/EOG data; they routinely achieve classification accuracy greater than 80%. However, the discrete output of these classification schemes is not conducive to real-time change detection. They accurately classify OFS, but they do not indicate when OFS has changed; the change points remain ambiguous and left to subjective interpretation. Thus, the present study introduces several online algorithms which objectively determine change in OFS via real-time psychophysiological signals. The following chapters describe the dataset evaluated, discuss the statistical properties of psychophysiological signals, and detail various algorithms which utilize these signals to detect real-time changes in OFS. The results of the algorithms are presented along with a discussion. Finally, the study is concluded with a comparison of each method and recommendations for future application.