Contact-free Cognitive Load Classification based on Psycho-Physiological Parameters

• Main Authors: Gestlöf, Rikard, Sörman, Johannes
• Format: Others
• Language: English
• Published: Mälardalens högskola, Akademin för innovation, design och teknik 2019
• Subjects: AI; Cognitve Load; Machine Learning; Computer Sciences; Datavetenskap (datalogi)
• Online Access: http://urn.kb.se/resolve?urn=urn:nbn:se:mdh:diva-44695

Cognitive load (CL) is a concept that describes the relationship between the cognitive demands from a task and the environment the task is taking place in, which influences the user’s cognitive resources. High cognitive load leads to higher chance of a mistake while a user is performing a task. CL has great impact on driving performance, although the effect of CL is task dependent. It has been proven that CL selectively impairs non-automized aspects of driving performance while automized driving tasks are unaffected. The most common way of measuring CL is electroencephalography (EEG), which might be a problem in some situations since its contact-based and must be connected to the head of the test subject. Contact-based ways of measuring different physiological parameters can be a problem since they might affect the results of the research. Since the wirings sometimes might be loose and that the test subject moves etc. However, the biggest concern with contact-based systems is that they are hard to involve practically. The reason for this is simply that a user cannot relax, and that the sensors attached to the test subjects can affect them to not provide normal results. The goal of the research is to test the performance of data gathered with a contact-free camera-based system compared to a contact-based shimmer GSR+ system in detecting cognitive load. Both data collection approaches will extract the heart rate (HR) and interbeat interval (IBI) while test subjects perform different tasks during a controlled experiment. Based on the gathered IBI, 13 different heart rate variability (HRV) features will be extracted to determine different levels of cognitive load.  In order to determine which system that is better at measuring different levels of CL, three major stress level phases were used in a controlled experiment. These three stress level phases were the reference point for low CL where test subjects sat normal (S0), normal CL where the test subjects performed easy puzzles and drove normally in a video game (S1) and high CL where the test subjects completed hard puzzles and drove on the hardest course of a video game while answering math questions (S2). To classify the extracted HRV features from the data into the three different levels of CL two different machine learning (ML) algorithms, support vector machine (SVM) and k-nearest-neighbor (KNN) were implemented. Both binary and multiclass feature matrixes were created with all combinations between the different stress levels of the collected data. In order to get the best classification accuracy with the ML algorithms, different optimizations such as kernelfunctions were chosen for the different feature matrixes. The results of this research proved that the ML algorithms achieved a higher classification accuracy for the data collected with the contact-free system than the shimmer sense system. The highest mean classification accuracy was 81% on binary classification for S0-S2 collected by the camera while driving using Fine KNN. The highest F1 score was 88%, which was achieved with medium gaussian SVM for the class combination S0-(S1/S2) feature matrix recorded with the camera system. It was concluded that the data gathered with the contact-free camera system achieved a higher accuracy than the contact-based system. Also, that KNN achieved the higher accuracy overall, than SVM for the data. This research proves that a contact-free camera-based system can detect cognitive better than a contact-based shimmer sense GSR+ system with a high classification accuracy.