Investigating an Integrated Sensor Fusion System for Mental Fatigue Assessment for Demanding Maritime Operations

• Main Authors: Thiago Gabriel Monteiro, Guoyuan Li, Charlotte Skourup, Houxiang Zhang
• Format: Article
• Language: English
• Published: MDPI AG 2020-05-01
• Series: Sensors
• Subjects: physiological sensors; mental fatigue; maritime operations; deep learning
• Online Access: https://www.mdpi.com/1424-8220/20/9/2588

Human-related issues are currently the most significant factor in maritime causalities, especially in demanding operations that require coordination between two or more vessels and/or other maritime structures. Some of these human-related issues include incorrect, incomplete, or nonexistent following of procedures; lack of situational awareness; and physical or mental fatigue. Among these, mental fatigue is especially dangerous, due to its capacity to reduce reaction time, interfere in the decision-making process, and affect situational awareness. Mental fatigue is also especially hard to identify and quantify. Self-assessment of mental fatigue may not be reliable and few studies have assessed mental fatigue in maritime operations, especially in real time. In this work we propose an integrated sensor fusion system for mental fatigue assessment using physiological sensors and convolutional neural networks. We show, by using a simulated navigation experiment, how data from different sensors can be fused into a robust mental fatigue assessment tool, capable of achieving up to <inline-formula> <math display="inline"> <semantics> <mrow> <mn>100</mn> <mo>%</mo> </mrow> </semantics> </math> </inline-formula> detection accuracy for single-subject classification. Additionally, the use of different sensors seems to favor the representation of the transition between mental fatigue states.