Recurrence plots and CNN based method for human activity recognition

• Online Access: http://hdl.handle.net/2047/D20416631

Human activity recognition (HAR) is the process of predicting different human activity states (walking, running, walking, etc.) based on sensor data. HAR is mainly used in healthcare for geriatric activity monitoring and human machine interaction. Recurrence plot (RP) is visualization of a non-linear time series in a matrix form where the matrix elements correspond to the recurrence of a dynamical state. Recurrent Quantification Analysis (RQA) is a popular method to extract features from recurrence plot that can be used to represent the underlying dynamics of the time series. RQA quantifies the number and duration of recurrences of a dynamical system presented by its phase space trajectory. Human activity monitoring generates continuous sensor data and RQA computation is computationally expensive for long time series. To address this issue, in this work we investigate the effectiveness of the image classification method to classify a given recurrence plot. We use real-world human activity data to demonstrate the proposed approach. The activities from human subjects are measured by accelerometers and gyroscopes positioned at 7 different body locations (head, forearm, upper arm, chest, thigh, shin, and waist). For the classification task, we select running, walking, climbing up, and climb-ing down. The sensor signals from the accelerometer and gyroscope are captured for the aforementioned human activities. These sensor signals are converted into recurrence plots. The threshold value is set as 0.5 and 0.8 for generating the recurrence plots. The size of the recurrence plot is set to 32x32 pixels. Then a Convolutional Neural Network (CNN) is used to classify the human activities using the recurrence plots. A Le-Net architecture is selected for the deep learning process. Recurrence plots of different human activities are fed into the LeNet5 algorithm in the form of epochs, with a batch size of 256 plots. We found that among all the sensor positions, the sensors fixed on shin thigh and waist perform best and when the threshold is set as 0.8, the recognition will have a higher ac-curacy. We also found that the signals from accelerometers perform better than those collected by gyroscopes for the recognition task. The results indicate that the method of recurrence plot in combination with CNN performs well in recognizing human activity with high accuracy and high efficiency. The proposed method can be used for classification of non-linear time series for applications other than HAR, providing an alternative to features generated using RQA.--Author's abstract