Service Action Recognition in Power Supply Business Hall with 3D-Fused ConvNet

• Main Authors: Tongyao Lin, Li Ouyang, He Wen, Dezhi Xiong, Janusz Smulko
• Format: Article
• Language: English
• Published: VSB-Technical University of Ostrava 2021-01-01
• Series: Advances in Electrical and Electronic Engineering
• Subjects: 3d convolution; action recognition; ensemble; power supply business hall.
• Online Access: http://advances.utc.sk/index.php/AEEE/article/view/3950

For the purpose of improving the service quality, video surveillance systems are widely used to standardize the service process in power supply business halls. If the employers check surveillance video to ensure predefined process of staff behaviours, it will be characterized as time-consuming. In recent years, great progress has been made in intelligent action recognition using Convolution Neural Networks (CNNs). However, due to the small range of staffs' motion and similar scene information of power supply business halls, the performance of using traditional CNNs to recognize service actions, e.g. bowing, standing and sitting, is general. For improving the recognition rate, this paper proposes a 3D-fused Convolutional Network (ConvNet) for service actions recognition, which focuses on detecting the actions in the typical scene of one staff person and one customer with a well-segmented video clip. The well-segmented video clips are sent as input to the 3D-fused ConvNet for action recognition. The 3D-fused ConvNet consists of two base learners, optical flow base learner and RGB base learner. Both learners use the Convolutional 3D (C3D) architecture. Specifically, the RGB learner can be used to capture the features of small staffs' motion while the optical flow base learner can be viewed as the key part to eliminate the influence of the background, especially in a similar scene. Furthermore, prediction scores of two base learners can be weighted by the softmax function according to the performance of each base learner. Finally, the prediction scores of the two base learners are fused to obtain the prediction result, namely the specific actions of the staffs in the videos. The experiment result shows that the proposed method achieves 92.41% accuracy on the service action dataset of the power supply business hall.