Aggregating Two-Stream Trajectory using Neural Network for Counting Arbitrary Human Action Repetition

• Main Authors: Chih-Yu Lin, 林之宇
• Other Authors: 徐宏民
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/k9343t

碩士 === 國立臺灣大學 === 資訊工程學研究所 === 106 === Although deep neural network has achieved great success in computer vision recently, the problem of determining repetitions of arbitrary periodic human actions is still challenging. The difficulties lay in varying frame length of repetitions, temporal localization of human beings and different features corresponding to different motions. Moreover, the demand of human action repetition counting is rising in medical rehabilitation and sport events, etc. To address this problem, we construct a human action dataset and propose a brand new framework, Human Action Repetition Counter (HARC), which could work on arbitrary human actions with a single architecture. Our HARC learns to count repetitions of human action in the time-frequency domain determined after few pilot studies. The experiments show that HARC outperforms previous counting methods on benchmarks. Additionally, we design novel learning strategies by generating effective synthetic data to pretrain our network, which can further boost the performance and reach more accurate results. We also demonstrate that our HARC is also capable of counting the periodic object motions. Our dataset, YT_Human_Segments dataset, will be publicly available which will benefit future researches.