Synthesizing Depth Hand Images with GANs and Style Transfer for Hand Pose Estimation

• Main Authors: Wangyong He, Zhongzhao Xie, Yongbo Li, Xinmei Wang, Wendi Cai
• Format: Article
• Language: English
• Published: MDPI AG 2019-07-01
• Series: Sensors
• Subjects: hand pose estimation; generative adversarial networks; style transfer; human-computer interaction; depth images
• Online Access: https://www.mdpi.com/1424-8220/19/13/2919

Hand pose estimation is a critical technology of computer vision and human-computer interaction. Deep-learning methods require a considerable amount of tagged data. Accordingly, numerous labeled training data are required. This paper aims to generate depth hand images. Given a ground-truth 3D hand pose, the developed method can generate depth hand images. To be specific, a ground truth can be 3D hand poses with the hand structure contained, while the synthesized image has an identical size to that of the training image and a similar visual appearance to the training set. The developed method, inspired by the progress in the generative adversarial network (GAN) and image-style transfer, helps model the latent statistical relationship between the ground-truth hand pose and the corresponding depth hand image. The images synthesized using the developed method are demonstrated to be feasible for enhancing performance. On public hand pose datasets (NYU, MSRA, ICVL), comprehensive experiments prove that the developed method outperforms the existing works.