Thumb Inclination-Based Manipulation and Exploration, a Machine Learning Based Interaction Technique for Virtual Environments

• Main Authors: Muhammad Raees, Sehat Ullah, Inam Ur Rehman, Muhammad Azhar
• Format: Article
• Language: English
• Published: Mehran University of Engineering and Technology 2021-04-01
• Series: Mehran University Research Journal of Engineering and Technology
• Online Access: https://publications.muet.edu.pk/index.php/muetrj/article/view/2091
• ISSN: 0254-7821; 2413-7219

In the context of Virtual Reality (VR), interactions refer to the plausible actions in a Virtual Environment (VE). To have an engrossing interface, interactions by the gestures of hand are becoming prominent. With this research work, a novel interaction technique is proposed where interactions are performed on the basis of the position of thumb in dynamic image stream. The technique needs no expensive tracker but an ordinary camera to trace hand movements and position of thumb. The interaction tasks are enacted in distinct interaction states, where the Angle of Inclination (AOI) of thumb is used for state-to-state transition. The angle is computed dynamically between the tip-of-thumb and the base of the Region of Interest (ROI) of an input image. The technique works in two phases: learning phase and application phase. In the learning phase, user-defined fistpostures with distinct AOI are learnt. The Support Vector Machine (SVM) classifier is trained by the AOI of the postures. In the application phase, interactions are performed in distinct interaction states whereas a particular state is activated by posing the known posture. To follow the trajectory of thumb, dynamic mapping is performed to control a virtual hand by the position of thumb in the input image. The technique is implemented in a Visual Studio project called Thumb-Based Interaction for Virtual Environments (TIVE). The project was evaluated by a group of 15 users in a moderate lighting condition. The 89.7% average accuracy rate of the evaluation proves suitability of the technique in the wide range VR applications.