Real time single and multi-gesture recognition based on skin colour and optical flow

• Main Author: Ali, Muhammad Raza
• Other Authors: Morris, Tim
• Published: University of Manchester 2013
• Subjects: 006.3
• Online Access: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.574309

This thesis discusses our research conducted in the area of hand gesture recognition. The research objectives were to develop techniques that lead to accurate and robust gesture recognition under everyday settings. And work with consistent accuracy in both single and multiuser scenarios. In this research, we propose techniques that rely on the combination of skin colour and optical flow. A background subtraction stage involves identifying skin regions in an image frame. We use skin colour thresholds in chromaticity space. In our work, we have simplified the process by identifying a reliable set of thresholds without camera calibration and a specialized imaging setup. In order to tackle the issue of false positives we combine skin colour with optical flow magnitude i.e. joint thresholding. We propose a novel skin colour-optical flow metric to track an arbitrarily changing number of skin regions. The proposed technique has been successfully applied to Bayesian and non-Bayesian tracking. We use a novel feature descriptor to represent a gesture making hand i.e. the Radon transform of its contour. The tracking mechanism and gesture classification is tested for single and simultaneous multi-gesture classification. We also propose a novel technique for grouping skin regions belonging to a particular person. In our work, we first try to establish the potential usefulness of using standard HCI techniques by evaluating our real time application. Based on the results, we propose a usability evaluation framework. We formalize usability evaluation for interactive vision systems by incorporating the standard practice of prototyping and user feedback. This framework can be helpful in conducting a well rounded evaluation.