A Smartphone-Based Cursor Position System in Cross-Device Interaction Using Machine Learning Techniques

• Main Authors: Juechen Yang, Jun Kong, Chunying Zhao
• Format: Article
• Language: English
• Published: MDPI AG 2021-02-01
• Series: Sensors
• Subjects: cross-device interaction; motion detection; gestural interaction; large display; mobile device
• Online Access: https://www.mdpi.com/1424-8220/21/5/1665

The use of mobile devices, especially smartphones, has become popular in recent years. There is an increasing need for cross-device interaction techniques that seamlessly integrate mobile devices and large display devices together. This paper develops a novel cross-device cursor position system that maps a mobile device’s movement on a flat surface to a cursor’s movement on a large display. The system allows a user to directly manipulate objects on a large display device through a mobile device and supports seamless cross-device data sharing without physical distance restrictions. To achieve this, we utilize sound localization to initialize the mobile device position as the starting location of a cursor on the large screen. Then, the mobile device’s movement is detected through an accelerometer and is accordingly translated to the cursor’s movement on the large display using machine learning models. In total, 63 features and 10 classifiers were employed to construct the machine learning models for movement detection. The evaluation results have demonstrated that three classifiers, in particular, gradient boosting, linear discriminant analysis (LDA), and naïve Bayes, are suitable for detecting the movement of a mobile device.