Probabilistic Mapping of Human Visual Attention from Head Pose Estimation

• Main Authors: Andrea Veronese, Mattia Racca, Roel Stephan Pieters, Ville Kyrki
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2017-10-01
• Series: Frontiers in Robotics and AI
• Subjects: object detection; attention detection; visual attention mapping; head pose; 3D point cloud; human–robot interaction
• Online Access: http://journal.frontiersin.org/article/10.3389/frobt.2017.00053/full

Effective interaction between a human and a robot requires the bidirectional perception and interpretation of actions and behavior. While actions can be identified as a directly observable activity, this might not be sufficient to deduce actions in a scene. For example, orienting our face toward a book might suggest the action toward “reading.” For a human observer, this deduction requires the direction of gaze, the object identified as a book and the intersection between gaze and book. With this in mind, we aim to estimate and map human visual attention as directed to a scene, and assess how this relates to the detection of objects and their related actions. In particular, we consider human head pose as measurement to infer the attention of a human engaged in a task and study which prior knowledge should be included in such a detection system. In a user study, we show the successful detection of attention to objects in a typical office task scenario (i.e., reading, working with a computer, studying an object). Our system requires a single external RGB camera for head pose measurements and a pre-recorded 3D point cloud of the environment.