Inferring Human Intent from Video by Sampling Hierarchical Plans

© 2016 IEEE. This paper presents a method which allows robots to infer a human's hierarchical intent from partially observed RGBD videos by imagining how the human will behave in the future. This capability is critical for creating robots which can interact socially or collaboratively with huma...

Full description

Bibliographic Details
Main Authors: Holtzen, Steven (Author), Zhao, Yibiao (Author), Gao, Tao (Author), Tenenbaum, Joshua B. (Author), Zhu, Song-Chun (Author)
Other Authors: Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences (Contributor)
Format: Article
Language:English
Published: Institute of Electrical and Electronics Engineers (IEEE), 2021-11-09T21:35:38Z.
Subjects:
Online Access:Get fulltext
Description
Summary:© 2016 IEEE. This paper presents a method which allows robots to infer a human's hierarchical intent from partially observed RGBD videos by imagining how the human will behave in the future. This capability is critical for creating robots which can interact socially or collaboratively with humans. We represent intent as a novel hierarchical, compositional, and probabilistic And-Or graph structure which describes a relationship between actions and plans. We infer human intent by reverseengineering a human's decision-making and action planning processes under a Bayesian probabilistic programming framework. We present experiments from a 3D environment which demonstrate that the inferred human intent (1) matches well with human judgment, and (2) provides useful contextual cues for object tracking and action recognition.