Conference on Virtual Reality and 3D User Interfaces, Abstracts and Workshops (VRW)

A Simulator for Human-Robot Interaction in Virtual Reality

Mark Murnane, Padraig Higgins, Monali Saraf, Francis Ferraro, Cynthia Matuszek, and Don Engel

March 27, 2021

We present a suite of tools to model a robot, its sensors, and the surrounding environment in VR, with the goal of collecting training data for real-world robots. The virtual robot observes a rigged avatar created in our photogrammetry facility and embodying a VR user. We are particularly interested in verbal human/robot interactions, which can be combined with the robot’s sensor data for grounded language learning. Because virtual scenes, tasks, and robots are easily reconfigured compared to their physical analogs, our approach proves extremely versatile in preparing a wide range of robot scenarios for an array of use cases.

BibTeX OWL Tweet Scholar

Tags: hri, robotics

Type: InProceedings

Publisher: IEEE

Downloads: 711 downloads