Mark Murnane
ABSTRACT
There is increasing interest in using robots in simulation to understand and improve human-robot interaction (HRI). At the same time, the use of simulated settings to gather training data promises to help address a major data bottleneck in allowing robots to take advantage of powerful machine learning approaches. In this paper, we describe a prototype system that combines the robot operating system (ROS), the simulator Gazebo, and the Unity game engine to create human-robot interaction scenarios. A person can engage with the scenario using a monitor wall, allowing simultaneous collection of realistic sensor data and traces of human actions.
Supplemental Material
- Robert Codd-Downey, P Mojiri Forooshani, Andrew Speers, Hui Wang, and Michael Jenkin. 2014. From ROS to Unity: Leveraging robot and virtual environment middleware for immersive teleoperation. In IEEE ICIA.Google Scholar
- DeepMotion. 2018. How To Make 3 Point Tracked Full-Body Avatars in VR. http://tiny.cc/3pt-deepmotionGoogle Scholar
- Mark Murnane, Max Breitmeyer, Cynthia Matuszek, and Don Engel. 2019. Virtual Reality and Photogrammetry for Improved Reproducibility of Human-Robot Interaction Studies. In IEEEVR. IEEE Press, Osaka, Japan.Google Scholar
- Adam Philpot, Maxine Glancy, Peter J Passmore, Andrew Wood, and Bob Fields. 2017. User Experience of Panoramic Video in CAVE-like and Head Mounted dDisplay Viewing Conditions. In ACM TVX. ACM, Hilversum, The Netherlands, 65--75. Google ScholarDigital Library
- David Whitney, Eric Rosen, Daniel Ullman, Elizabeth Phillips, and Stefanie Tellex. 2018. ROS Reality: A Virtual Reality Framework Using Consumer-Grade Hardware for ROS-Enabled Robots. In IROS. IEEE, 1--9.Google Scholar
Index Terms
- Learning from human-robot interactions in modeled scenes
Recommendations
Acquiring Human-Robot Interaction skills with Transfer Learning Techniques
HRI '17: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot InteractionRobotics is a reasonably mature technology when robots are restricted to operating with well-known and well-engineered environments, e.g. in manufacturing robotics or domestic applications such as vacuum cleaning or lawn mowing. For more diverse tasks ...
Supervised autonomy for online learning in human-robot interaction
When a robot is learning it needs to explore its environment and how its environment responds on its actions. When the environment is large and there are a large number of possible actions the robot can take, this exploration phase can take ...
Deep Reinforcement Learning for Humanoid Robot Behaviors
AbstractRoboCup 3D Soccer Simulation is a robot soccer competition based on a high-fidelity simulator with autonomous humanoid agents, making it an interesting testbed for robotics and artificial intelligence. Due to the recent success of Deep ...
Reviews
Giuseppina Carla Gini
Usually robots are checked and tested in virtual environments before delivering them to the real world. In this poster, however, the robot is only virtual, and users interacting with it are also rendered in virtual reality. The authors hope to improve human-robot interaction through learning, and so the virtual game is a way to generate and collect large quantities of sensor data and sequences of human actions. This data could be given to learning algorithms, which cannot work in the absence of data, as for most human-robot interactions. Moreover, adding a microphone and speech recognition would aid in collecting training data for grounded natural language systems. The interesting point is that the described system can allow learning in new domains. However, no dataset is yet available, and learning is not yet there. Robotics researchers can work on similar ideas to create datasets, and cognitive scientists could evaluate how the simulated environment can generate the same behaviors in real life.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments