NUST Institutions Library Catalogue

Existing frameworks lack the support for testing Human Robot Interaction (HRI) research
which in turn often has to be tested practically making it time consuming and expensive. To
overcome this issue, a HRI framework based on Unreal Engine is proposed which consists of a
virtual Nao or Pepper robot along with virtual humans with verbal and non-verbal behaviours in
an environment. Machine Learning (ML) algorithms along with the behaviour of the virtual
robot in response to the interaction with the virtual humans and the environment can be
programmed using a Python API which communicates with Unreal Engine C++ in real time.
Several experiments related to multiple aspects of HRI: (1) Verbal Interaction; (2) Non-Verbal
Interaction; (3) Emotional Interaction were conducted in both the virtual and real world
environments and the results were compared to validate the feasibility of the framework. A
Reinforcement Learning (RL) algorithm was also tested to further indicate the usefulness of the
framework. Through the use of a Virtual Reality (VR) headset, a human can be immersed in the
framework to interact with the robot in real time