Generating Robotic Emotional Body Language of Targeted Valence and Arousal with a CVAE

Emotional body language (or EBL for short) is crucial in human communication so it comes as no surprise that it is also important in social HRI too. Our goal is to propose an automatic method to generate numerous robotic EBL animations of high granularity and believability, for robot specific morphology and kinetics. Furthermore, we want to be able to generate animations of targeted valence and arousal. In our previous work, we used a deep Variational Autoencoder network which was trained with a small set of EBL motion sequences, specifically designed for a Pepper robot by professional animators. However, the valence and arousal of the generated expressions was not controlled.

00:54 - In our current work, we solve this by using a Conditional Variational Autoencoder. Valence conditioning is concatenated to the input (that is the c block in the graph), while arousal was modeled as a parameter for sampling the latent space of the model. To boost expressiveness even further, this time we train the net with eye LED color sequences concatenated with motion. We tested how distinguishable the emotion conditioning is with 20 participants who watched 18 generated animation. For valence conditioning, on the left, animations generated as positive or neutral received significantly higher valence ratings compared to animations generated as negative.

01:44 - For arousal conditioning, on the right, animations generated as low or medium arousal were given significantly lower arousal ratings compared to animations generated as high arousal. .