Introduction

The developed methodology allows to carry out the recognition of facial expressions (FER) in real time. The data is acquired through RGB-D cameras, which simultaneously capture a color and a depth video stream; the use of a multimodal approach based on Deep Learning allows you to maximize the flexibility and accuracy of recognition.

Technical features

The developed methodology consists of two main phases. The first one is the processing of the frames, which aims to adapt the images acquired with RGB-D cameras to to use them as input for the second phase, within which the classification of facial expressions takes place. The classification using Deep Learning algorithms is composed of a training phase, during which the network learns how to carry out the classification, and the testing phase, during which the network can be used to perform the recognition. In our case the data used for the training are ecologically valid, therefore the facial expressions are more authentic and consequently suitable for a real context; moreover, the data used are color and depth images and related geometric descriptors, to maximize the information used and improve the recognition rate. TRL: 4

Possible Applications

• Emotional design: support for defining the requirements of a product/service;
• Marketing: tool for consumer profiling;
• Robotics: improvement of human-machine interaction;
• Ethics: aid for subjects with problems relating to the understanding and manifestation of emotions.

Advantages

• Ensures a higher recognition rate;
• Allows for recognition in a greater number of use cases;
• RGB-D cameras can be integrated into personal devices  and are low cost.