Remote heart rate variability for emotional state monitoring

Yannick Benezeth (1) , Peixi Li , Richard Macwan (1) , Keisuke Nakamura (2) , Randy Gomez (2) , Fan Yang (1)

1 Le2i - Laboratoire d'Electronique, d'Informatique et d'Image [EA 7508]
2 Honda Research Institute

Several researches have been conducted to recognize emotions using various modalities such as facial expressions , gestures, speech or physiological signals. Among all these modalities, physiological signals are especially interesting because they are mainly controlled by the autonomic nervous system. It has been shown for example that there is an undeniable relationship between emotional state and Heart Rate Variability (HRV). In this paper, we present a methodology to monitor emotional state from physiological signals acquired remotely. The method is based on a remote photoplethysmography (rPPG) algorithm that estimates remote Heart Rate Variability (rHRV) using a simple camera. We first show that the rHRV signal can be estimated with a high accuracy (more than 96% in frequency domain). Then, frequency-feature of rHRV is calculated and we show that there is a strong correlation between the rHRV feature and different emotional states. This observation has been validated on 12 out of 16 volunteers and video-induced emotions which opens the way to contactless monitoring of emotions from physiological signals.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement du signal et de l'image [eess.SP]

Yannick Benezeth, Peixi Li, Richard Macwan, Keisuke Nakamura, Randy Gomez, et al.. Remote heart rate variability for emotional state monitoring. IEEE International Conference on Biomedical and Health Informatics, 2018, Las Vegas, United States. ⟨hal-01678244v2⟩