The Seven Faces of Stress: Understanding Facial Activity Patterns During Cognitive Stress

Stress has been recognized as one of the main contributors to mental health problems, as well as cardiovascular diseases. To reduce the risk of severe diseases, early detection of stress is needed. One of the recent methods studied to detect stress is through facial expression analysis from videos. Although computer vision techniques combined with deep learning have been shown to detect stressful faces, there is a lack of work attempting to define how stressful faces look. One of the main challenges is that the expression of stress is person-dependent and one individual can show stress in various ways. In this work, we present a semi-automatic method that allows to distill from a large quantity of data facial activity patterns that are recognized to show stress. We are the first to combine quantitative and qualitative methods on data from 115 subjects to identify and propose seven facial activity patterns during stress. We support this proposal by analyzing the relationship of the different stress facial expressions with the basic emotions and show how individual components of anger, fear, surprise, and sadness co-occur during our defined stress facial activity patterns.