“You made me feel this way”: Investigating Partners’ Influence in Predicting Emotions in Couples’ Conflict Interactions using Speech Data
Type
conference paper
Date Issued
2021-10-18
Author(s)
Abstract
How romantic partners interact with each other during a conflict influences how they feel at the end of the interaction and is predictive of whether the partners stay together in the long term. Hence understanding the emotions of each partner is important. Yet current approaches that are used include self-reports which are burdensome and hence limit the frequency of this data collection. Automatic emotion prediction could address this challenge. Insights from psychology research indicate that partners’ behaviors influence each other’s emotions in conflict interaction and hence, the behavior of both partners could be considered to better predict each partner’s emotion. However, it is yet to be investigated how doing so compares to only using each partner’s own behavior in terms of emotion prediction performance. In this work, we used BERT to extract linguistic features (i.e., what partners said) and openSMILE to extract paralinguistic features (i.e., how they said it) from a data set of 368 German-speaking Swiss couples (N = 736 individuals) who were videotaped during an 8-minutes conflict interaction in the laboratory. Based on those features, we trained machine learning models to predict if partners feel positive or negative after the conflict interaction. Our results show that including the behavior of the other partner improves the prediction performance. Furthermore, for men, considering how their female partners spoke is most important and for women considering what their male partner said is most important in getting better prediction performance. This work is a step towards automatically recognizing each partners’ emotion based on the behavior of both, which would enable a better understanding of couples in research, therapy, and the real world.
Language
English
Keywords
couples
emotion recognition
multimodal fusion
linguistic
par- alinguistic
conflict
HSG Classification
contribution to scientific community
HSG Profile Area
SoM - Business Innovation
Book title
Companion Publication of the 2021 International Conference on Multimodal Interaction
Publisher
ACM
Publisher place
New York, NY, USA
Start page
390
End page
394
Event Title
ACM 2021 International Conference on Multimodal Interaction
Event Location
Montréal, QC, Canada
Event Date
October 18–22, 2021
Official URL
Division(s)
Eprints ID
265545
File(s)![Thumbnail Image]()
open.access
Name
Boateng et al 2021 Emotions in Couples Conflict.pdf
Size
455.31 KB
Format
Adobe PDF
Checksum (MD5)
a413fb276b7fbe7c5577dc4415583feb