Speech Emotion Recognition among Couples using the Peak-End Rule and Transfer Learning
Type
conference paper
Date Issued
2020-10-25
Author(s)
Abstract
Extensive couples’ literature shows that how couples feel after a conflict is predicted by certain emotional aspects of that conver- sation. Understanding the emotions of couples leads to a better understanding of partners’ mental well-being and consequently their relationships. Hence, automatic emotion recognition among couples could potentially guide interventions to help couples im- prove their emotional well-being and their relationships. It has been shown that people’s global emotional judgment after an experience is strongly influenced by the emotional extremes and ending of that experience, known as the peak-end rule. In this work, we leveraged this theory and used machine learning to investigate, which au- dio segments can be used to best predict the end-of-conversation emotions of couples. We used speech data collected from 101 Dutch- speaking couples in Belgium who engaged in 10-minute long con- versations in the lab. We extracted acoustic features from (1) the audio segments with the most extreme positive and negative rat- ings, and (2) the ending of the audio. We used transfer learning in which we extracted these acoustic features with a pre-trained convolutional neural network (YAMNet). We then used these fea- tures to train machine learning models — support vector machines — to predict the end-of-conversation valence ratings (positive vs negative) of each partner. The results of this work could inform how to best recognize the emotions of couples after conversation- sessions and eventually, lead to a better understanding of couples’ relationships either in therapy or in everyday life.
Language
English
HSG Classification
contribution to scientific community
HSG Profile Area
SoM - Business Innovation
Book title
ICMI 2020 Late Breaking Results, Companion
Publisher
ACM
Publisher place
New York, NY, USA
Start page
17
End page
21
Event Title
22nd ACM International Conference on Multimodal Interaction (ICMI)
Event Location
Virtual
Event Date
October 25-29
Official URL
Division(s)
Eprints ID
261889
File(s)![Thumbnail Image]()
Name
Boateng et al 2020 ICMI Couples.pdf
Size
1.06 MB
Format
Adobe PDF
Checksum (MD5)
bc178605e235c07534065d568c902778