Options
Speech Emotion Recognition among Elderly Individuals using Multimodal Fusion and Transfer Learning
Type
conference paper
Date Issued
2020-10-25
Author(s)
Boateng, George
Abstract
Recognizing the emotions of the elderly is important as it could give an insight into their mental health. Emotion recognition systems that work well on the elderly could be used to assess their emotions in places such as nursing homes and could inform the development of various activities and interventions to improve their mental health. However, several emotion recognition systems are developed using data from younger adults. In this work, we train machine learning models to recognize the emotions of elderly individuals via performing a 3-class classification of valence and arousal as part of the INTERSPEECH 2020 Computational Paralinguistics Challenge (COMPARE). We used speech data from 87 participants who gave spontaneous personal narratives. We leveraged a transfer learning approach in which we used pretrained CNN and BERT models to extract acoustic and linguistic features respectively and fed them into separate machine learning models. Also, we fused these two modalities in a multimodal approach. Our best model used a linguistic approach and outperformed the official competition of unweighted average recall (UAR) baseline for valence by 8.8% and the mean of valence and arousal by 3.2%. We also showed that feature engineering is not necessary as transfer learning without fine-tuning performs as well or better and could be leveraged for the task of recognizing the emotions of elderly individuals. This work is a step towards better recognition of the emotions of the elderly which could eventually inform the development of interventions to manage their mental health.
Language
English
HSG Classification
contribution to scientific community
HSG Profile Area
SoM - Business Innovation
Book title
ICMI 2020 Late Breaking Results, Companion
Publisher
ACM
Publisher place
New York, NY, USA
Start page
12
End page
16
Event Title
22nd ACM International Conference on Multimodal Interaction (ICMI)
Event Location
virtual
Event Date
October 25-29
Official URL
Division(s)
Eprints ID
261890