Publication Details

Multimodal Emotion Recognition for AVEC 2016 Challenge

POVOLNÝ Filip, MATĚJKA Pavel, HRADIŠ Michal, POPKOVÁ Anna, OTRUSINA Lubomír, SMRŽ Pavel, WOOD Ian, ROBIN Cécile and LAMEL Lori. Multimodal Emotion Recognition for AVEC 2016 Challenge. In: AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. Amsterdam: Association for Computing Machinery, 2016, pp. 75-82. ISBN 978-1-4503-4516-3. Available from: http://dl.acm.org/citation.cfm?id=2988268
Czech title
Multimodální rozpoznávání emocí pro AVEC 2016
Type
conference paper
Language
english
Authors
Povolný Filip, Ing. (Phonexia)
Matějka Pavel, Ing., Ph.D. (DCGM FIT BUT)
Hradiš Michal, Ing., Ph.D. (DCGM FIT BUT)
Popková Anna, Ing. (FIT BUT)
Otrusina Lubomír, Ing. (DCGM FIT BUT)
Smrž Pavel, doc. RNDr., Ph.D. (DCGM FIT BUT)
Wood Ian (NUIG)
Robin Cécile (NUIG)
Lamel Lori (LIMSI)
URL
Keywords

emotion recognition, valence, arousal, bottleneck features, neural networks, regression, speech transcription, word embedding

Abstract

This paper describes a systems for emotion recognition and its application on the dataset from the AV+EC 2016 Emotion Recognition Challenge. The realized system was produced and submitted to the AV+EC 2016 evaluation, making use of all three modalities (audio, video, and physiological data). Our work primarily focused on features derived from audio. The original audio features were complement with bottleneck features and also text-based emotion recognition which is based on transcribing audio by an automatic speech recognition system and applying resources such as word embedding models and sentiment lexicons. Our multimodal fusion reached CCC=0.855 on dev set for arousal and 0.713 for valence. CCC on test set is 0.719 and 0.596 for arousal and valence respectively.

Published
2016
Pages
75-82
Proceedings
AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge
Conference
The 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, NL
ISBN
978-1-4503-4516-3
Publisher
Association for Computing Machinery
Place
Amsterdam, NL
DOI
UT WoS
000390604700011
EID Scopus
BibTeX
@INPROCEEDINGS{FITPUB11345,
   author = "Filip Povoln\'{y} and Pavel Mat\v{e}jka and Michal Hradi\v{s} and Anna Popkov\'{a} and Lubom\'{i}r Otrusina and Pavel Smr\v{z} and Ian Wood and C\'{e}cile Robin and Lori Lamel",
   title = "Multimodal Emotion Recognition for AVEC 2016 Challenge",
   pages = "75--82",
   booktitle = "AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge",
   year = 2016,
   location = "Amsterdam, NL",
   publisher = "Association for Computing Machinery",
   ISBN = "978-1-4503-4516-3",
   doi = "10.1145/2988257.2988268",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/11345"
}
Files
Back to top