Publication Details

Analysis of impact of emotions on target speech extraction and speech separation

ŠVEC Ján, ŽMOLÍKOVÁ Kateřina, KOCOUR Martin, DELCROIX Marc, OCHIAI Tsubasa, MOŠNER Ladislav and ČERNOCKÝ Jan. Analysis of impact of emotions on target speech extraction and speech separation. In: Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022). Bamberg: IEEE Signal Processing Society, 2022, pp. 1-5. ISBN 978-1-6654-6867-1. Available from: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9914718
Czech title
Analýza vlivu emocí na extrakci řeči cílového mluvčího a na separaci řeči
Type
conference paper
Language
english
Authors
Švec Ján, Ing. (DCGM FIT BUT)
Žmolíková Kateřina, Ing., Ph.D. (DCGM FIT BUT)
Kocour Martin, Ing. (DCGM FIT BUT)
Delcroix Marc (NTT)
Ochiai Tsubasa (NTT)
Mošner Ladislav, Ing. (DCGM FIT BUT)
Černocký Jan, prof. Dr. Ing. (DCGM FIT BUT)
URL
Keywords

target speech extraction, SpeakerBeam, speech separation, Conv-TasNet, emotion

Abstract

Recently, the performance of blind speech separation (BSS) and target speech extraction (TSE) has greatly progressed. Most works, however, focus on relatively well-controlled conditions using, e.g., read speech. The performance may degrade in more realistic situations. One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech. In this paper, we investigate the influence of emotions on TSE and BSS. We create a new test dataset of emotional mixtures for the evaluation of TSE and BSS. This dataset combines LibriSpeech and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). Through controlled experiments, we can analyze the impact of different emotions on the performance of BSS and TSE.We observe that BSS is relatively robust to emotions, while TSE, which requires identifying and extracting the speech of a target speaker, is much more sensitive to emotions. On comparative speaker verification experiments we show that identifying the target speaker may be particularly challenging when dealing with emotional speech. Using our findings, we outline potential future directions that could improve the robustness of BSS and TSE systems toward emotional speech.

Published
2022
Pages
1-5
Proceedings
Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022)
Conference
The 17th International Workshop on Acoustic Signal Enhancement, Bamberg, DE
ISBN
978-1-6654-6867-1
Publisher
IEEE Signal Processing Society
Place
Bamberg, DE
DOI
10.1109/IWAENC53105.2022.9914718
EID Scopus
2-s2.0-85141348338
BibTeX 
@INPROCEEDINGS{FITPUB12856,
   author = "J\'{a}n \v{S}vec and Kate\v{r}ina \v{Z}mol\'{i}kov\'{a} and Martin Kocour and Marc Delcroix and Tsubasa Ochiai and Ladislav Mo\v{s}ner and Jan \v{C}ernock\'{y}",
   title = "Analysis of impact of emotions on target speech extraction and speech separation",
   pages = "1--5",
   booktitle = "Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022)",
   year = 2022,
   location = "Bamberg, DE",
   publisher = "IEEE Signal Processing Society",
   ISBN = "978-1-6654-6867-1",
   doi = "10.1109/IWAENC53105.2022.9914718",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/12856"
}
Back to top