Publication Details

Speaker-aware neural network based beamformer for speaker extraction in speech mixtures

ŽMOLÍKOVÁ Kateřina, DELCROIX Marc, KINOSHITA Keisuke, HIGUCHI Takuya, OGAWA Atsunori and NAKATANI Tomohiro. Speaker-aware neural network based beamformer for speaker extraction in speech mixtures. In: Proceedings of Interspeech 2017. Stocholm: International Speech Communication Association, 2017, pp. 2655-2659. ISSN 1990-9772. Available from: http://www.isca-speech.org/archive/Interspeech_2017/pdfs/0667.PDF
Czech title
Směrovač paprsku založený na neuronové síti poučené o řečníkovi pro extrakci řečníka ze směsi řečových signálů
Type
conference paper
Language
english
Authors
Žmolíková Kateřina, Ing., Ph.D. (DCGM FIT BUT)
Delcroix Marc (NTT)
Kinoshita Keisuke (NTT)
Higuchi Takuya (NTT)
Ogawa Atsunori (NTT)
Nakatani Tomohiro (NTT)
URL
Keywords

speaker extraction, speaker-aware neural network, beamforming, mask estimation

Abstract

This article is about the speaker-aware neural network based beamformer for speaker extraction in speech mixtures. In this work, we address the problem of extracting one target speaker from a multichannel mixture of speech. We use a neural network to estimate masks to extract the target speaker and derive beamformer filters using these masks, in a similar way as the recently proposed approach for extraction of speech in presence of noise. To overcome the permutation ambiguity of neural network mask estimation, which arises in presence of multiple speakers, we propose to inform the neural network about the target speaker so that it learns to follow the speaker characteristics through the utterance. We investigate and compare different methods of passing the speaker information to the network such as making one layer of the network dependent on speaker characteristics. Experiments on mixture of two speakers demonstrate that the proposed scheme can track and extract a target speaker for both closed and open speaker set cases.

Annotation

In this work, we address the problem of extracting one target speaker from a multichannel mixture of speech. We use a neural network to estimate masks to extract the target speaker and derive beamformer filters using these masks, in a similar way as the recently proposed approach for extraction of speech in presence of noise. To overcome the permutation ambiguity of neural network mask estimation, which arises in presence of multiple speakers, we propose to inform the neural network about the target speaker so that it learns to follow the speaker characteristics through the utterance. We investigate and compare different methods of passing the speaker information to the network such as making one layer of the network dependent on speaker characteristics. Experiments on mixture of two speakers demonstrate that the proposed scheme can track and extract a target speaker for both closed and open speaker set cases.

Published
2017
Pages
2655-2659
Journal
Proceedings of Interspeech - on-line, vol. 2017, no. 8, ISSN 1990-9772
Proceedings
Proceedings of Interspeech 2017
Conference
Interspeech Conference, Stockholm, SE
Publisher
International Speech Communication Association
Place
Stocholm, SE
DOI
UT WoS
000457505000551
EID Scopus
BibTeX
@INPROCEEDINGS{FITPUB11587,
   author = "Kate\v{r}ina \v{Z}mol\'{i}kov\'{a} and Marc Delcroix and Keisuke Kinoshita and Takuya Higuchi and Atsunori Ogawa and Tomohiro Nakatani",
   title = "Speaker-aware neural network based beamformer for speaker extraction in speech mixtures",
   pages = "2655--2659",
   booktitle = "Proceedings of Interspeech 2017",
   journal = "Proceedings of Interspeech - on-line",
   volume = 2017,
   number = 08,
   year = 2017,
   location = "Stocholm, SE",
   publisher = "International Speech Communication Association",
   ISSN = "1990-9772",
   doi = "10.21437/Interspeech.2017-667",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/11587"
}
Back to top