Conference paper

VESELÝ Karel, HANNEMANN Mirko and BURGET Lukáš. Semi-supervised Training of Deep Neural Networks. In: Proceedings of ASRU 2013. Olomouc: IEEE Signal Processing Society, 2013, pp. 267-272. ISBN 978-1-4799-2755-5.
Publication language:english
Original title:Semi-supervised Training of Deep Neural Networks
Title (cs):Částečně kontrolované trénování hlubokých neuronových sítí
Pages:267-272
Proceedings:Proceedings of ASRU 2013
Conference:IEEE 2013 Workshop on Automatic Speech Recognition and Understanding
Place:Olomouc, CZ
Year:2013
ISBN:978-1-4799-2755-5
Publisher:IEEE Signal Processing Society
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2013/vesely_asru2013_0000267.pdf [PDF]
Keywords
semi-supervised training, self-training, deep network, DNN, Babel program
Annotation
Our quest in this paper is to search for an optimal dataselection strategy for the semi-supervised DNN training. We performed an analysis at all the three stages of DNN training.
Abstract
In this paper we search for an optimal strategy for semisupervised Deep Neural Network (DNN) training. We assume that a small part of the data is transcribed, while the majority of the data is untranscribed. We explore self-training strategies with data selection based on both the utterance-level and frame-level confidences. Further on, we study the interactions between semi-supervised frame-discriminative training and sequence-discriminative sMBR training. We found it beneficial to reduce the disproportion in amounts of transcribed and untranscribed data by including the transcribed data several times, as well as to do a frame-selection based on per-frame confidences derived from confusion in a lattice. For the experiments, we used the Limited language pack condition for the Surprise language task (Vietnamese) from the IARPA Babel program. The absolute Word Error Rate (WER) improvement for frame cross-entropy training is 2.2%, this corresponds to WER recovery of 36% when compared to the identical system, where the DNN is built on the fully transcribed data.
BibTeX:
@INPROCEEDINGS{
   author = {Karel Vesel{\'{y}} and Mirko Hannemann and Luk{\'{a}}{\v{s}}
	Burget},
   title = {Semi-supervised Training of Deep Neural Networks},
   pages = {267--272},
   booktitle = {Proceedings of ASRU 2013},
   year = {2013},
   location = {Olomouc, CZ},
   publisher = {IEEE Signal Processing Society},
   ISBN = {978-1-4799-2755-5},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=10509}
}

Your IPv4 address: 54.145.92.172
Switch to IPv6 connection

DNSSEC [dnssec]