Conference paper

MIKOLOV Tomáš, KOMBRINK Stefan, BURGET Lukáš, ČERNOCKÝ Jan and KHUDANPUR Sanjeev. Extensions of Recurrent Neural Network Language Model. In: Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011. Praha: IEEE Signal Processing Society, 2011, pp. 5528-5531. ISBN 978-1-4577-0537-3.
Publication language:english
Original title:Extensions of Recurrent Neural Network Language Model
Title (cs):Rozšíření jazykového modelu založeného na rekurentních neuronových sítích
Pages:5528-5531
Proceedings:Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011
Conference:International Conference on Acoustics, Speech and Signal Processing 2011
Place:Praha, CZ
Year:2011
ISBN:978-1-4577-0537-3
Publisher:IEEE Signal Processing Society
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2011/mikolov_icassp2011_5528.pdf [PDF]
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2011/mikolov_icassp2011_presentation_rnnlm-extension.pdf [PDF]
Keywords
language modeling, recurrent neural networks, speech recognition
Annotation
This paper describes results that we obtained when using extensions of Recurrent Neural Network (RNN) Language Model.
Abstract
We present several modifications of the original recurrent neural network language model (RNN LM).While this model has been shown to significantly outperform many competitive language modeling techniques in terms of accuracy, the remaining problem is the computational complexity. In this work, we show approaches that lead to more than 15 times speedup for both training and testing phases. Next, we show importance of using a backpropagation through time algorithm. An empirical comparison with feedforward networks is also provided. In the end, we discuss possibilities how to reduce the amount of parameters in the model. The resulting RNN model can thus be smaller, faster both during training and testing, and more accurate than the basic one.
BibTeX:
@INPROCEEDINGS{
   author = {Tomáš Mikolov and Stefan Kombrink and Lukáš Burget and Jan
	Černocký and Sanjeev Khudanpur},
   title = {Extensions of Recurrent Neural Network Language Model},
   pages = {5528--5531},
   booktitle = {Proceedings of the 2011 IEEE International Conference on
	Acoustics, Speech, and Signal Processing, ICASSP 2011},
   year = {2011},
   location = {Praha, CZ},
   publisher = {IEEE Signal Processing Society},
   ISBN = {978-1-4577-0537-3},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=9658}
}

Your IPv4 address: 54.234.0.85
Switch to IPv6 connection

DNSSEC [dnssec]