Článek ve sborníku konference

 
Mikolov, T., Kombrink, S., Burget, L., Černocký, J., Khudanpur, S.: Extensions of Recurrent Neural Network Language Model, In: Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, Praha, CZ, IEEESP, 2011, s. 5528-5531, ISBN 978-1-4577-0537-3
Jazyk publikace:angličtina
Název publikace:Extensions of Recurrent Neural Network Language Model
Název (cs):Rozšíření jazykového modelu založeného na rekurentních neuronových sítích
Strany:5528-5531
Sborník:Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011
Konference:International Conference on Acoustics, Speech and Signal Processing 2011
Místo vydání:Praha, CZ
Rok:2011
ISBN:978-1-4577-0537-3
Vydavatel:IEEE Signal Processing Society
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2011/mikolov_icassp2011_5528.pdf [PDF]
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2011/mikolov_icassp2011_presentation_rnnlm-extension.pdf [PDF]
Klíčová slova
language modeling, recurrent neural networks, speech recognition
Anotace
Publikace pojednává o rozšíření jazykového modelu založeného na rekurentních neuronových sítích (RNN).
Abstrakt
We present several modifications of the original recurrent neural network language model (RNN LM).While this model has been shown to significantly outperform many competitive language modeling techniques in terms of accuracy, the remaining problem is the computational complexity. In this work, we show approaches that lead to more than 15 times speedup for both training and testing phases. Next, we show importance of using a backpropagation through time algorithm. An empirical comparison with feedforward networks is also provided. In the end, we discuss possibilities how to reduce the amount of parameters in the model. The resulting RNN model can thus be smaller, faster both during training and testing, and more accurate than the basic one.
BibTeX:
@INPROCEEDINGS{
   author = {Tomáš Mikolov and Stefan Kombrink and Lukáš Burget and Jan
	Černocký and Sanjeev Khudanpur},
   title = {Extensions of Recurrent Neural Network Language Model},
   pages = {5528--5531},
   booktitle = {Proceedings of the 2011 IEEE International Conference on
	Acoustics, Speech, and Signal Processing, ICASSP 2011},
   year = {2011},
   location = {Praha, CZ},
   publisher = {IEEE Signal Processing Society},
   ISBN = {978-1-4577-0537-3},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=9658}
}