Conference paper

 
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., Khudanpur, S.: Recurrent neural network based language model, In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010), Makuhari, Chiba, JP, ISCA, 2010, p. 1045-1048, ISBN 978-1-61782-123-3, ISSN 1990-9772
Publication language:english
Original title:Recurrent neural network based language model
Title (cs):Jazykový model založený na rekurentních neuronových sítích
Pages:1045-1048
Proceedings:Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010)
Conference:Interspeech 2010
Place:Makuhari, Chiba, JP
Year:2010
ISBN:978-1-61782-123-3
Journal:Proceedings of Interspeech, Vol. 2010, No. 9, FR
ISSN:1990-9772
Publisher:International Speech Communication Association
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf [PDF]
Keywords
language modeling, recurrent neural networks, speech recognition
Annotation
This paper is on new application to speech recognition, the recurrent neural network based language model (RNN LM).
Abstract
A new recurrent neural network based language model (RNN LM) with applications to speech recognition is presented. Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model. Speech recognition experiments show around 18% reduction of word error rate on the Wall Street Journal task when comparing models trained on the same amount of data, and around 5% on the much harder NIST RT05 task, even when the backoff model is trained on much more data than the RNN LM. We provide ample empirical evidence to suggest that connectionist language models are superior to standard n-gram techniques, except their high computational (training) complexity.
BibTeX:
@INPROCEEDINGS{
   author = {Tomáš Mikolov and Martin Karafiát and Lukáš Burget and Jan
	Černocký and Sanjeev Khudanpur},
   title = {Recurrent neural network based language model},
   pages = {1045--1048},
   booktitle = {Proceedings of the 11th Annual Conference of the
	International Speech Communication Association (INTERSPEECH
	2010)},
   journal = {Proceedings of Interspeech},
   volume = {2010},
   number = {9},
   year = {2010},
   location = {Makuhari, Chiba, JP},
   publisher = {International Speech Communication Association},
   ISBN = {978-1-61782-123-3},
   ISSN = {1990-9772},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=9362}
}