Conference paper

 
Veselý, K., Burget, L., Grézl, F.: Parallel Training of Neural Networks for Speech Recognition, In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010), Makuhari, Chiba, JP, ISCA, 2010, p. 2934-2937, ISSN 1990-9772
Publication language:english
Original title:Parallel Training of Neural Networks for Speech Recognition
Title (cs):Paralelní trénování neuronových sítí pro rozpoznávání řeči
Pages:2934-2937
Proceedings:Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010)
Conference:Interspeech 2010
Place:Makuhari, Chiba, JP
Year:2010
Journal:Proceedings of Interspeech, Vol. 2010, No. 9, FR
ISSN:1990-9772
Publisher:International Speech Communication Association
URL:http://www.fit.vutbr.cz/research/groups/speech/publi/2010/vesely_interspeech2010_IS100045.pdf [PDF]
Keywords
Artificial Neural Network, GPU, CUDA, Phoneme Classification, Fast Training
Annotation
The paper is on study and implementation of two different approaches to the parallelization of the ANN training procedure for sequential patterns.
Abstract
In this paper we describe parallel implementation of ANN training procedure based on block mode back-propagation learning algorithm. Two different approaches to parallelization were implemented. The first is data parallelization using POSIX threads, it is suitable for multi-core computers. The second is node parallelization using high performance SIMD architecture of GPU with CUDA, suitable for CUDA enabled computers. We compare the speed-up of both approaches by learning typically-sized network on the real-world phoneme-state classification task, showing nearly 10 times reduction when using CUDA version, while the 8-core server with multi-thread version gives only 4 times reduction. In both cases we compared to an already BLAS optimized implementation. The training tool will be released as Open-Source software under project name TNet.
BibTeX:
@INPROCEEDINGS{
   author = {Karel Veselý and Lukáš Burget and František Grézl},
   title = {Parallel Training of Neural Networks for Speech Recognition},
   pages = {2934--2937},
   booktitle = {Proceedings of the 11th Annual Conference of the
	International Speech Communication Association (INTERSPEECH
	2010)},
   journal = {Proceedings of Interspeech},
   volume = {2010},
   number = {9},
   year = {2010},
   location = {Makuhari, Chiba, JP},
   publisher = {International Speech Communication Association},
   ISSN = {1990-9772},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php?id=9364}
}