Seminář UPSY - Koutnik J.: Deep Reinforcement Learning
Neuroevolution is a powerful technique for training neural networks for tasks like reinforcement learning in which, because there are no output targets, gradient information for adapting the weights can be unreliable. This talk will introduce two methods for scaling up neuroevolution in order to move away from toy problems towards challenging high-dimensional continuous reinforcement learning problems: compressed network search that represents neural network weights indirectly as a set of frequency domain coefficients, which allow very large networks to be evolved by searching in low-dimensional coefficient space; and deep-convolutional pre-processors, that transform high-dimensional input to low-dimensional feature vectors that are sufficiently compact and can be used as an input for small recurrent neural network controller. The performance of the methods is demonstrated on controlling a race car to drive along a track using solely a high-dimensional visual input.
Jan Koutnik received his Ph.D. in computer science from the Czech Technical University at Prague in 2008. He works as machine learning researcher at The Swiss AI Lab IDSIA. His research is mainly focused on artificial neural networks, recurrent neural networks, evolutionary algorithms and deep-learning applied to reinforcement learning, control problems, image classification, handwriting and speech recognition.