Seminář UPSY - Koutnik J.: Deep Reinforcement Learning

Neuroevolution is a powerful technique for training neural networks for tasks like reinforcement learning in which, because there are no output targets, gradient information for adapting the weights can be unreliable. This talk will introduce two methods for scaling up neuroevolution in order to move away from toy problems towards challenging high-dimensional continuous reinforcement learning problems: compressed network search that represents neural network weights indirectly as a set of frequency domain coefficients, which allow very large networks to be evolved by searching in low-dimensional coefficient space; and deep-convolutional pre-processors, that transform high-dimensional input to low-dimensional feature vectors that are sufficiently compact and can be used as an input for small recurrent neural network controller. The performance of the methods is demonstrated on controlling a race car to drive along a track using solely a high-dimensional visual input.