Andrey Malinin: Predictive Uncertainty Estimation via Prior Networks (DCPGM seminar)
FIT VUT, E105 28.5.2019
room E105, start at 15:00
Predictive Uncertainty Estimation via Prior Networks
speaker: Andrey Malinin, University of Cambridge (UK)
Abstract: Estimating how uncertain an AI system is in its predictions is important to improve the safety of such systems. Uncertainty in predictions can result from uncertainty in model parameters, data uncertainty and uncertainty due to mismatch between the test and training data distributions. Different actions might be taken depending on the source of the uncertainty so it is important to be able to distinguish between them. Recently, baseline tasks and metrics have been defined and several practical methods to estimate uncertainty developed. These methods, however, attempt to model uncertainty due to distributional mismatch either implicitly through model uncertainty or as data uncertainty. This work proposes a new framework for modeling predictive uncertainty called Prior Networks (PNs) which explicitly models distributional uncertainty. PNs do this by parameterizing a prior distribution over predictive distributions. This work focuses on uncertainty for classification and evaluates PNs on the tasks of identifying out-of-distribution (OOD) samples and detecting misclassification on the MNIST, SVHN and CIFAR-10 datasets, where they are found to outperform previous methods.
BIO: Andrey is a Ph.D. student at the Cambridge University Engineering Department, supervised by Prof. Mark Gales. His work forms part of the ALTA Project to develop automated approaches to second language learning and assessment. Andrey's primary research interest is the estimation of predictive uncertainty for Deep Learning, which is important for high risk applications, such as medicine, self driving cars, finance and high-stakes examinations. Additionally, Andrey investigates the application of Deep Learning to topic relevance assessment and grading for automatic assessment of spoken language proficiency.