Black Box Variational Inference and Deep Exponential Families
Profressor David Blei, Columbia University
Bayesian statistics and expressive probabilistic modeling have become
key tools for the modern statistician. These tools let us express
complex assumptions about the hidden elements that underlie our data,
and they have been successfully applied in numerous fields. The
central computational problem in Bayesian statistics is posterior
inference, the problem of approximating the conditional distribution
of the hidden variables given the observations. Approximate posterior
inference algorithms have revolutionized the field, revealing its
potential as a usable and general-purpose language for data analysis.
In this talk, I will discuss two related innovations in modeling and
inference: deep exponential families and black box variational
inference. Deep exponential families (DEFs) adapt the main ideas
behind deep learning to expressive probabilistic models. DEFs provide
principled probabilistic models that can uncover layers of
representations of high-dimensional data. I will show how to use DEFs
to analyze text, recommendation data, and electronic health records.
I will then discuss the key algorithm that enables DEFs: Black box
variational inference (BBVI). BBVI is a generic and scalable algorithm
for approximating the posterior. BBVI easily applies to many models,
with little model-specific derivation and few restrictions on their
David Blei is a Professor of Statistics and Computer Science at Columbia University,
and a member of the Columbia Data Science Institute. His research is in statistical machine learning, involving
probabilistic topic models, Bayesian nonparametric methods, and approximate posterior inference with massive data.
He works on a variety of applications, such as text, images, music, social networks, user behavior, and scientific
data. David has received several awards for his research, including a Sloan Fellowship (2010), Office of Naval
Research Young Investigator Award (2011), Presidential Early Career Award for Scientists and Engineers (2011),
Blavatnik Faculty Award (2013), ACM-Infosys Foundation Award (2013), and a Guggenheim fellowship.
He is a fellow of the ACM and the IMS.