Linear response methods for accurate covariance estimates from mean field variational bayes

Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, a well known failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable c...

Full description

Bibliographic Details
Main Authors: Giordano, Ryan (Author), Jordan, Michael (Author), Broderick, Tamara A (Contributor)
Other Authors: Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science (Contributor)
Format: Article
Language:English
Published: Neural Information Processing Systems Foundation, 2017-07-20T14:53:40Z.
Subjects:
Online Access:Get fulltext
LEADER 01646 am a22001933u 4500
001 110786
042 |a dc 
100 1 0 |a Giordano, Ryan  |e author 
100 1 0 |a Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science  |e contributor 
100 1 0 |a Broderick, Tamara A  |e contributor 
700 1 0 |a Jordan, Michael  |e author 
700 1 0 |a Broderick, Tamara A  |e author 
245 0 0 |a Linear response methods for accurate covariance estimates from mean field variational bayes 
260 |b Neural Information Processing Systems Foundation,   |c 2017-07-20T14:53:40Z. 
856 |z Get fulltext  |u http://hdl.handle.net/1721.1/110786 
520 |a Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, a well known failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable covariance. We generalize linear response methods from statistical physics to deliver accurate uncertainty estimates for model variables---both for individual variables and coherently across variables. We call our method linear response variational Bayes (LRVB). When the MFVB posterior approximation is in the exponential family, LRVB has a simple, analytic form, even for non-conjugate models. Indeed, we make no assumptions about the form of the true posterior. We demonstrate the accuracy and scalability of our method on a range of models for both simulated and real data. 
546 |a en_US 
655 7 |a Article 
773 |t Advances in Neural Information Processing Systems 28 (NIPS 2015)