Ladder variational autoencoders

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

  • Casper Kaae Sønderby
  • Tapani Raiko
  • Lars Maaløe
  • Søren Kaae Sønderby
  • Winther, Ole

Variational autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch-normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.

OriginalsprogEngelsk
TitelAdvances in Neural Information Processing Systems 29 (NIPS 2016)
RedaktørerD. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, R. Garnett
Antal sider9
ForlagCurran Associates, Inc.
Publikationsdato2016
Sider3745-3753
StatusUdgivet - 2016
Begivenhed30th Annual Conference on Neural Information Processing Systems - Barcelona, Spanien
Varighed: 5 dec. 201610 dec. 2016
Konferencens nummer: 30

Konference

Konference30th Annual Conference on Neural Information Processing Systems
Nummer30
LandSpanien
ByBarcelona
Periode05/12/201610/12/2016
NavnAdvances in Neural Information Processing Systems
Vol/bind29
ISSN1049-5258

ID: 179322079