Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Bond-Taylor, Sam; Leach, Adam; Long, Yang; Willcocks, Chris G.

Abstract

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.

Citation

Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2021). Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 7327-7347. https://doi.org/10.1109/tpami.2021.3116668

• Journal Article Type: Article
• Acceptance Date: Sep 22, 2021
• Online Publication Date: Sep 30, 2021
• Publication Date: 2021-11
• Deposit Date: Oct 29, 2021
• Publicly Available Date: Dec 16, 2022
• Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence
• Print ISSN: 0162-8828
• Electronic ISSN: 1939-3539
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 44
• Issue: 11
• Pages: 7327-7347
• DOI: https://doi.org/10.1109/tpami.2021.3116668
• Public URL: https://durham-repository.worktribe.com/output/1232078