Gradient Origin Networks

Bond-Taylor, Sam; Willcocks, Chris G.

Abstract

This paper proposes a new type of generative model that is able to quickly learn a latent representation without an encoder. This is achieved using empirical Bayes to calculate the expectation of the posterior, which is implemented by initialising a latent vector with zeros, then using the gradient of the log-likelihood of the data with respect to this zero vector as new latent points. The approach has similar characteristics to autoencoders, but with a simpler architecture, and is demonstrated in a variational autoencoder equivalent that permits sampling. This also allows implicit representation networks to learn a space of implicit functions without requiring a hypernetwork, retaining their representation advantages across datasets. The experiments show that the proposed method converges faster, with significantly lower reconstruction error than autoencoders, while requiring half the parameters.

Citation

Bond-Taylor, S., & Willcocks, C. G. (2021, May). Gradient Origin Networks. Presented at International Conference on Learning Representations, Vienna / Virtual

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Learning Representations
• Start Date: May 3, 2021
• End Date: May 7, 2021
• Acceptance Date: Jan 12, 2021
• Publication Date: 2021
• Deposit Date: Nov 27, 2020
• Publicly Available Date: Oct 28, 2021
• Public URL: https://durham-repository.worktribe.com/output/1140006
• Publisher URL: https://iclr.cc/