[2002.06224] Top-K Training of GANs: Improving Generators by Making Critics Less Criticalcontact arXivarXiv Twitter

We introduce a simple (one line of code) modification to the Generative Adversarial Network (GAN) training algorithm that materially improves results with no increase in computational cost: When updating the generator parameters, we simply zero out the gradient contributions from the elements of the batch that the critic scores as `least realistic'. Through experiments on many different GAN variants, we show that this `top-k update' procedure is a generally applicable improvement. In order to understand the nature of the improvement, we conduct extensive analysis on a simple mixture-of-Gaussians dataset and discover several interesting phenomena. Among these is that, when gradient updates are computed using the worst-scoring batch elements, samples can actually be pushed further away from the their nearest mode.

5 mentions: @stfrolov@_sam_sinha_@_sam_sinha_
Keywords: gan
Date: 2020/02/23 03:50

Referring Tweets

@stfrolov Updating the generator only with the top-k realistic samples surprisingly improves your GAN. Updating on bad samples pushes further away from your target distribution. Cool work by @_sam_sinha_ t.co/R9Kc96ZFeg t.co/ha9QGuM8F2

Related Entries

Read more [2002.02385] Product Kanerva Machines: Factorized Bayesian Memorycontact arXivarXiv Twitter
0 users, 1 mentions 2020/02/23 06:51
Read more [2002.09402] Accessing Higher-level Representations in Sequential Transformers with Feedback Memoryc...
0 users, 2 mentions 2020/02/24 23:21
Read more [2002.12915] The Implicit and Explicit Regularization Effects of Dropoutcontact arXivarXiv Twitter
0 users, 2 mentions 2020/03/03 00:52
Read more [2006.11834] AdvAug: Robust Adversarial Augmentation for Neural Machine Translationopen searchopen n...
0 users, 1 mentions 2020/06/23 23:21
Read more [2006.14090] Neural Architecture Design for GPU-Efficient Networksopen searchopen navigation menucon...
0 users, 1 mentions 2020/06/28 11:21