Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP | OpenReview

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP Sep 25, 2019 Blind Submission readers: everyone Show Bibtex Keywords: lottery tickets, nlp, transformer, rl, reinforcement learning TL;DR: We find that the lottery ticket phenomenon is present in both NLP and RL, and find that it can be used to train compressed Transformers to high performance Abstract: The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) a

Keywords: nlp
Date: 2020/05/03 00:51

Related Entries

Read more Are Transformers universal approximators of sequence-to-sequence functions? | OpenReview
0 users, 1 mentions 2020/05/03 00:51
Read more Finite Depth and Width Corrections to the Neural Tangent Kernel | OpenReview
0 users, 1 mentions 2020/05/03 00:51
Read more GitHub - reviewingNLP/ACL2020T3material: Contains all teaching material used in ACL 2020 Tutorial "R...
0 users, 1 mentions 2020/07/08 17:21
Read more [2006.13707] Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functionso...
0 users, 1 mentions 2020/07/19 00:52
Read more [2002.08563] The continuous categorical: a novel simplex-valued exponential familyopen searchopen na...
0 users, 1 mentions 2020/07/19 02:21