BackPACK: Packing more into Backprop | OpenReview

BackPACK: Packing more into Backprop Sep 25, 2019 ICLR 2020 Conference Blind Submission readers: everyone Show Bibtex Abstract: Automatic differentiation frameworks are optimized for exactly one thing: computing the average mini-batch gradient. Yet, other quantities such as the variance of the mini-batch gradients or many approximations to the Hessian can, in theory, be computed efficiently, and at the same time as the gradient. While these quantities are of great interest to researchers a

3 mentions: @ankesh_anand@asam9891@hirotomusiker
Date: 2019/11/08 12:52

Referring Tweets

@asam9891 BackPACK: Packing more into Backprop 従来の深層学習フレームワークは一次微分しかサポートしてないため二次微分やその近似を利用する場合は頑張って実装する必要がある。辛いので勾配の分散とかKFAC とかを簡単に計算できるようなPyTorch 拡張作った話。ICLR2020 top score t.co/Gi6DWVSlY1
@hirotomusiker Official PyTorch implementation of BackPack (backprop with first and second order extensions), the perfect-scored paper on ICLR'20 openreview. link: t.co/F6xuc2qSXw t.co/1CNbpoRgk7
@ankesh_anand ICLR papers with perfect scores (all 8s, total 11 papers): 1. t.co/XOouanwJDI "FreeLB: Enhanced Adversarial Training for Language Understanding" 2. t.co/c1ORyFEhQA "BackPACK: Packing more into Backprop"

Related Entries

Read more ICLR2018参加報告第4回(強化学習と逆強化学習の他分野への応用) | DeepX AI Blog
0 users, 0 mentions 2018/08/10 09:23
Read more [ICLR2017読み会 @ DeNA] ICLR2017紹介
0 users, 0 mentions 2018/04/22 03:40
Read more ICLR2018参加報告第4回(強化学習と逆強化学習の他分野への応用) | DeepX AI Blog
0 users, 0 mentions 2018/08/10 12:00
Read more Program Synthesis Papers at ICLR 2018 – NEAR AI – Medium
0 users, 0 mentions 2018/05/06 20:28
Read more [DL輪読会]ICLR2020の分布外検知速報
0 users, 1 mentions 2019/09/27 06:49