Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps | OpenReview

Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps Sep 25, 2019 ICLR 2020 Conference Blind Submission readers: everyone Show Bibtex TL;DR: We propose a differentiable family of "kaleidoscope matrices," prove that all structured matrices can be represented in this form, and use them to replace hand-crafted linear maps in deep learning models. Abstract: Modern neural network architectures use structured linear transformations, such as low-rank matrices, spa

2 mentions: @hillbig@hillbig
Date: 2019/11/08 00:52

Referring Tweets

@hillbig 構造化行列(FFT, アダマール等)は高速行列積を実現できるが人手で設計していた。万華鏡行列(Kaleidoscope行列)は既知の殆どの構造化行列、疎行列をほぼ最適な空間/時間計算量で表現でき、微分可能であり実際高速である。離散的な操作(置換)も高速に学習できる t.co/PkjltMXJDq
@hillbig Structured linear transformation (e.g., FFT, Hadamard) is often designed for each task manually. Kaleidoscope matrix (K-matrix) can represent any structured matrix with near-optimal space and time. Efficient in practice. looks promising with hardware impl. t.co/PkjltMXJDq

Related Entries

Read more ICLR2018参加報告第4回(強化学習と逆強化学習の他分野への応用) | DeepX AI Blog
0 users, 0 mentions 2018/08/10 09:23
Read more [ICLR2017読み会 @ DeNA] ICLR2017紹介
0 users, 0 mentions 2018/04/22 03:40
Read more ICLR2018参加報告第4回(強化学習と逆強化学習の他分野への応用) | DeepX AI Blog
0 users, 0 mentions 2018/08/10 12:00
Read more Program Synthesis Papers at ICLR 2018 – NEAR AI – Medium
0 users, 0 mentions 2018/05/06 20:28
Read more [DL輪読会]ICLR2020の分布外検知速報
0 users, 1 mentions 2019/09/27 06:49