Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps | OpenReview

Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps Sep 25, 2019 ICLR 2020 Conference Blind Submission readers: everyone Show Bibtex TL;DR: We propose a differentiable family of "kaleidoscope matrices," prove that all structured matrices can be represented in this form, and use them to replace hand-crafted linear maps in deep learning models. Abstract: Modern neural network architectures use structured linear transformations, such as low-rank matrices, spa

2 mentions: @hillbig@hillbig
Date: 2019/11/08 00:52

Referring Tweets

@hillbig 構造化行列(FFT, アダマール等)は高速行列積を実現できるが人手で設計していた。万華鏡行列(Kaleidoscope行列)は既知の殆どの構造化行列、疎行列をほぼ最適な空間/時間計算量で表現でき、微分可能であり実際高速である。離散的な操作(置換)も高速に学習できる t.co/PkjltMXJDq
@hillbig Structured linear transformation (e.g., FFT, Hadamard) is often designed for each task manually. Kaleidoscope matrix (K-matrix) can represent any structured matrix with near-optimal space and time. Efficient in practice. looks promising with hardware impl. t.co/PkjltMXJDq

Related Entries

Read more [1801.04295] Generalization Error Bounds for Noisy, Iterative Algorithms
0 users, 2 mentions 2019/01/21 00:46
Read more [1901.08584] Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Laye...
0 users, 3 mentions 2019/02/05 00:47
Read more [2002.03629] Nonlinear Equation Solving: A Faster Alternative to Feedforward Computationcontact arXi...
0 users, 5 mentions 2020/02/14 18:52
Read more [1912.07372] Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Su...
0 users, 2 mentions 2020/05/07 00:52
Read more [2005.03101] Scale-Equalizing Pyramid Convolution for Object Detectionopen searchopen navigation men...
0 users, 2 mentions 2020/05/18 00:52