[2006.04439] Liquid Time-constant Networksopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., \emph{liquid}) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics, and compute their expressive power by the \emph{trajectory length} measure in a latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of L

5 mentions: @ankurhandos@iamrws@Eun_Chang_Choi
Date: 2020/10/14 17:22

Referring Tweets

@ankurhandos “Nature shows us that there is still a lot of room for improvement,” says Prof. Daniela Rus, director of MIT CSAIL. “Therefore, our goal was to massively reduce complexity and develop a new kind neural network architecture.” Liquid Time-Constant Networks t.co/xV3mdOCNaY t.co/qAGRpcFRCn
@Eun_Chang_Choi MIT Computer Science & Artificial Intelligence Lab announced New deep learning models require fewer neurons called "Liquid Time-constant Networks" t.co/eV528FoZd4

Related Entries

Read more +Alex Krizhevsky's talk at the ImageNet ECCV workshop yesterday made a bit of...
0 users, 4 mentions 2019/03/27 21:49
Read more [1906.06558] Mask Based Unsupervised Content Transferopen searchopen navigation menucontact arXivarX...
0 users, 1 mentions 2020/04/26 18:52
Read more Image synthesis at CVPR 2020 | Generative blog
0 users, 2 mentions 2020/06/29 00:00
Read more Multitask Learning in Keras with an augmented Fashion MNIST. · GitHub
0 users, 1 mentions 2020/09/04 20:22
Read more [1811.03268] Ordinal Regression using Noisy Pairwise Comparisons for Body Mass Index Range Estimatio...
0 users, 1 mentions 2020/10/06 20:22