[2009.05359] Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the Brainopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

Can the powerful backpropagation of error (backprop) reinforcement learning algorithm be formulated in a manner suitable for implementation in neural circuitry? The primary challenge is to ensure that any candidate formulation uses only local information, rather than relying on global (error) signals, as in orthodox backprop. Recently several algorithms for approximating backprop using only local signals, such as predictive coding and equilibrium-prop, have been proposed. However, these algorithms typically impose other requirements which challenge biological plausibility: for example, requiring complex and precise connectivity schemes (predictive coding), or multiple sequential backwards phases with information being stored across phases (equilibrium-prop). Here, we propose a novel local algorithm, Activation Relaxation (AR), which is motivated by constructing the backpropagation gradient as the equilibrium point of a dynamical system. Our algorithm converges robustly and exactly to t

4 mentions: @tak_yamm@data4sci@bgoncalves@BerenMillidge
Date: 2020/09/14 11:22

Referring Tweets

@BerenMillidge Excited to announce a new preprint: "Activation Relaxation: A Local, Dynamical Approximation to Backprop in the Brain. " paper: t.co/RsMjFc0PwW code: t.co/RaJS9fFQqs with @a_tschantz , @anilkseth, and @drclbuckley

Related Entries

Read more [1907.09987] Bayesian Inference with Generative Adversarial Network Priors
1 users, 3 mentions 2019/07/24 15:47
Read more [1908.11152] A Summarization System for Scientific Documents
0 users, 3 mentions 2019/08/30 09:47
Read more [1911.01429] The frontier of simulation-based inference
0 users, 4 mentions 2019/11/06 17:20
Read more [2002.10061] Rethinking 1D-CNN for Time Series Classification: A Stronger Baselinecontact arXivarXiv...
0 users, 3 mentions 2020/02/28 18:52
Read more [2009.05528] Modeling growth of urban firm networksopen searchopen navigation menucontact arXivsubsc...
0 users, 4 mentions 2020/09/15 08:22