[2008.06996] Large Associative Memory Problem in Neurobiology and Machine Learningopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological, since it seemingly requires the existence of many-body synaptic junctions between the neurons. We show that these models are effective descriptions of a more microscopic (written in terms of biological degrees of freedom) theory that has additional (hidden) neurons and only requires two-body interactions between them. For this reason our proposed microscopic theory is a valid model of large associative memory with a degree of biological plausibility. The dynamics of our network and its reduced dimensional equivalent both minimize energy (Lyapunov) functions. When certain dynamical variables (hidden neurons) are integrated out from our microscopic theory, one can recover many of the models that were previously discussed in the literature, e.g. the mode

7 mentions: @HopfieldJohn@DimaKrotov@hardmaru@AlifePapers
Date: 2020/08/19 09:52

Referring Tweets

@HopfieldJohn Dense Associative Memories, aka modern Hopfield networks, have a huge memory storage capacity. But are they biologically realistic? In our new paper with @DimaKrotov we argue that they can be written in terms of biological variables. t.co/w58EAQ54xm t.co/dGp4npMRo2
@DimaKrotov New microscopic theory of Dense Associative Memory aka modern Hopfield network can be reduced to the model proposed in “Hopfield Networks is All You Need’’ paper, which is equivalent to self-attention mechanism of Transformers. Work with @HopfieldJohn t.co/jYAxIjvgKf t.co/212qRx9JrQ
@hardmaru Large Associative Memory Problem in Neurobiology and Machine Learning Biological plausible explanation of “Hopfield Networks is All You Need” by Krotov and Hopfield Paper: t.co/8k8as68S4E Discussion: t.co/DoEgQppEA3 t.co/tDWf4QgYpE
@AlifePapers Large Associative Memory Problem in Neurobiology and Machine Learning "our proposed microscopic theory is a valid model of large associative memory with a degree of biological plausibility" t.co/xoYCXefdVe t.co/ro6U2Xgi37

Related Entries

Read more [1906.05433] Tackling Climate Change with Machine Learning
0 users, 73 mentions 2019/06/14 02:18
Read more Machine learning and Deep Learning Day Boston 2020 | AI & ML Events
0 users, 3 mentions 2020/01/16 14:21
Read more Machine learning and Deep Learning Day Boston 2020 | AI & ML Events
0 users, 2 mentions 2020/01/23 18:52
Read more GitHub - skrish13/ml-contests-conf: ML and DL related contests, competitions and conference challeng...
0 users, 4 mentions 2020/01/28 20:20
Read more An Opinionated Guide to ML Research
0 users, 9 mentions 2020/01/31 09:50

ML-Newsについて

ML-Newsは機械学習に関するニュースサイトです。機械学習に関する論文ブログライブラリコンペティション発表資料勉強会などの最新の情報にアクセスできます。

機械学習を応用した自然言語処理、画像認識、情報検索などの分野の情報や機械学習で必要になるデータ基盤やMLOpsの話題もカバーしています。
安定したサイト運営のためにGitHub sponsorを募集しています。

お知らせ