Branch Specialization

When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings.

12 mentions: @csvoss@distillpub@StartupYou@patrickmineault@F5WebStudio@AKatossky
Date: 2021/04/06 16:49

Referring Tweets

@csvoss Do neural networks have any emergent structures that are larger-scale than circuits? In our new Circuits article, we investigate the tendency of neural network branches to *specialize* – sorting closely-related features together across distinct branches.
@patrickmineault Different branches of CNNs specialize for different features; the latest deep dive into CNN receptive fields on distill
@F5WebStudio This article describes branch specialization, one of three larger “structural phenomena” we’ve been able observe in neural networks.
@AKatossky There seems to be evidence of "natural" structures forming in artificial neural networks, that people diversely call "branches" or "modules" : highly connected units very loosely connected to the rest of the network, and specialised into specific tasks:
@StartupYou Anatomy of neural networks
@distillpub Branch Specialization — A new Distill article by @csvoss, @gabeeegoooh, @nickcammarata, @mpetrov, @ludwigschubert, and @ch402. This is the eighth article in the Circuits thread.

Related Entries

Read more [1910.01526] Gated Linear Networksopen searchopen navigation menucontact arXivarXiv Twitter
1 users, 8 mentions 2020/06/15 09:52
Read more Researching and Developing an Autonomous Vehicle Lane-Following System | NVIDIA Developer Blog
0 users, 4 mentions 2021/01/20 22:24
Read more [2009.05673] Applications of Deep Neural Networks
1 users, 13 mentions 2021/01/25 12:58
Read more Google AI Blog: Stabilizing Live Speech Translation in Google Translate
0 users, 5 mentions 2021/01/26 20:21
Read more JMIR Preprints #26107: Physiological Data from a Wearable Device Identifies SARS-CoV-2 Infection and...
0 users, 46 mentions 2021/02/10 18:52