Language, trees, and geometry in neural networks

Part I of a series of expository notes accompanying this paper, by Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, and Martin Wattenberg. These notes are designed as an expository walk through some of the main results. Please see the paper for full references and details. Language is made of discrete structures, yet neural networks operate on continuous data: vectors in high-dimensional space. A successful language-processing network must translate this symbolic infor

30 mentions: @wattenberg@viegasf@earnmyturns@wattenberg@jakubzavrel@mat_kelcey@FabienCampagne@wzuidema
Date: 2019/06/08 14:17

Referring Tweets

@wattenberg The blog at explores the shape of these representations, with math and visualization. A mysterious “squared distance” effect they found turns out to be surprisingly natural!
@earnmyturns Very cool exploration of the geometry of language embeddings, with some fun math I did not know.
@wattenberg How does a neural net represent language? See the visualizations and geometry in this PAIR team paper and blog post
@viegasf Analyzing and visualizing syntax trees in the high-dimensional spaces of neural nets. Check out the new PAIR paper on BERT geometry And the blog post on “Language, trees, and geometry in neural networks”
@wzuidema Jawahar, Sagot, Seddah What does BERT learn about the structure of language? ACL2019 Coenen, Reif, Yuan, Kim, Pearce, Viégas, Wattenberg Visualizing and Measuring the Geometry of BERT Blog:
@jakubzavrel How sentences and their trees can be represented in the geometry of BERT neural network spaces ... we dreamed about this in the 90’s.
@mat_kelcey "Language, trees, and geometry in neural networks" this post is awesome. connects a number of things in ways I'd never thought of. can't wait to properly digest the paper :D
@zhongxuank This is such an interesting blog post - saving it here to read in the future.

Related Entries

Read more Neural networks for Graph Data NeurIPS2018読み会@PFN
25 users, 9 mentions 2019/01/26 09:46
Read more GitHub - thunlp/GNNPapers: Must-read papers on graph neural networks (GNN)
1 users, 0 mentions 2019/08/18 08:16
Read more Deep Forest :Deep Neural Networkの代替へ向けて - QiitaQiita
0 users, 0 mentions 2018/04/25 17:22
Read more GitHub - nnzhan/Awesome-Graph-Neural-Networks: Paper Lists for Graph Neural Networks
0 users, 2 mentions 2019/05/21 15:48
Read more Deep Forest: Towards An Alternative to Deep Neural Networks
2 users, 0 mentions 2019/08/19 08:16