The mechanism of additive composition | SpringerLink

The mechanism of additive composition | SpringerLink

Additive composition (Foltz et al. in Discourse Process 15:285–307, 1998; Landauer and Dumais in Psychol Rev 104(2):211, 1997; Mitchell and Lapata in Cognit Sci 34(8):1388–1429, 2010) is a widely used method for computing meanings of phrases, which takes the average of vector representations of the constituent words. In this article, we prove an upper bound for the bias of additive composition, which is the first theoretical analysis on compositional frameworks from a machine learning point of view. The bound is written in terms of collocation strength; we prove that the more exclusively two successive words tend to occur together, the more accurate one can guarantee their additive composition as an approximation to the natural phrase vector. Our proof relies on properties of natural language data that are empirically verified, and can be theoretically derived from an assumption that the data is generated from a Hierarchical Pitman–Yor Process. The theory endorses additive composition

Date: 2020/05/23 23:21

Related Entries

Read more [DL輪読会]YOLOv4: Optimal Speed and Accuracy of Object Detection
1 users, 1 mentions 2020/05/15 11:21
Read more What is the TensorFloat-32 Precision Format? | NVIDIA Blog ...
0 users, 17 mentions 2020/05/14 13:38
Read more A highly efficient, real-time text to speech system deployed on CPUs
0 users, 16 mentions 2020/05/15 17:21
Read more Siamese and Dual BERT for Multi Text Classification
0 users, 3 mentions 2020/05/15 06:44
Read more WT5?! Text-to-TextモデルでNLPタスクの予測理由を説明する手法! | AI-SCHOLAR.TECH
1 users, 0 mentions 2020/05/22 11:55