[1906.02361] Explain Yourself! Leveraging Language Models for Commonsense Reasoning

Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world-knowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the state-of-the-art by 10% on the challenging CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations including transfer to out-of-domain tasks. Empirical results indicate that we can effectively leverage language models for commonsense reasoning.

2 mentions: @Miles_Brundage@mihail_eric
Date: 2019/06/07 08:18

Referring Tweets

@Miles_Brundage "Explain Yourself! Leveraging Language Models for Commonsense Reasoning," Rajani et al.: https://t.co/teRY3MXfBE
@mihail_eric Very cool work from @SFResearch @RichardSocher on imbuing language models with commonsense reasoning: https://t.co/VI286WeLBZ Not only a clever system but big new state-of-the-art on the CommonsenseQA task!

Related Entries

Read more 子どもの言語獲得のモデル化とNN Language ModelsNN
Read more Generalized Language Models
Read more [1902.10547] An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models
Read more CMU Neural Nets for NLP 2019 (5): Recurrent Networks for Sentence or Language Modeling - YouTube
Read more [1903.09722] Pre-trained Language Model Representations for Language Generation