[2206.14858] Solving Quantitative Reasoning Problems with Language Models

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and engineering problems at the college level. To help close this gap, we introduce Minerva, a large language model pretrained on general natural language data and further trained on technical content. The model achieves state-of-the-art performance on technical benchmarks without the use of external tools. We also evaluate our model on over two hundred undergraduate-level problems in physics, biology, chemistry, economics, and other sciences that require quantitative reasoning, and find that the model can correctly answer nearly a third of them.

22 mentions: @ethansdyer@dmdohan@Hiro_Tsukamoto@RishiBommasani@dandre@ethansdyer@gwr3n@tokumini_ss
Date:

Referring Tweets

@ethansdyer
@ethansdyer 3/ Find out more about Minerva in the blog post (t.co/UI7zV0IXlS), paper (t.co/RBS70Y20Ww) or explore more minerva samples (t.co/zMcW595QpD)! t.co/aKp6prj3OF
@dmdohan
@dmdohan Teaching Minerva🦉 math & science has been a ton of fun. What else were we supposed to do after realizing all the LaTeX on arXiv is available? Check out the sample explorer: t.co/pFiMr337S8 paper: t.co/EHe3SSbxFB t.co/1BHXUy1Bea
@Hiro_Tsukamoto
@Hiro_Tsukamoto Googleが数学の証明問題を解いてくれるAI発表してる、SFの世界じゃんわくわく t.co/YEPdfepOan
@RishiBommasani
@RishiBommasani @GaryMarcus @tdietterich @erikbryn @percyliang @ilyasut @fchollet @JeffDean @DigEconLab Whether in hindsight ladders, rockets, or something else will be most apt (imo we could use more imaginative analogies), currently FMs have taken significant strides in various arenas of mathematics: t.co/7telNEVXsw t.co/Y7i2tO6IxP t.co/6yobyu1OTZ
@dandre
@dandre Find out more: about Minerva in the blog post (t.co/GLPXDHEISw), paper (t.co/7iYxwUOfXC) or explore more minerva samples (t.co/mWCwIdzhNp)! t.co/SGv6mNfR8v
@ethansdyer
@ethansdyer @amirzait Great question! In t.co/RBS70Y20Ww we began to study memorization. We indeed looked at acc on modified questions, checked for MATH in the training data, and compared acc when removing answers similar to MATH. But this is an important direction for more follow up!
@gwr3n
@gwr3n @JFPuget t.co/Zbwbf2UoBd "Our model does not make use of external tools, and at inference time relies exclusively on autoregressive sampling." The example in which the model simply removes square roots in an equation and carries out additions show the thing does not have a clue.
@tokumini_ss
@tokumini_ss minervaとかいうやつの論文眺めてるけど、サンプルの例が普通に難しくて、こんなのできるのマジ? って感じだな。まだ手法とか実験内容のところまで行き着いてないので実はなんか出力を作り込めるような仕組みがあるのかもしれないけど t.co/0GQqzLjDZ2
@IncompetKyusyuD
@IncompetKyusyuD Solving Quantitative Reasoning Problems with Language Models t.co/k9SLHTlvu1 大学レベルの数学問題などの定量的推論を必要とするタスクを実行する言語モデルMinervaを学習させたGoogle著の論文. MinervaはPaLMをarxivと数学に特化したWebページのデータでfine-tuneしたモデル.

Related Entries

[2109.02555] GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
Read more [2109.02555] GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
0 users, 1 mentions 2021/09/20 16:37
AI DEBATE 3
Read more AI DEBATE 3
0 users, 1 mentions 2021/12/14 21:09
Sparks! | Anja Kaspersen | Good and evil: ethics and fairness in AI, who makes the rules? - YouTube
Read more Sparks! | Anja Kaspersen | Good and evil: ethics and fairness in AI, who makes the rules? - YouTube
0 users, 2 mentions 2021/12/20 16:37
PhD Position for Uncertainty-aware AI-based Predictive Model for Treatment Decision Making
Read more PhD Position for Uncertainty-aware AI-based Predictive Model for Treatment Decision Making
0 users, 2 mentions 2022/07/07 13:40
Google AI Blog: Training Generalist Agents with Multi-Game Decision Transformers
Read more Google AI Blog: Training Generalist Agents with Multi-Game Decision Transformers
0 users, 18 mentions 2022/07/28 06:09

ML-Newsについて

機械学習の技術に関する情報は流速も早いし、分野も多様でキャッチアップが大変です。Twitterで機械学習用のリストを作っても、普段は機械学習以外の話題が多く流れており、効率的に情報収集するのは困難です。

ML-NewsはSNSを情報源とした機械学習に特化したニュースサイトです。機械学習に関する論文ブログライブラリコンペティション発表資料勉強会などの最新の情報を効率的に収集できます。

機械学習を応用した自然言語処理、画像認識、情報検索などの分野の情報や機械学習で必要になるデータ基盤やMLOpsの話題もカバーしています。
安定したサイト運営のためにGitHub sponsorを募集しています。

お知らせ

  • 2021/12/31: デザインを刷新しました
  • 2021/04/08: 日本語Kaggleのカテゴリを新設しました