Improving Language Model Behavior by Training on a Curated Dataset

Improving Language Model Behavior by Training on a Curated Dataset

Our latest research finds we can improve language model behavior with respect to specific behavioral values by fine-tuning on a small, curated dataset.

9 mentions: @gdb@NPCollapse@ak92501@E0M@sleepinyourhat@nealkhosla@adventurared
Date: 2021/06/10 15:00

Referring Tweets

@ak92501 Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets pdf: t.co/sV744sgNz3 blog: t.co/G4NpYu2dRg performs significantly better on all metrics compared to baseline and control models for a broad range of GPT-3 language model sizes t.co/uBAgDMPi5V
@NPCollapse Another day, another step towards this meme becoming reality. It's very interesting work, training on just <100 (!) samples greatly increases human evaluation of the models performance. t.co/WL5KMoH6fA t.co/mQXyI94dZE
@gdb We've found that it's possible to target GPT-3's behaviors to a chosen set of values, by carefully creating a small dataset of behavior that reflects those values. A step towards OpenAI users setting the values within the context of their application: t.co/CVwnwKd6UJ t.co/ly8EkTDTW2
@E0M Amazing work by @IreneSolaiman & @cbd shows how to teach our language models to behave with far less toxicity and much more aligned to the values we aspire towards. The internet is an awful place. This shows models trained on it don't have to be awful too! t.co/zxzhctnBbP
@nealkhosla t.co/z9fGbueX72 Language Models are surprisingly responsive to attempts to give them values. But the most important line in this whole thread of work is at the bottom, "Who should be consulted when designing a values-targeted dataset?"
@adventurared “We’ve found we can improve language model behavior with respect to specific behavioral values by fine-tuning on a curated dataset of <100 examples of those values. We also found that this process becomes more effective as models get larger.“ @OpenAI t.co/J4ObxjF1TO
@sleepinyourhat @DanielKhashabi @brianchristian How do you think about these results in light of the OpenAI results from yesterday? That seems like a very partial/tentative/maybe-not-robust step toward yes. t.co/bWEjbu1f0A

Related Entries

Read more [2004.07213] Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimsopen sear...
0 users, 33 mentions 2020/04/16 17:21
Read more Vid2Player
1 users, 11 mentions 2020/08/13 14:22
Read more Patch Forensics
0 users, 5 mentions 2020/08/25 14:21
Read more [2010.04301] Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsuperv...
0 users, 3 mentions 2020/10/14 00:53
Read more [2105.08399] Relative Positional Encoding for Transformers with Linear Complexity
0 users, 4 mentions 2021/05/19 15:19

ML-Newsについて

ML-Newsは機械学習に関するニュースサイトです。機械学習に関する論文ブログライブラリコンペティション発表資料勉強会などの最新の情報にアクセスできます。

機械学習を応用した自然言語処理、画像認識、情報検索などの分野の情報や機械学習で必要になるデータ基盤やMLOpsの話題もカバーしています。
安定したサイト運営のためにGitHub sponsorを募集しています。

お知らせ