[2005.08100] Conformer: Convolution-augmented Transformer for Speech Recognitionopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1%/4.3% without using a language model and 1.9%/3.9% with an external language model on test/testother. We also observe competitive performance of 2.7%/6.3% with a small model o

1 mentions: @Maxwell_110
Keywords: Transformer
Date:

Referring Tweets

@Maxwell_110
@Maxwell_110 Conformer 👇 Gulati, Convolution-augmented Transformer for Speech Recognition, 2020 t.co/vu2r20xtXo t.co/tY1CjcN0iz t.co/2MnqmqgC0K

Related Entries

Dstl Satellite Imagery Feature Detection | Kaggle
Read more Dstl Satellite Imagery Feature Detection | Kaggle
2 users, 1 mentions 2020/05/31 09:51
Cornell Birdcall Identification | Kaggle
Read more Cornell Birdcall Identification | Kaggle
0 users, 2 mentions 2020/09/16 03:51
Segmentation fault in DataLoader worker in PyTorch 1.8.0 if set_num_threads is called beforehand · I...
Read more Segmentation fault in DataLoader worker in PyTorch 1.8.0 if set_num_threads is called beforehand · I...
0 users, 1 mentions 2021/07/31 15:09
GitHub - ksnxr/GWC_solution: 1st place solution of the Global Wheat Challenge 2021, by randomTeamNam...
Read more GitHub - ksnxr/GWC_solution: 1st place solution of the Global Wheat Challenge 2021, by randomTeamNam...
0 users, 1 mentions 2021/08/04 10:40
GitHub - Wenxuan-1119/TransBTS: This repo provides the official code for TransBTS: Multimodal Brain ...
Read more GitHub - Wenxuan-1119/TransBTS: This repo provides the official code for TransBTS: Multimodal Brain ...
0 users, 1 mentions 2021/08/13 10:37
detectron2/MODEL_ZOO.md at main · facebookresearch/detectron2 · GitHub
Read more detectron2/MODEL_ZOO.md at main · facebookresearch/detectron2 · GitHub
0 users, 1 mentions 2021/11/09 22:38

ML-Newsについて

機械学習の技術に関する情報は流速も早いし、分野も多様でキャッチアップが大変です。Twitterで機械学習用のリストを作っても、普段は機械学習以外の話題が多く流れており、効率的に情報収集するのは困難です。

ML-NewsはSNSを情報源とした機械学習に特化したニュースサイトです。機械学習に関する論文、ブログ、ライブラリ、コンペティション、発表資料、勉強会などの最新の情報を効率的に収集できます。

機械学習を応用した自然言語処理、画像認識、情報検索などの分野の情報や機械学習で必要になるデータ基盤やMLOpsの話題もカバーしています。
安定したサイト運営のためにGitHub sponsorを募集しています。

お知らせ

  • 2021/12/31: デザインを刷新しました
  • 2021/04/08: 日本語とKaggleのカテゴリを新設しました