[2010.13117] Hyperparameter Transfer Across Developer Adjustmentsopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

After developer adjustments to a machine learning (ML) algorithm, how can the results of an old hyperparameter optimization (HPO) automatically be used to speedup a new HPO? This question poses a challenging problem, as developer adjustments can change which hyperparameter settings perform well, or even the hyperparameter search space itself. While many approaches exist that leverage knowledge obtained on previous tasks, so far, knowledge from previous development steps remains entirely untapped. In this work, we remedy this situation and propose a new research framework: hyperparameter transfer across adjustments (HT-AA). To lay a solid foundation for this research framework, we provide four simple HT-AA baseline algorithms and eight benchmarks changing various aspects of ML algorithms, their hyperparameter search spaces, and the neural architectures used. The best baseline, on average and depending on the budgets for the old and new HPO, reaches a given performance 1.2--2.6x faster t

Date:

Related Entries

GitHub - deepmind/neural-processes: This repository contains notebook implementations of the followi...
Read more GitHub - deepmind/neural-processes: This repository contains notebook implementations of the followi...
0 users, 1 mentions 2020/02/20 14:20
[1912.02805] KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objectsopen sea...
Read more [1912.02805] KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objectsopen sea...
0 users, 1 mentions 2020/09/05 05:22
[2012.03854] Forecasting: theory and practice
Read more [2012.03854] Forecasting: theory and practice
0 users, 1 mentions 2020/12/30 08:21
Forecasting: principles and practice (UWA) | Rob J Hyndman
Read more Forecasting: principles and practice (UWA) | Rob J Hyndman
0 users, 1 mentions 2020/12/30 08:21
Test TensorFlow Lite for microcontrollers on GR-ROSE - YouTube
Read more Test TensorFlow Lite for microcontrollers on GR-ROSE - YouTube
0 users, 1 mentions 2021/01/05 12:58
[1801.06862] Testing the Number of Regimes in Markov Regime Switching Models
Read more [1801.06862] Testing the Number of Regimes in Markov Regime Switching Models
0 users, 1 mentions 2021/02/14 14:21

ML-Newsについて

機械学習の技術に関する情報は流速も早いし、分野も多様でキャッチアップが大変です。Twitterで機械学習用のリストを作っても、普段は機械学習以外の話題が多く流れており、効率的に情報収集するのは困難です。

ML-NewsはSNSを情報源とした機械学習に特化したニュースサイトです。機械学習に関する論文ブログライブラリコンペティション発表資料勉強会などの最新の情報を効率的に収集できます。

機械学習を応用した自然言語処理、画像認識、情報検索などの分野の情報や機械学習で必要になるデータ基盤やMLOpsの話題もカバーしています。
安定したサイト運営のためにGitHub sponsorを募集しています。

お知らせ

  • 2021/12/31: デザインを刷新しました
  • 2021/04/08: 日本語Kaggleのカテゴリを新設しました