[2002.04019] Be Like Water: Robustness to Extraneous Variables Via Adaptive Feature Normalizationopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

Extraneous variables are variables that are irrelevant for a certain task, but heavily affect the distribution of the available data. In this work, we show that the presence of such variables can degrade the performance of deep-learning models. We study three datasets where there is a strong influence of known extraneous variables: classification of upper-body movements in stroke patients, annotation of surgical activities, and recognition of corrupted images. Models trained with batch normalization learn features that are highly dependent on the extraneous variables. In batch normalization, the statistics used to normalize the features are learned from the training set and fixed at test time, which produces a mismatch in the presence of varying extraneous variables. We demonstrate that estimating the feature statistics adaptively during inference, as in instance normalization, addresses this issue, producing normalized features that are more robust to changes in the extraneous variabl

1 mentions: @kchonyc
Date: 2020/06/27 00:52

Referring Tweets

@kchonyc @zacharynado @brandondamos a related paper by @aakashkaku et al. t.co/lgixIpgAv4: "In batch normalization, the statistics ... are learned from the training set and fixed at test time .. estimating the feature statistics adaptively during inference ... normalized features that are more robust"

Related Entries

Read more NeurIPS 2018 Workshop: Machine Learning for Molecules and Materials
0 users, 1 mentions 2019/12/31 20:21
Read more [1912.05827] An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generat...
0 users, 1 mentions 2020/02/13 20:21
Read more Soft $k$-NN - HackMD
0 users, 1 mentions 2020/02/25 15:50
Read more Algorithms for Hyper-Parameter Optimization
0 users, 1 mentions 2020/04/16 18:52
Read more Emergent Translation in Multi-Agent Communication | OpenReview
0 users, 1 mentions 2020/06/06 21:51