KDD 2019 | Shrinkage Estimators in Online Experiments

h2 p We develop and analyze empirical Bayes Stein-type estimators for use in the estimation of causal effects in large-scale online experiments. While online experiments are generally thought to be distinguished by their large sample size, we focus on the multiplicity of treatment groups. The typical analysis practice is to use simple differences-in-means (perhaps with covariate adjustment) as if all treatment arms were independent. In this work we develop consistent, small bias, shrinkage estimators for this setting. In addition to achieving lower mean squared error these estimators retain important frequentist properties such as coverage under most reasonable scenarios. Modern sequential methods of experimentation and optimization such as multi-armed bandit optimization (where treatment allocations adapt over time to prior responses) benefit from the use of our shrinkage estimators. Exploration under empirical Bayes focuses more efficiently on near-optimal arms, improving the resul...

1 mentions: @dropout009
Keywords: kdd
Date: 2019/08/10 15:47

Referring Tweets

@dropout009 KDD’19でのFacebookの論文。トリートメントがたくさんあるオンライン実験で縮小推定を行うことを提案している。 https://t.co/P1Lf1dKcDL

Related Entries

Read more [KDD 2018 tutorial] End to-end goal-oriented question answering syste…
Read more KDD17 Tutorial: Learning Representations of Large-scale Networks - Jian Tang's Homepage
Read more KDD 2018 | Accepted Papers
Read more KDD 2018 | Accepted Papers
Read more [DL輪読会]Collaborative Deep Metric Learning for Video Understanding(KDD…