[1909.03577] A New Analysis of Differential Privacy's Generalization Guaranteesopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

We give a new proof of the "transfer theorem" underlying adaptive data analysis: that any mechanism for answering adaptively chosen statistical queries that is differentially private and sample-accurate is also accurate out-of-sample. Our new proof is elementary and gives structural insights that we expect will be useful elsewhere. We show: 1) that differential privacy ensures that the expectation of any query on the posterior distribution on datasets induced by the transcript of the interaction is close to its true value on the data distribution, and 2) sample accuracy on its own ensures that any query answer produced by the mechanism is close to its posterior expectation with high probability. This second claim follows from a thought experiment in which we imagine that the dataset is resampled from the posterior distribution after the mechanism has committed to its answers. The transfer theorem then follows by summing these two bounds, and in particular, avoids the "monitor argument"

2 mentions: @boazbaraktcs
Date: 2020/11/18 21:51

Referring Tweets

@boazbaraktcs 2/6 t.co/AxXS5cPO4L (ITCS 20) gives "transfer theorem" for translating privacy guarantees to generalization performance. Though such results were known (respecting privacy ⇒ can't overfit) previous proofs often "hairy". This gives "book proof" with much better bounds.

Related Entries

Read more Detectron2: A PyTorch-based modular object detection library
0 users, 16 mentions 2019/10/11 00:49
Read more Results of the first fastMRI image reconstruction challenge
0 users, 7 mentions 2019/12/01 15:51
Read more Introducing Opacus: A high-speed library for training PyTorch models with differential privacy
0 users, 18 mentions 2020/08/31 17:22
Read more Introducing Opacus: A high-speed library for training PyTorch models with differential privacy
0 users, 24 mentions 2020/09/05 05:22
Read more Harvard ML Theory
0 users, 2 mentions 2020/10/20 17:22