Deep Learning with Differential Privacy (DP-SGD explained)

We're performing a technical deep dive into differential privacy: preventing models from memorising private data. Theory + Colab notebook using Tensorflow Privacy!

Social:
Twitter: https://twitter.com/mukulrathi_
My website (+ blog): https://mukulrathi.com/
My email newsletter: https://newsletter.mukulrathi.com

-----------------
Links:
DP-SGD paper: https://arxiv.org/pdf/1607.00133.pdf
Tensorflow Privacy Tutorials: https://github.com/tensorflow/privacy/tree/master/tutorials
Tensorflow Privacy: https://github.com/tensorflow/privacy
Pytorch Opacus: https://github.com/pytorch/opacus
Moments accountant implementation: https://github.com/marcotcr/tf-models/blob/master/differential_privacy/privacy_accountant/tf/accountant.py
GPT-2 memorises private data: https://ai.googleblog.com/2020/12/privacy-considerations-in-large.html
Netflix dataset deanonymised: https://www.wired.com/2007/12/why-anonymous-data-sometimes-isnt/
Netflix deanonymisation paper: https://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf
Strava heatmap leaks: https://www.zdnet.com/article/strava-anonymized-fitness-tracking-data-government-opsec/

------------------------------------------------------------------------------------
Timestamps:

00:00 Introduction
00:32 Overview
01:26 Why Anonymisation Isn't Enough
02:38 Intuition for Differential Privacy
03:12 Example: Predict whether Bob has Cancer
04:11 Privacy Intuition
04:51 Privacy Loss Definition
05:26 Definition of Differential Privacy
06:40 Role of Noise in DP
07:08 Privacy Amplification Theorem
07:26 Fundamental Law of Information Recovery
07:51 Composition in DP
08:19 DP-SGD
09:20 Moments Accountant
12:55 Google Colab Notebook
14:39 Limitations of DP-SGD

---------------------------------------------------------------------------

Music: Coffee Break by Pyrosion is licensed under a Creative Commons License.
https://creativecommons.org/licenses/...
https://soundcloud.com/pyrosion
Support by RFM - NCM: https://bit.ly/2xGHypM

Видео Deep Learning with Differential Privacy (DP-SGD explained) канала Mukul Rathi