Leaking training data from GPT-2. How is this possible?
Ms. Coffee Bean explains how a huge collaboration of researchers managed to extract training data from large language models like GPT-2. Why is this even possible and what does this mean for even larger models like GPT-3?
Discussed paper:
" Extracting Training Data from Large Language Models" explained. Paper by Carlini et al. 2020.
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
📺 GPT-3 – what you need to know: https://youtu.be/5fqxPOaaqi0
Outline:
* 00:00 Large Language Models
* 01:55 GPT-2
* 02:33 Why is it possible?
* 03:38 k-eidetic memorization
* 04:21 How does the attack work?
* 05:49 How bad is it?
* 08:12 What to do?
📄 Paper explained: Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., ... & Oprea, A. (2020). Extracting Training Data from Large Language Models. arXiv preprint arXiv:2012.07805. https://arxiv.org/pdf/2012.07805.pdf
-------------------------------------------------------------
🔗 Links:
YouTube: https://www.youtube.com/AICoffeeBreak
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
#AICoffeeBreak #MsCoffeeBean #GPT #MachineLearning #AI #research
Видео Leaking training data from GPT-2. How is this possible? канала AI Coffee Break with Letitia
Discussed paper:
" Extracting Training Data from Large Language Models" explained. Paper by Carlini et al. 2020.
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
📺 GPT-3 – what you need to know: https://youtu.be/5fqxPOaaqi0
Outline:
* 00:00 Large Language Models
* 01:55 GPT-2
* 02:33 Why is it possible?
* 03:38 k-eidetic memorization
* 04:21 How does the attack work?
* 05:49 How bad is it?
* 08:12 What to do?
📄 Paper explained: Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., ... & Oprea, A. (2020). Extracting Training Data from Large Language Models. arXiv preprint arXiv:2012.07805. https://arxiv.org/pdf/2012.07805.pdf
-------------------------------------------------------------
🔗 Links:
YouTube: https://www.youtube.com/AICoffeeBreak
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
#AICoffeeBreak #MsCoffeeBean #GPT #MachineLearning #AI #research
Видео Leaking training data from GPT-2. How is this possible? канала AI Coffee Break with Letitia
Показать
Комментарии отсутствуют
Информация о видео
10 января 2021 г. 19:03:12
00:09:34
Другие видео канала
Deep Learning for Symbolic Mathematics!? | Paper EXPLAINED[Quiz] Eigenfaces, Domain adaptation, Causality, Manifold Hypothesis, Denoising Autoencoder[RANT] Adversarial attack on OpenAI’s CLIP? Are we the fools or the foolers?Our paper at CVPR 2020 - MUL Workshop and ACL 2020 - ALVR WorkshopData BAD | What Will it Take to Fix Benchmarking for NLU?Preparing for Virtual Conferences – 7 Tips for recording a good conference talkCan a neural network tell if an image is mirrored? – Visual ChiralityAI Coffee Break - Channel Trailer[Quiz] Interpretable ML, VQ-VAE w/o Quantization / infinite codebook, Pearson’s, PointCloudsWhat is the model identifiability problem? | Explained in 60 seconds! | ❓ #AICoffeeBreakQuiz #ShortsAdding vs. concatenating positional embeddings & Learned positional encodingsTransformer in Transformer: Paper explained and visualized | TNTPre-training of BERT-based Transformer architectures explained – language and vision!Training learned optimizers: VeLO paper EXPLAINEDAdversarial Machine Learning explained! | With examples.Transformer LLMs are Turing Complete after all !?Are Pre-trained Convolutions Better than Pre-trained Transformers? – Paper ExplainedFNet: Mixing Tokens with Fourier Transforms – Paper ExplainedAI understanding language!? A roadmap to natural language understanding.Transformers explained | The architecture behind LLMsWhy ChatGPT fails | Language Model Limitations EXPLAINED