GAN BERT: Generative Adversarial Learning for Robust Text Classification (Paper Explained) #GANBERT

GAN is a powerful way to generate data more pseudo data to train models. This paper shows how to integrate semi-supervised GAN with the most popular NLP pre-trained model, BERT.
Connect
Linkedin https://www.linkedin.com/in/xue-yong-fu-955723a6/
Twitter https://twitter.com/home
Email edwindeeplearning@gmail.com
0:00 - Intro
2:19 - Semi-supervised GANs
4:00 - Discriminator Loss Function
6:24 - Generator Loss Function
9:06 - GAN-BERT
11:59 - Unlabeled Real Examples
13:36 - Experiments
17:32 - Takeaways

GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
Paper: https://www.aclweb.org/anthology/2020.acl-main.191/

Abstract
Recent Transformer-based architectures, e.g., BERT, provide impressive results in many Nat- ural Language Processing tasks. However, most of the adopted benchmarks are made of (sometimes hundreds of) thousands of exam- ples. In many real scenarios, obtaining high- quality annotated data is expensive and time- consuming; in contrast, unlabeled examples characterizing the target task can be, in gen- eral, easily collected. One promising method to enable semi-supervised learning has been proposed in image processing, based on Semi- Supervised Generative Adversarial Networks. In this paper, we propose GAN-BERT that ex- tends the fine-tuning of BERT-like architec- tures with unlabeled data in a generative adver- sarial setting. Experimental results show that the requirement for annotated examples can be drastically reduced (up to only 50-100 an- notated examples), still obtaining good perfor- mances in several sentence classification tasks.

Code: https://github.com/crux82/ganbert

Видео GAN BERT: Generative Adversarial Learning for Robust Text Classification (Paper Explained) #GANBERT канала Deep Learning Explainer