[DSBA] CS224n 2021 Study | #10 Transformers and Pretraining

발표자: 석박사통합과정 김재희

lecture 10 : Transformers and Pretraining

---
DSBA CS224n winter 2021 Study

스터디 정보: https://github.com/yookyungkho/DSBA_CS224N_2021

CS224n 강의 영상: https://youtu.be/j9AcEI98C0o

Reference
- CS224N Lecture 10 강의 : https://youtu.be/j9AcEI98C0o
- CS224N Lecture 10 슬라이드 : https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1214/
- GPT-2 figure : https://jalammar.github.io/illustrated-gpt2/
- Wordpiece : https://ratsgo.github.io/nlpbook/docs/preprocess/bpe/
- Other review :
DSBA T5 : http://dsba.korea.ac.kr/seminar/?mod=document&uid=247
투빅스 BERT : https://velog.io/@tobigs-nlp/BERT-Pre-training-of-Deep-Bidirectional-Transformers-for-Language-Understanding
투빅스 GPT-1 : https://velog.io/@tobigs-nlp/Improving-Language-Understandingby-Generative-Pre-Training-GPT-1

Видео [DSBA] CS224n 2021 Study | #10 Transformers and Pretraining канала 고려대학교 산업경영공학부 DSBA 연구실