PEGASUS Explained!

This video explains the PEGASUS model and new pre-training objective from researchers at Google AI. PEGASUS shows that there's no free lunch for downstream NLP tasks with respect to the pretraining task. Using this new self-supervised learning task, gap sentences generation, they dramatically surpass the baseline performance, especially in the low resource setting. Thanks for watching! Please Subscribe! Links: PEGASUS (arxiv): https://ift.tt/3hhCFGM PEGASUS (blog post): https://ift.tt/2MIkXyf Follow Anna Rogers on Twitter!: https://twitter.com/annargrs/status/1270058442523369473 What is ROUGE: https://ift.tt/3cRPu7x The Illustrated Transformer: https://ift.tt/2NLJXmf GPT-3: https://ift.tt/3et6QZt