Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey

🤖 Plain-English Summary

Large, pre-trained language models (PLMs) such as BERT and GPT have drastically changed the Natural Language Processing (NLP) field. It surveys work applying the pre-training then fine-tuning, prompting, and text generation approaches.

🔑 Key Findings

• For numerous NLP tasks, approaches leveraging PLMs have achieved advanced performance.
• The key idea is to learn a generic, latent representation of language from a generic task once, then share it across disparate NLP tasks.
• Language modeling serves as the generic task, one with abundant self-supervised text available for extensive training.

🔗 Related Resources