Emergent Abilities of Large Language Models

🤖 Plain-English Summary

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models.

🔑 Key Findings

• This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models.
• We consider an ability to be emergent if it is not present in smaller models but is present in larger models.
• Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models.

🔗 Related Resources