LaMDA: Language Models for Dialog Applications

🤖 Plain-English Summary

We present LaMDA: Language Models for Dialog Applications. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible.

🔑 Key Findings

• LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text.
• While model scaling alone can improve quality, it shows less improvements on safety and factual grounding.
• We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.

🔗 Related Resources