Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

🤖 Plain-English Summary

Pre-trained large language models (“LLMs”) like GPT-3 can engage in fluent, multi-turn instruction-taking out-of-the-box, making them attractive materials for designing natural language interactions. Expectations stemming from human-to-human instructional experiences, and a tendency to overgeneralize, were barriers to effective prompt design.

🔑 Key Findings

• Using natural language to steer LLM outputs (“prompting”) has emerged as an important design technique potentially accessible to non-AI-experts.
• Crafting effective prompts can be challenging, however, and prompt-based interactions are brittle.
• Here, we explore whether non-AI-experts can successfully engage in “end-user prompt engineering” using a design probe—a prototype LLM-based chatbot design tool supporting development and systematic evaluation of prompting strategies.

🔗 Related Resources