Why Prompting is Hard

We are excited to be joined by J.D. Zamfirescu-Pereira, a Ph.D. student at UC Berkeley. He focuses on the intersection of human-computer interaction (HCI) and artificial intelligence (AI). He joins us to share his work in his paper, Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts.

J.D. began the conversation by discussing how chatbots have changed over time and how LLMs impact them. He then discussed what it means to build a chatbot. He shared the tools he utilized to create chatbots, including those for improving the chatbot’s adaptability, managing dialogue flow, and handling decision points.

J.D. discussed a challenge where people with less technical know-how struggle to utilize LLM-enabled chatbots fully. He discussed the tool he and his colleagues built, which helped non-AI experts create better prompts. He provided an example of how a chatbot creates an instructional cooking recipe. He also talked about having chatbots give results that people actually want, although measuring desirability is challenging.

J.D. spoke on the future of bot development. He also talked about certain concerning aspects of LLMs. Rounding up, he shared his future research. Follow J.D. on Twitter @jdzamfi.