Welcome to the latest issue of the AI Community Learning Series – a space to explore how leaders and innovators are reshaping industries with AI 🚀. In this session, Priya Mathew Badger, Principal Product Manager at Yelp, showed us how she crafts agentic AI features, balances speed and quality, and prototypes voice-based AI flows—all without writing a single line of code. If you missed it, catch the replay here.
Here are the five key takeaways from our discussion. 👇
1. Gen AI product development: Small details matter
One of the first lessons Priya shared was deceptively simple: when building features with Gen AI, start with what problems it solves for the user and sweat the details. She explained how there have been lots of iterations on the product experience both in the user interface and the behind the scenes system instructions and prompts.
She described how they built Yelp assistant for handling requests to service providers because they saw that consumers had a lot of context they wanted to provide about their service needs that was hard to fit into the existing closed form-like structure. Moving to a more conversational flow helped with this but users need education that they can do that so things like the prompt suggestions leading with problem first statements (“I need…”, “I want…”) and even the ghost text make a big difference.
2. Don’t be afraid of evals - they don’t require technical knowledge
It’s also important to have feedback loops on both the user interface side and how the conversation flow is working. Priya’s team uses feedback that users give in the interface and through A/B testing data but also does regular conversation reviews and evaluations to track performance over time.
“Conversation evals don’t need to be a technical thing from a PM perspective. It’s about looking at what your consumers are actually doing with your product and evaluating if that’s a good experience or not.”
3. PMs Can—and Should—Collaborate on the Prompt
When working on the assistant’s new memory feature, Priya noticed that the LLM was saving the wrong types of data—like appointment time of requests that didn’t apply to future needs. Instead of filing a ticket, she asked the engineer to expose the prompt field in their internal tool.
Just 30 minutes and a few prompt tweaks later, the assistant resolved the issues saving only stable information like lawn size, number of bedrooms and bathrooms in a home instead of transient information that was likely to change. “This is a really exciting time for PMs who are creator-oriented,” Priya said. “System instructions and prompts are natural language. It doesn’t actually require code. It just requires a little bit of curiosity to learn prompt engineering best practices.”
4. For Conversational Prototyping - start with examples
When working on a new conversational experience you can prototype fast with a lightweight method using Claude and Claude artifacts.
She showed how she used Claude to quickly come up with example conversations for a new do it yourself capability within the assistant. Then prompted it to create an AI prototype of that conversation that simulated the use case.
You can try it out by creating examples for the type of conversation you want an assistant to have with a user and then using this prompt to create a clickable prototype that incorporates those examples as system instructions for the assistant.
“Let's use these examples to now create an actual AI assistant app as an artifact that showcases this. It should have a chat interface that allows the user to enter their issue and then the AI responds using LLM based on system instructions generated from these examples.”
🛠️ Try building your own Claude Artifact
5. For exploring new ideas - AI prototypes are the new wireframe
In one of the session’s highlights, Priya walked through how she prototypes new features using Claude and Magic Patterns—without writing any code. She demoed how to simulate a DIY troubleshooting flow for Yelp users by feeding prompts into Claude, refining the conversation, and then spinning it into a clickable prototype.
“It’s like handing your intern a conversation and watching them build the app,” she joked.
With Magic Patterns, she showed how to explore visual variations like floating buttons or voice input interfaces. She even simulated a full-screen recording UX with waveform animations and voice transcription all built through natural language instructions.
🛠️ Try building your own Magic Patterns prototype
If you max out credits, get a discount for the hobby/pro plans with Priya’s referral code, PRIYA20.
💡 Final Reflection
This is an exciting time for product managers who want to create.
“We’re trying to figure out what are the new collaboration models between PM, Eng, and design. It’s a fun time for PMs where we can really be a part of the creating process”.
What was your favorite insight from Priya’s session? Reply and share what you’re excited to try in your own AI prototyping journey!