Supra

When you launch an AI feature, the user experience isn't defined by your UI design. It's defined by what comes OUT of the model. So why aren't more PMs involved in AI model evaluation? Usually because: ? They think they need deep ML knowledge ? They're intimidated by the technical jargon ? They believe it's "the data team's job" The truth? You need ZERO ML expertise to start evaluating AI outputs. What you DO need is empathy for both users and models. In a recent Supra roundtable, Julia Winn shared these tips for developing model evaluation skills: 1/ Find existing model work in your org. ??Surprise them with: "I'd like to help with your evaluation." ??(Spoiler: They'll be thrilled. Nobody enjoys this work.) ?? ??Don't wait for a formal AI project. Start with something simple like a recommendation engine already in production. 2/ Start with a small dataset. ?? ??Example: For a customer support classifier that detects urgent tickets: ??? Pull 20-30 tickets across categories ??? Compare the model's output against human judgment ?? ??The goal is spotting patterns like "model misses urgency when multiple issues are in one ticket." 3/ Create a simple framework: ??? Strong pass (delivers clear value) ??? Borderline pass (technically correct but weak) ??? Clear fail (would hurt user experience) ?? ??Avoid complex rubrics with multiple dimensions. This simple framework keeps discussions focused on what matters. 4/ Focus on examining the failures. ??Ask: "What context is missing?" not "Why is the model wrong?" ?? ??When a model gets something wrong, it's rarely because the algorithm is flawed—it's almost always missing critical context. 5/ Build your own evaluations for practice. ??Use ChatGPT or Claude to test ideas systematically. ?? ??Try an AI summarizer for your favorite blog. Define what "good" looks like and analyze the patterns. The goal isn't becoming a data scientist. It's developing an intuition for how models perceive your users' world. Where are you currently involved in AI evaluation?

Most of us know that switching jobs is the fastest way to boost your compensation. But what if you love your current role and team? Here's how Supra members have negotiated a meaningful raise without having to leave: 1/ Do the homework for your manager Come prepared with a specific target number: "I'm currently at $X and would like to be at $Y." That number should be based on internal bands and reliable benchmarks. 2/ Justify the increase with quantifiable value Document how you've taken on responsibilities beyond your original role. Focus on three value dimensions: ? Increased revenue ? Reduced costs ? Improved speed/efficiency 3/ Bring accolades and testimonials High performers are often "invisible" to busy managers. Include praise from other teams, clients, or stakeholders who've noticed your impact. 4/ Deliver as an organized package Present your case in person, then follow up with an email summary that can be forwarded to decision-makers. If you hear "no" or "not yet," don't give up. Ask for a specific action plan: "What do I need to accomplish to earn this compensation, and on what timeline?" This gives you clarity to either pursue that roadmap or recognize it's time to look elsewhere. The key difference from new job negotiation? Your current employer already knows your salary, so your leverage comes from documenting the gap between your initial compensation and your current value. What other strategies have worked for you when negotiating internal raises?

Curious about what top Product Leaders are reading right now? Here are the top 5 pieces of content shared in Supra this month: 1/ How I Use LLMs by Andrej Karpathy. A masterclass in practical LLM usage. Karpathy breaks down his workflow with ChatGPT, Claude, Cursor, and NotebookLM—from coding to podcast generation. Learn when to use different models, how to think about pricing tiers, and combining tools like voice input with advanced data analysis. A treasure trove of hands-on tips from one of the best in the field. https://lnkd.in/dii4_KQM 2/ Prototype Development Lifecycle by Colin Matthews The guide you need if you're stuck between moving fast and building right. It outlines a clear four-stage lifecycle—from "prove value" to "scale confidently"—helping teams avoid endless MVP loops. Perfect for PMs juggling early-stage AI prototypes while thinking about long-term stability. https://lnkd.in/dBxvXp8r 3/ The Lazy PM's Guide to Customer Research by Justin Williams A goldmine for PMs who want to stay close to users without launching full research sprints. From setting up passive feedback loops to using AI for faster synthesis, it's packed with practical tips to embed lightweight research into your weekly workflow. If you've ever skipped research because it felt too heavy, this guide will help you build smarter habits without burning out.?https://lnkd.in/dfsrnHvW 4/ Career Advice for 2025 by Will Larson. Larson urges professionals to treat their career like a product—know your differentiators, build long-term leverage, and invest in relationships. Key themes: bet on learning curves, don't chase titles, and optimize for scope over prestige. A grounded, strategic guide for navigating tech careers in an evolving industry. https://lnkd.in/dAfBgdrC 5/ Time Management for Hands-On Product Leaders by Ravi Mehta Breaking the false choice between strategy and execution, Ravi introduces a simple yet powerful framework—Lead, Manage, Build—to balance your time. With practical tips on avoiding context-switching traps, this is essential for PMs scaling into bigger roles without losing touch with the craft. https://lnkd.in/d_inKwpz What other great content did you read this month? For context, I run Supra, a private community of high-caliber product leaders. Our members share fantastic pieces via our Slack channel.

Had a blast chatting with my friend Marc Baselga on the Supra Insider podcast! We dove into my journey from product leadership at Google to starting Blossomer, where I help early-stage B2B founders get their first 1,000 customers—without hiring a sales team. We went deep on some topics I’ve been writing reflecting on: ? Why I left a job I loved to build something of my own ? The tradeoffs between bootstrapping and raising VC ? What most founders get wrong about distribution—and how to fix it ? How AI tools have changed the game for solo builders and consultants Plus, we got tactical on things like: 1/ How to get early traction without a full GTM team 2/ Why cold outbound works (if you do it right) 3/ How I use custom GPTs + automation to save hours every week 4/ How product leaders can sharpen their growth intuition without leaving their job If you’re a founder, builder, or product leader trying to level up your go-to-market skills—or wondering what it’s like to go solo—hope you’ll find this one helpful! ??? You can listen to the full episode here: https://lnkd.in/gfVebyvn

Building product feels so right, but distribution always wins the fight. In this week's Super Insider episode, Phil Ou ?? (former Google PM who led growth for Google Photos) shares why he left his prestigious role to start Blossomer, helping early-stage B2B founders get their first 1,000 customers without hiring a sales team. During this conversation, Phil reveals: ? Why he decided to bootstrap rather than pursue venture funding (and how this approach gives him more freedom) ? What most founders get completely wrong about go-to-market (hint: it's all about relevance, not just personalization) ? How PMs can develop stronger distribution intuition (and why tagging along on sales calls was one of the most valuable things he did at Google) ? Time-saving AI tools and workflows that have transformed his productivity (including his custom GPT setup that helps find blind spots) I was bummed to miss this recording due to dad duties with our 2 year old who had an eye infection but loved it as a listener and learned a ton. If you're a current or aspiring founder, this one has lots of gold nuggets. Episode link coming soon.

Why Distribution Skills Are Becoming Essential for Product Leaders Most PMs spend 90% of their time building and only 10% thinking about distribution. In a world where AI commoditizes building speed, the products that win aren't just well-built - they're well-distributed. In our recent Supra Insider episode, Phil Ou ?? (founder of Blossomer and ex-Google product leader) explained why distribution skills are becoming non-negotiable for today's product leaders: 1/ Traditional silos are disappearing fast ? Marketing, sales, and product teams no longer operate in isolation ? AI has torn down boundaries between these functions ? Product leaders increasingly need to understand all customer touchpoints ? Messaging consistency across touchpoints is now critical for success 2/ Early distribution testing delivers crucial product insights ? Cold outreach provides real-time market feedback ? Sales conversations reveal positioning strengths and weaknesses ? Customer objections highlight product gaps ? Win/loss analysis uncovers competitive advantages "Whether it's marketing, the cold email you're sending, or your product experience - those are touchpoints at a specific point in time for your customer. You need that level of coordination and consistency for your message to land," Phil explained. Don't assume that great products will naturally find their audience. Even with AI accelerating development cycles, products that win are those with intentional, systematic distribution strategies from day one. How are you building your distribution muscles as a product leader? --- Our full episode with Phil covers:? ? Why he left his dream PM role at Google to start his own agency ? Common go-to-market mistakes founders make ? The power of relevance over personalization in outreach ? His framework for using AI as a thinking partner vs. automator ? Practical systems for learning from wins and losses Link to full episode :https://lnkd.in/df53S6VN

Want to network with other high-caliber product leaders in person? In March and April, we are hosting in-person events for Product leaders in Oakland, Atlanta, and Boston. The events are small (8-14 attendees), and we only have two spots for non-Supra members, so I apologize in advance for not being able to accommodate everyone. If you are interested, check out the links below ?? ?? 03/27 - Supra Oakland Meetup: https://lu.ma/0pk0pl97 ?? 03/27- Supra Boston Meetup: https://lu.ma/irh2ze26 ?? 04/03 - Supra Atlanta Meetup: https://lu.ma/zr2mek1e

The most valuable change AI brings to product development isn't the end product. It's the speed of iteration. In our recent Supra Insider episode, Rodrigo Davies (AI Product Leader at Asana) revealed how AI is completely transforming their product process: "With AI, getting from 0 to 80 is really fast. Getting from 80 to 100 can be really slow." This new reality is upending how Asana approaches product development: 1/ AI enables lightweight prototyping before research ? Before: Write a lengthy spec, debate hypothetical features ? Now: "Don't wait to do tons of research. Start prototyping early." The old objection "Can AI do X?" has disappeared. Now you can just go find out in minutes. As Rodrigo explained: "If I had access to this data, and I gave it these instructions, would I get something useful at the end?" 2/ AI flips the traditional product process ? Before: PRDs as alignment artifacts → design → prototype → iterate ? Now: Rapid prototype → show customers → iterate → document decisions "We used to use the PRD as the place to figure out the ideas, get alignment. Why was that? It was because it was seen as expensive to have to go do designs or build prototypes. What AI is doing is just collapsing the cost of doing that." 3/ AI democratizes feature exploration ? "Everyone can ride along... You don't need to be an engineer to do prompt prototyping" 4/ Engineers aren't the bottleneck anymore ? Anyone can validate if an idea is feasible ? Cross-functional teams can iterate together on solutions ? Teams can validate ideas with real customer data quickly But there's a critical insight: AI does NOT replace thoughtful product development. It just accelerates the journey from idea to feedback. "The beauty of AI is you think about how AI was trained... it's like being able to run the simulation N times. If we could do that in product development, how much better choices would we make?" What if every time there was a product disagreement, you could instantly visualize multiple solutions instead of debating hypotheticals? --- Our full episode with Rodrigo covers:? ? How Asana drove AI adoption across the company ? Creating a culture of AI tinkering ? Balancing AI innovation with large enterprise customers ? Using AI for personal use cases (including his fascinating Strava running coach setup) ? Building intuition for what's possible with AI Link to episode: https://lnkd.in/d2Qgza93

Positive feedback has a higher ROI than constructive feedback. This wasn't obvious to me before, but now it seems so clear: When I give positive feedback, I'm reinforcing behavior someone is already doing well. They know how to do it, they're capable of it, and they're motivated to continue. When I give constructive feedback, I'm highlighting a gap. The person needs to: ? Learn a new skill ? Change an existing habit ? Figure out how to implement the feedback One requires immediate action, the other requires a learning curve.

Everyone's amazed by how fast AI tools can create prototypes. But the real game-changer isn't the speed - it's how these tools transform our conversations with key stakeholders. The best product reviews I've participated in always involve a working prototype. Why? Because prototypes shift discussions from abstract concepts to concrete experiences.? They bypass the limitations of language, reduce misinterpretations, and create a shared reference point that words alone cannot achieve. AI prototyping doesn't just save time building mockups - it multiplies the number of high-quality, specific conversations we can have. Teams align faster.? Stakeholders give more useful feedback. Engineers understand requirements immediately. The result? Faster convergence on great ideas and a dramatic acceleration in how quickly teams can move from concept to validated solution. -- Credit to Colin Matthews for illuminating this perspective in our recent Supra roundtable. If you don't follow him already, he's my go-to person for AI prototyping and his courses are top-notch.