
Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon
Descripción del Episodio
Aishwarya Naresh Reganti and Kiriti Badam have helped build and launch more than 50 enterprise AI products across companies like OpenAI, Google, Amazon, and Databricks. Based on these experiences, they’ve developed a small set of best practices for building and scaling successful AI products. The goal of this conversation is to save you and your team a lot of pain and suffering.
We discuss:
1. Two key ways AI products differ from traditional software, and why that fundamentally changes how they should be built
2. Common patterns and anti-patterns in companies that build strong AI products versus those that struggle
3. A framework they developed from real-world experience to iteratively build AI products that create a flywheel of improvement
4. Why obsessing about customer trust and reliability is an underrated driver of successful AI products
5. Why evals aren’t a cure-all, and the most common misconceptions people have about them
6. The skills that matter most for builders in the AI era
—
Brought to you by:
Merge—The fastest way to ship 220+ integrations: https://merge.dev/lenny
Strella—The AI-powered customer research platform: https://strella.io/lenny
Brex—The banking solution for startups: https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs
—
Transcript: https://www.lennysnewsletter.com/p/what-openai-and-google-engineers-learned
—
My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/183007822/referenced
—
Get 15% off Aishwarya and Kiriti’s Maven course, Building Agentic AI Applications with a Problem-First Approach, using this link: https://bit.ly/3V5XJFp
—
Where to find Aishwarya Naresh Reganti:
• LinkedIn: https://www.linkedin.com/in/areganti
• GitHub: https://github.com/aishwaryanr/awesome-generative-ai-guide
• X: https://x.com/aish_reganti
—
Where to find Kiriti Badam:
• LinkedIn: https://www.linkedin.com/in/sai-kiriti-badam
• X: https://x.com/kiritibadam
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Introduction to Aishwarya and Kiriti
(05:03) Challenges in AI product development
(07:36) Key differences between AI and traditional software
(13:19) Building AI products: start small and scale
(15:23) The importance of human control in AI systems
(22:38) Avoiding prompt injection and jailbreaking
(25:18) Patterns for successful AI product development
(33:20) The debate on evals and production monitoring
(41:27) Codex team’s approach to evals and customer feedback
(45:41) Continuous calibration, continuous development (CC/CD) framework
(58:07) Emerging patterns and calibration
(01:01:24) Overhyped and under-hyped AI concepts
(01:05:17) The future of AI
(01:08:41) Skills and best practices for building AI products
(01:14:04) Lightning round and final thoughts
—
Referenced:
• LevelUp Labs: https://levelup-labs.ai/
• Why your AI product needs a different development lifecycle: https://www.lennysnewsletter.com/p/why-your-ai-product-needs-a-different
• Booking.com: https://www.booking.com
• Research paper on agents in production (by Matei Zaharia’s lab): https://arxiv.org/pdf/2512.04123
• Matei Zaharia’s research on Google Scholar: https://scholar.google.com/citations?user=I1EvjZsAAAAJ&hl=en
• The coming AI security crisis (and what to do about it) | Sander Schulhoff: https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis
• Gajen Kandiah on LinkedIn: https://www.linkedin.com/in/gajenkandiah
• Rackspace: https://www.rackspace.com
• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper
• Semantic Diffusion: https://martinfowler.com/bliki/SemanticDiffusion.html
• LMArena: https://lmarena.ai
• Artificial Analysis: https://artificialanalysis.ai/leaderboards/providers
• Why humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead): https://www.lennysnewsletter.com/p/why-humans-are-ais-biggest-bottleneck
• Airline held liable for its chatbot giving passenger bad advice—what this means for travellers: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know
• Demis Hassabis on LinkedIn: https://www.linkedin.com/in/demishassabis
• We replaced our sales team with 20 AI agents—here’s what happened | Jason Lemkin (SaaStr): https://www.lennysnewsletter.com/p/we-replaced-our-sales-team-with-20-ai-agents
• Socrates’s quote: https://en.wikipedia.org/wiki/The_unexamined_life_is_not_worth_living
• Noah Smith’s newsletter: https://www.noahpinion.blog
• Silicon Valley on HBO Max: https://www.hbomax.com/shows/silicon-valley/b4583939-e39f-4b5c-822d-5b6cc186172d
• Clair Obscur: Expedition 33: https://store.steampowered.com/app/1903340/Clair_Obscur_Expedition_33/
• Wisprflow: https://wisprflow.ai
• Raycast: https://www.raycast.com
• Steve Jobs’s quote: https://www.goodreads.com/quotes/463176-you-can-t-connect-the-dots-looking-forward-you-can-only
—
Recommended books:
• When Breath Becomes Air: https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X
• The Three-Body Problem: https://www.amazon.com/Three-Body-Problem-Cixin-Liu/dp/0765382032
• A Fire Upon the Deep: https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email
Episodios Recientes

Más podcasts de Sociedad y Cultura
Ver toda la categoría →
老婆等我之火影重生|火影同人|多人有声剧|多播
By shows
<p style="color:#333333;font-weight:normal;font-size:16px;line-height:30px;font-family:Helvetica,Arial,sans-serif;hyphens:auto;text-align:justify;">“表哥!”佐助回头大叫一声,然而,背后突然出现了一幅惊心动魄的画面:一群从来没有见过的怪兽咆哮着向他冲来,怪售长着一副闪亮的铠甲,扁平的身体上方有一个凸起,凸起中伸出一根长长的管子,与其说怪兽,不如说是一群钢铁机器!只见那群钢铁机器长长、的管子里火光一闪,一颗颗尖头的金属就被抛射出来,在佐助周围炸开,周围一片火海滔滔,瞬间毁灭了周围的建筑和街道!佐助心里紧张万分,只剩下一个念头,那就是逃!赶快逃!以最快的速度逃走!他不想死在这里!虽然身上现在加上自己的负重铁块总共有约三倍的重力,但是,他心底的潜力被极度激发,完全忽视了中立的影响。</p><br /><p style="color:#333333;font-weight:normal;font-size:16px;line-height:30px;font-family:Helvetica,Arial,sans-serif;hyphens:auto;text-align:justify;"> 我站在一棵大树上,看到佐助在空无一人的街道上发狂奔跑,好几次快要跑不动了,却又重新激发了潜能,一次次突破了身体的障碍,最后竟然跑起来就不停了!我笑着一挥手,佐助立即喘着气瘫倒在地上。</p><br /><span><br /></span><span><br /></span><br />

Materia Oscura
By shows
Materia Oscura es un programa especializado en Ciencia del periodista José Manuel Nieves. Universo, física, hallazgos, nuevas investigaciones... Todo pasa por el tamiz de Nieves.

Next Question with Katie Couric
By shows
<p>Tired of political headlines that feel like déjà vu? Wondering if you <em>actually</em> need to care about every controversy? This season, Katie’s asking those same questions—and talking to the people who have real answers. From political insiders to sharp-eyed journalists, she’s breaking down what’s worth your attention (and what’s not), with smarts, sanity, and even a little humor. Because none of us can be tuned in 24/7—but we <em>can</em> stay informed without losing our minds. </p><p>Tune in every Thursday and join Katie Couric and her guests for a conversation on NEXT QUESTION.</p><hr><p style='color:grey; font-size:0.75em;'> Hosted on Acast. See <a style='color:grey;' target='_blank' rel='noopener noreferrer' href='https://acast.com/privacy'>acast.com/privacy</a> for more information.</p>

Serialously with Annie Elise
By shows
Serialously is hosted by Annie Elise, best known for her true-crime YouTube channel “10 to Life” with over 1.5M subscribers. Annie created Serialously after decades of listening to true crime and craving more than just the headlines; she wanted all the overlooked details everyone else seemed to miss. Here, you’ll find everything you’d normally end up Googling yourself - the timelines, the red flags, the social media history, and everything in between. Even better, you’ll hear the story the way you’d want to: like your best friend is filling you in. Annie covers cases in an engaging, conversational way with every detail sleuthed out. Whether it’s breaking true crime news, viral cases, or ongoing trials, Serialously keeps you in the loop in real time - just like having a conversation with your best friend, or hopefully your new true crime bestie! If you've been looking for someone to chat all things true crime with you and a community that’s just as curious as you are - welcome, we're happy you're here! xx *This podcast is not affiliated with The New York Times.

My Mom’s Murder
By shows
<p>Lauren Malloy was a baby when her mother, Lori, was murdered in East Providence, Rhode Island. For thirty years, no one told her. Everyone said Lori died of natural causes. Then a stranger sent Lauren a message — and the lie collapsed. Now Lauren is investigating her own mother’s murder. She’s recorded hundreds of hours of interviews with law enforcement, family, childhood friends, and people who won’t stop contradicting each other. Every answer opens a darker question. Every person she trusts seems to be hiding something. And the closer she gets to the truth, the more she realizes the people she trusts may be the ones with the most to hide. A 10-part true crime investigation hosted by the victim’s daughter. Lauren’s mission: to finally uncover the identity of Lori’s killer.</p><p>Audible subscribers can binge all 10 episodes early and ad-free right now. Join Audible in the Audible App or by subscribing on Apple Podcasts.</p><p><br></p>

Reaching Out
By shows
<p>Hailed by Ira Glass as the “best new show I’ve heard in years,” and winner of the Audio Memoir Award of the Tribeca Film Festival, <em>Reaching Out</em> is a story about fathers, sons, and the distance between.<strong> </strong>The story begins<strong> </strong>on the eve of twin brothers’ Sayre and Connor’s 30th birthday, when they set out to find the biological father who abandoned them before they were born. Their search pulls them through desert raves, drunken nightclubs, and across borders of time and language, forcing the brothers to confront the truths they’ve hidden from each other and themselves. And it’s not long before the twins discover that their father is closer than they ever could have imagined. </p><p><br></p><hr><p style='color:grey; font-size:0.75em;'> Hosted on Acast. See <a style='color:grey;' target='_blank' rel='noopener noreferrer' href='https://acast.com/privacy'>acast.com/privacy</a> for more information.</p>

PSICOCAFÉ
By shows
<p>Psicocafé es un espacio de conversación donde la psicología se vuelve cercana, clara y necesaria.</p><p>Aquí hablamos de ansiedad, relaciones, familia, emociones y todo eso que vivimos… pero no siempre sabemos cómo entender.</p><p>Un podcast para cuestionarte, reflexionar y ver tu vida desde otra perspectiva.</p><p>Prepara tu café y acompáñanos.</p>

Sucedió una noche
By shows
Un podcast de película sobre cine clásico: las estrellas míticas, las películas que todos recordamos y las miles de anécdotas y curiosidades que hicieron del cine la gran pasión de nuestro tiempo. Con Antonio Martínez y Elio Castro. Suscríbete y escucha cine cuando y donde quieras.