Background

Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon

Lenny's Podcast: Product | Career | Growth•11 de enero de 2026•5182•
Compartir episodio:Descargar

Descripción del Episodio

Aishwarya Naresh Reganti and Kiriti Badam have helped build and launch more than 50 enterprise AI products across companies like OpenAI, Google, Amazon, and Databricks. Based on these experiences, they’ve developed a small set of best practices for building and scaling successful AI products. The goal of this conversation is to save you and your team a lot of pain and suffering.

We discuss:

1. Two key ways AI products differ from traditional software, and why that fundamentally changes how they should be built

2. Common patterns and anti-patterns in companies that build strong AI products versus those that struggle

3. A framework they developed from real-world experience to iteratively build AI products that create a flywheel of improvement

4. Why obsessing about customer trust and reliability is an underrated driver of successful AI products

5. Why evals aren’t a cure-all, and the most common misconceptions people have about them

6. The skills that matter most for builders in the AI era

—

Brought to you by:

Merge—The fastest way to ship 220+ integrations: https://merge.dev/lenny

Strella—The AI-powered customer research platform: https://strella.io/lenny

Brex—The banking solution for startups: https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs

—

Transcript: https://www.lennysnewsletter.com/p/what-openai-and-google-engineers-learned

—

My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/183007822/referenced

—

Get 15% off Aishwarya and Kiriti’s Maven course, Building Agentic AI Applications with a Problem-First Approach, using this link: https://bit.ly/3V5XJFp

—

Where to find Aishwarya Naresh Reganti:

• LinkedIn: https://www.linkedin.com/in/areganti

• GitHub: https://github.com/aishwaryanr/awesome-generative-ai-guide

• X: https://x.com/aish_reganti

—

Where to find Kiriti Badam:

• LinkedIn: https://www.linkedin.com/in/sai-kiriti-badam

• X: https://x.com/kiritibadam

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Aishwarya and Kiriti

(05:03) Challenges in AI product development

(07:36) Key differences between AI and traditional software

(13:19) Building AI products: start small and scale

(15:23) The importance of human control in AI systems

(22:38) Avoiding prompt injection and jailbreaking

(25:18) Patterns for successful AI product development

(33:20) The debate on evals and production monitoring

(41:27) Codex team’s approach to evals and customer feedback

(45:41) Continuous calibration, continuous development (CC/CD) framework

(58:07) Emerging patterns and calibration

(01:01:24) Overhyped and under-hyped AI concepts

(01:05:17) The future of AI

(01:08:41) Skills and best practices for building AI products

(01:14:04) Lightning round and final thoughts

—

Referenced:

• LevelUp Labs: https://levelup-labs.ai/

• Why your AI product needs a different development lifecycle: https://www.lennysnewsletter.com/p/why-your-ai-product-needs-a-different

• Booking.com: https://www.booking.com

• Research paper on agents in production (by Matei Zaharia’s lab): https://arxiv.org/pdf/2512.04123

• Matei Zaharia’s research on Google Scholar: https://scholar.google.com/citations?user=I1EvjZsAAAAJ&hl=en

• The coming AI security crisis (and what to do about it) | Sander Schulhoff: https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis

• Gajen Kandiah on LinkedIn: https://www.linkedin.com/in/gajenkandiah

• Rackspace: https://www.rackspace.com

• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper

• Semantic Diffusion: https://martinfowler.com/bliki/SemanticDiffusion.html

• LMArena: https://lmarena.ai

• Artificial Analysis: https://artificialanalysis.ai/leaderboards/providers

• Why humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead): https://www.lennysnewsletter.com/p/why-humans-are-ais-biggest-bottleneck

• Airline held liable for its chatbot giving passenger bad advice—what this means for travellers: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know

• Demis Hassabis on LinkedIn: https://www.linkedin.com/in/demishassabis

• We replaced our sales team with 20 AI agents—here’s what happened | Jason Lemkin (SaaStr): https://www.lennysnewsletter.com/p/we-replaced-our-sales-team-with-20-ai-agents

• Socrates’s quote: https://en.wikipedia.org/wiki/The_unexamined_life_is_not_worth_living

• Noah Smith’s newsletter: https://www.noahpinion.blog

• Silicon Valley on HBO Max: https://www.hbomax.com/shows/silicon-valley/b4583939-e39f-4b5c-822d-5b6cc186172d

• Clair Obscur: Expedition 33: https://store.steampowered.com/app/1903340/Clair_Obscur_Expedition_33/

• Wisprflow: https://wisprflow.ai

• Raycast: https://www.raycast.com

• Steve Jobs’s quote: https://www.goodreads.com/quotes/463176-you-can-t-connect-the-dots-looking-forward-you-can-only

—

Recommended books:

•  When Breath Becomes Air: https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X

• The Three-Body Problem: https://www.amazon.com/Three-Body-Problem-Cixin-Liu/dp/0765382032

• A Fire Upon the Deep: https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email

Episodios Recientes

Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon

Más podcasts de Sociedad y Cultura

Ver toda la categoría →
PSICOCAFÉ

PSICOCAFÉ

By shows

<p>Psicocafé es un espacio de conversación donde la psicología se vuelve cercana, clara y necesaria.</p><p>Aquí hablamos de ansiedad, relaciones, familia, emociones y todo eso que vivimos… pero no siempre sabemos cómo entender.</p><p>Un podcast para cuestionarte, reflexionar y ver tu vida desde otra perspectiva.</p><p>Prepara tu café y acompáñanos.</p>

Sucedió una noche

Sucedió una noche

By shows

Un podcast de película sobre cine clásico: las estrellas míticas, las películas que todos recordamos y las miles de anécdotas y curiosidades que hicieron del cine la gran pasión de nuestro tiempo. Con Antonio Martínez y Elio Castro. Suscríbete y escucha cine cuando y donde quieras.

Young and Profiting (YAP) with Hala Taha: Entrepreneurship and Self-Improvement Podcast

Young and Profiting (YAP) with Hala Taha: Entrepreneurship and Self-Improvement Podcast

By shows

YAP is the must-listen podcast for driven entrepreneurs and high achievers who are hardcore about business, personal growth, and achieving their potential. Hosted by Hala Taha, self-made entrepreneur and CEO of YAP Media, this #1 entrepreneurship podcast features mini-masterclasses with business icons and industry legends like GaryVee, Alex Hormozi, Mel Robbins, Reid Hoffman, Kevin O’Leary, and Codie Sanchez who help you profit in all aspects of life—from boosting sales and beating algorithms to biohacking and productivity. Whether you’re launching a startup, scaling your business, running a side hustle, or upgrading your mindset, every YAP episode delivers actionable advice that WILL help you profit. Listen, Learn, and Profit with the Young and Profiting (YAP) podcast! For ad inquiries, please reach out to: [email protected]

Tony Mantor's : Almost Live..... Nashville

Tony Mantor's : Almost Live..... Nashville

By shows

<p>Nashville based veteran music producer Tony Mantor’s Almost Live… Nashville podcast takes its listener on a journey through the business of music and entertainment. Following the success and popularity of his top-rated podcast on Autism &amp; Mental Health awareness - Why Not Me? - Mantor hosts thought-provoking conversations and behind-the-scenes stories from various celebrity guests from the world of music, television and entertainment. With more than 30 years as an accomplished music producer based in Nashville, Mantor welcomes a worldly guest list of musicians, actors and entertainers who share their unique stories about the roles they’ve played in creating and sustaining what we know as popular American culture. Conversations and stories that also give a deeper understanding on what it takes to achieve success in the entertainment industry for aspiring entertainers who are currently pursuing their own dreams of being a star someday. Whether listening for the entertainment or storytelling factor alone, or for tips on how his guests faced their life’s challenges, Almost Live… Nashville will have a story for anyone who has been a fan of popular culture in music, movies and television over the last few decades.</p>

Power On with Mark Gurman

Power On with Mark Gurman

By shows

<p>Power On delivers Apple and consumer technology scoops, analysis and insider access you won&rsquo;t find anywhere else. Every Thursday, Bloomberg&rsquo;s Mark Gurman brings you exclusive reporting, behind-the-scenes insight, product reviews and expert analysis, with special guests from inside Bloomberg, including tech editor Vlad Savov, and across the technology industry. The podcast takes you through this wild moment in tech &mdash; one where AI is shaking up development, Apple&rsquo;s Ternus era has begun and consumer technology is set to change more in the next five years than it has in the last 20. Get the news first on the products, people and decisions shaping the world&rsquo;s most influential technology companies. From the next iPhone and the future of AI to the industry&rsquo;s biggest shake-ups, Power On tells you what&rsquo;s happening, why it matters and what&rsquo;s coming next.</p>

Tony Mantor's : Almost Live..... Nashville

Tony Mantor's : Almost Live..... Nashville

By shows

<p>Nashville based veteran music producer Tony Mantor’s Almost Live… Nashville podcast takes its listener on a journey through the business of music and entertainment. Following the success and popularity of his top-rated podcast on Autism &amp; Mental Health awareness - Why Not Me? - Mantor hosts thought-provoking conversations and behind-the-scenes stories from various celebrity guests from the world of music, television and entertainment. With more than 30 years as an accomplished music producer based in Nashville, Mantor welcomes a worldly guest list of musicians, actors and entertainers who share their unique stories about the roles they’ve played in creating and sustaining what we know as popular American culture. Conversations and stories that also give a deeper understanding on what it takes to achieve success in the entertainment industry for aspiring entertainers who are currently pursuing their own dreams of being a star someday. Whether listening for the entertainment or storytelling factor alone, or for tips on how his guests faced their life’s challenges, Almost Live… Nashville will have a story for anyone who has been a fan of popular culture in music, movies and television over the last few decades.</p>

The Woman in the Woods

The Woman in the Woods

By shows

Nick and Mindi Kassotis believed they were being hunted. According to friends and family, Nick's classified military work had put them both in danger, so much so that Mindi rarely left the house. But the couple also shared they were expecting a child. Then, just after Thanksgiving, Mindi died suddenly in the hospital. Soon after, Nick died too. That's what everyone who loved them believed, that is until a woman's dismembered remains turned up in a wooded area. Stories started to unravel and a mysterious new figure emerged. "48 Hours" correspondent Erin Moriarty separates fact from fiction in a case built upon unimaginable layers of deception.

 Inteligencia Artificial, tu resumen semanal

Inteligencia Artificial, tu resumen semanal

By shows

El resumen semanal de Inteligencia Artificial más útil para tu negocio, con tutoriales y estrategias que puedes aplicar desde hoy + entrevistas con las mentes más grandes de los negocios en México y Latinoamérica para que escuches directamente de ellos cómo funciona su negocio.