The Five-Layer Cake Approach to Scaling AI Without Wasting Money

The Five-Layer Cake Approach to Scaling AI Without Wasting Money

Author: Demetrios September 4, 2026 Duration: 38:55

In this episode of Agentic Conversations, we sit down with Ambud Sharma, Principal Engineer at Pinterest, responsible for general technology efficiency, fresh off delivering a controversial keynote on AI infrastructure optimization at scale.


Ambud walks us through his Five Layer Cake framework - a structured approach to driving efficiency across every level of the AI stack, from silicon and hardware procurement to model selection, inference engine design, and governance. We explore how decisions compound across layers to unlock real business growth, and how the wrong choices can lock you into expensive commitments for years.


We stress test the framework against two very different business models: what the stack looks like if you are building the next Cursor, and how it changes entirely if you are building the next YouTube. Along the way we cover hardware immutability, inference engine warm-up costs, GPU occupancy, context switching, quantization trade-offs, model routing, and why experimentation discipline is the only thing that keeps AI infrastructure costs from getting out of control.


We also look at how this framework holds up in the emerging agent era, what changes when agent-to-agent communication becomes the norm, and why agent traffic just passed bot traffic on Cloudflare. The conversation closes on a deceptively simple takeaway: there is no silver bullet, and experimentation at every layer always comes first.


Pinterest: https://about.pinterest.com/


Alex Salkever: https://www.linkedin.com/in/alexsalkever

Ambud Sharma: https://www.linkedin.com/in/ambud


Timestamps:

[0:00] Introduction and the controversial keynote

[2:09] The five-layer cake explained

[4:30] Why hardware decisions are irreversible

[6:47] Two business models: building Cursor vs YouTube

[10:42] Applying the five layers to a YouTube-style company

[14:23] Experimentation as the core efficiency method

[17:09] Inference stack: context switching and warm-up costs

[20:07] Model layer: why changing models breaks everything

[24:10] When you should not use an LLM at all

[26:41] Governance and routing: right model for the right task

[29:20] Horror stories of unchecked token spend

[31:37] Experimentation discipline without stifling innovation

[34:35] How the five layers change in the agent era

[36:05] Agent-to-agent communication and governance complexity

[38:27] Core takeaway: experimentation first at every layer


Hosted by Demetrios, MLOps.community is a space for honest, meandering talks about the real work of making artificial intelligence systems actually work. This isn't about hype or theoretical papers; it's about the messy, practical, and often surprising journey of taking models from a notebook into a live environment. You'll hear from engineers and practitioners who are in the trenches, discussing the tools, the frustrations, and the occasional breakthroughs that define the day-to-day. The conversations are deliberately relaxed, covering everything from traditional machine learning pipelines to the new world of large language models and even the intangible "vibes" of team culture and process. Each episode peels back a layer on what "production" really means, whether that involves deploying a predictive service, managing an agentic system, or maintaining reliability as everything scales. Tuning into this podcast feels like grabbing a coffee with colleagues who aren't afraid to dig into the technical nitty-gritty while keeping the tone conversational and accessible. It's for anyone who builds, manages, or is just curious about the operational backbone that allows AI to deliver value, offering a grounded perspective often missing from the broader conversation.
Author: Language: en-us Episodes: 50

Agentic Conversations (formally mlops.community)
Podcast Episodes
Inside Just Eat's AI Lab: Voice Agents & Agentic Commerce [not-audio_url] [/not-audio_url]

Duration: 1:18:38
Guthrie Cooper (Senior Group Product Manager, AI & Robotics) and Nidhi Sharma (Global Head of Engineering AI & Incubation) from Just Eat Takeaway.com join the MLOps.community to pull back the curtain on how one of Europe…
Autonomous Agents at Work: From OpenClaw Hype to Enterprise Reality [not-audio_url] [/not-audio_url]

Duration: 42:19
Pramod Krishnan is a Managing Director - AI Managed Services at PwC, specializing in enterprise AI transformation — helping large organizations move from AI experimentation to production operating models. In this episode…
Agents are Just While Loops [not-audio_url] [/not-audio_url]

Duration: 41:11
Hamza Tahir, co-founder of ZenML, joins the show to cut through the hype around long-running agents — arguing that at the end of the day, an agent is just a while loop that talks to a model, calls a tool, and writes to a…
The Latency Goldilocks Zone Explained [not-audio_url] [/not-audio_url]

Duration: 48:13
Rafael (Head of Innovation, iFood) and Daniel (Data and AI Manager, iFood) pull back the curtain on ILO-Agent — iFood's conversational AI ordering system built for 200 million users across Latin America. Recorded live at…
Building MCP Before MCP Existed: Inside Despegar's Sofia Agent [not-audio_url] [/not-audio_url]

Duration: 41:13
Nicolas Alejandro Bogliolo is the AI PM at Despegar, the largest online travel agency in Latin America, and the engineer-product-hybrid behind Sofia, the GenAI travel concierge that beat most of the OTA world to a workin…
Voice Agent Use Cases [not-audio_url] [/not-audio_url]

Duration: 51:04
This episode is brought to you by the MLflow team. Check out more information at MLflow.org.What does it actually take to build voice AI at a billion-interaction scale? This episode features an ex-Amazon voice AI enginee…
It's 2026, and We're Still Talking Evals [not-audio_url] [/not-audio_url]

Duration: 40:56
Maggie Konstanty is an AI Product Manager at Prosus, one of the world's largest consumer internet companies, where she builds and evaluates AI agents for food ordering and ecommerce at scale. She's been inside the messy…
Why Agents are Driving Software Development to the Cloud [not-audio_url] [/not-audio_url]

Duration: 51:07
This episode is brought to you by Hyperbolic and the MLflow team. Check out more information at hyperbolic.ai and MLflow.org.Why AI Coding Agents Are Moving to the Cloud — With Zach Lloyd, CEO of WarpZach Lloyd is the fo…
The Modern Software Engineer [not-audio_url] [/not-audio_url]

Duration: 53:37
This episode is brought to you by the MLflow team. Check out more information at MLflow.org.Mihail Eric is Head of AI at Monaco and Adjunct Lecturer at Stanford University, where he teaches CS146S: "The Modern Software D…