Why Cost Per Million Tokens Is A Useless KPI?

Why Cost Per Million Tokens Is A Useless KPI?

Author: Demetrios September 14, 2026 Duration: 38:49

A year ago, Palo Alto Networks built dashboards to track AI spend. Today those dashboards are useless, and the team that built them thinks that's the whole story.

Recorded at FinOps X in San Diego, this conversation brings together Abhinav Lad, who leads cloud and AI finance at Palo Alto Networks, and Kuntal Patel, who runs the cloud engineering function behind it. They explain what happened when agents entered the picture, and AI stopped behaving like a service anyone could forecast.

The short version: consumption went from linear to exponential almost overnight. Agents are goal-oriented rather than task-oriented, so they plan, call tools, verify, fail, retry, and keep looping until they hit the outcome, and every iteration is billable.

So how do you run finance on top of that? Abhinav and Kuntal walk through the metrics that replaced their old forecasts: adoption rate, cost per user, AI as a percentage of revenue - and the budget limits that let engineering leaders choose between the newest model and a longer runway. They get into the open question of whether a cheaper model saves money or just burns more tokens thinking. They explain why an AI gateway became the control plane for cost and security at the same time, why retry caps belong in the design phase instead of the postmortem, and how FinOps starts to resemble product QA once the bill becomes the clearest signal that something is broken.

They close on a warning worth sitting with: cost per million tokens is a number that means almost nothing on its own, and a value story built on it will point you somewhere you don't want to go.


Palo Alto Networks: https://www.paloaltonetworks.com


Abhinav Lad: https://www.linkedin.com/in/abhinav-lad

Kuntal Patel: https://www.linkedin.com/in/kuntalpatel35

Alex Salkever: https://www.linkedin.com/in/alexsalkever


Timestamps:

[0:00] Intro

[1:00] Who runs FinOps for AI at Palo Alto Networks

[2:10] Last year's AI dashboards are already useless

[4:26] Agents turned linear forecasts exponential

[7:27] Three traits that make agents expensive

[8:34] The hidden bill: RAG, vectors and egress

[9:16] Cost per user and adoption rate

[11:21] Giving engineering leaders a budget

[12:07] Using DORA metrics to prove value

[13:53] Where DORA stops fitting AI

[16:20] Does the cheaper model actually save money

[17:57] Why you need an AI gateway

[20:05] Inside Prisma AIRS

[21:00] Three cost models for three use cases

[22:52] Forecasting lessons from Electronic Arts

[24:03] Runaway agents and endless loops

[25:59] Capping retries before they burn cash

[28:06] Writing cost policy at design time

[29:01] When FinOps becomes product QA

[32:17] Explaining AI spend to the C-suite

[34:51] Valuing AI beyond engineering

[37:04] Crawl, walk, run: where they are today

[38:20] Why cost per million tokens is meaningless

[39:26] Closing thoughts


Hosted by Demetrios, MLOps.community is a space for honest, meandering talks about the real work of making artificial intelligence systems actually work. This isn't about hype or theoretical papers; it's about the messy, practical, and often surprising journey of taking models from a notebook into a live environment. You'll hear from engineers and practitioners who are in the trenches, discussing the tools, the frustrations, and the occasional breakthroughs that define the day-to-day. The conversations are deliberately relaxed, covering everything from traditional machine learning pipelines to the new world of large language models and even the intangible "vibes" of team culture and process. Each episode peels back a layer on what "production" really means, whether that involves deploying a predictive service, managing an agentic system, or maintaining reliability as everything scales. Tuning into this podcast feels like grabbing a coffee with colleagues who aren't afraid to dig into the technical nitty-gritty while keeping the tone conversational and accessible. It's for anyone who builds, manages, or is just curious about the operational backbone that allows AI to deliver value, offering a grounded perspective often missing from the broader conversation.
Author: Language: en-us Episodes: 50

Agentic Conversations (formally mlops.community)
Podcast Episodes
What Happens When Every Developer Has 20 AI Agents? [not-audio_url] [/not-audio_url]

Duration: 34:37
In this episode, we're joined by Stephen O'Grady, Co-Founder and Principal Analyst at RedMonk, to explore one of the biggest shifts happening in software engineering: AI is making code dramatically cheaper to produce, bu…
AI Agents Should Be Treated Like Hackers [not-audio_url] [/not-audio_url]

Duration: 31:35
In this episode, we're joined by Matt DeBergalis, CTO and Co-Founder of Apollo GraphQL, to explore what happens when AI agents start interacting with enterprise systems that were never designed for them.We dive into the…
Developers May Stop Depending on Libraries [not-audio_url] [/not-audio_url]

Duration: 46:52
In this episode of Agentic Conversations, we're joined by Shaun Smith, software engineer, open source advocate, and contributor at Hugging Face, to explore how AI coding has changed almost overnight.We dive into reinforc…
10 Cities. 4 Countries. One Unexpected MCP Lesson. [not-audio_url] [/not-audio_url]

Duration: 22:18
In this episode, we're joined by Ben Morss, Developer Advocate at DeepL, who spent months traveling across North America and Europe teaching developers about MCP, building MCP servers, and helping teams understand how AI…
The Next Programming Language Is English [not-audio_url] [/not-audio_url]

Duration: 38:13
In this episode, we're joined by Cornelia Davis, Developer Advocate at Temporal and a longtime software architect who has spent decades helping shape modern cloud-native systems.We explore how programming has evolved fro…
Omnigent: Composition, Control, and Collaboration for AI Agents [not-audio_url] [/not-audio_url]

Duration: 58:16
Denny Lee is PM Director, Startups & Ecosystem at Databricks, a longtime Apache Spark, MLflow, and Delta Lake contributor — and one of the people behind Omnigent, the open-source meta-harness Databricks just released und…
The Current State of Agentic Retrieval - Qdrant Roundtable [not-audio_url] [/not-audio_url]

Duration: 58:54
Qdrant Roundtable episode: The Current State of Agentic RetrievalJoin the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/YTNewsletterMLOps GPU Guide: https://go.mlops.communi…
AI Agents in Healthcare? [not-audio_url] [/not-audio_url]

Duration: 39:28
Kingsley Madikaegbu is the founder of HealID, a startup building agentic AI on top of the Model Context Protocol (MCP) for one of the most heavily regulated environments there is: healthcare.Recorded at MCP Dev Summit No…
Coding Agents Are Secretly General Agents [not-audio_url] [/not-audio_url]

Duration: 1:12:02
In this episode:🧠 Coding agents are generalist agents — why "positive transfer" means an agent that's better at code is better at everything, and how that makes them "AGI-complete"⏳ "Code will be solved in a year" — what…
The Dark Side of MCP Servers [not-audio_url] [/not-audio_url]

Duration: 1:09:59
Sam Partee (CTO & co-founder of Arcade.dev) and Nate Barbettini (Founding Engineer at Arcade.dev) sit down at the MCP Dev Summit to unpack what nobody wants to admit about the Model Context Protocol: the security model i…