Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber

Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber

Author: Demetrios November 21, 2025 Duration: 44:55

Jeff Huber is the CEO of ​Chroma, working on context engineering and building reliable retrieval infrastructure for AI systems.


Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber // MLOps Podcast #348.


Join the Community:

https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter


// Abstract

Jeff Huber drops some hard truths about “context rot” — the slow decay of AI memory that’s quietly breaking your favorite models. From retrieval chaos to the hidden limits of context windows, he and Demetrios Brinkmann unpack why most AI systems forget what matters and how Chroma is rethinking the entire retrieval stack. It’s a bold look at whether smarter AI means cleaner context — or just better ways to hide the mess.


// Bio

Jeff Huber is the CEO and cofounder of Chroma. Chroma has raised $20M from top investors in Silicon Valley and builds modern search infrastructure for AI.


// Related Links

Website: https://www.trychroma.com/


~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Jeff on LinkedIn: /jeffchuber/


Timestamps:

[00:00] AI intelligence context clarity

[00:37] Context rot explanation

[03:02] Benchmarking context windows

[05:09] Breaking down search eras

[10:50] Agent task memory issues

[17:21] Semantic search limitations

[22:54] Context hygiene in AI

[30:15] Chroma on-device functionality

[38:23] Vision for precision systems

[43:07] ML model deployment challenges

[44:17] Wrap up


Hosted by Demetrios, MLOps.community is a space for honest, meandering talks about the real work of making artificial intelligence systems actually work. This isn't about hype or theoretical papers; it's about the messy, practical, and often surprising journey of taking models from a notebook into a live environment. You'll hear from engineers and practitioners who are in the trenches, discussing the tools, the frustrations, and the occasional breakthroughs that define the day-to-day. The conversations are deliberately relaxed, covering everything from traditional machine learning pipelines to the new world of large language models and even the intangible "vibes" of team culture and process. Each episode peels back a layer on what "production" really means, whether that involves deploying a predictive service, managing an agentic system, or maintaining reliability as everything scales. Tuning into this podcast feels like grabbing a coffee with colleagues who aren't afraid to dig into the technical nitty-gritty while keeping the tone conversational and accessible. It's for anyone who builds, manages, or is just curious about the operational backbone that allows AI to deliver value, offering a grounded perspective often missing from the broader conversation.
Author: Language: en-us Episodes: 100

MLOps.community
Podcast Episodes
A Candid Conversation with the CEO of Stack Overflow [not-audio_url] [/not-audio_url]

Duration: 32:50
AI Conversations Powered by Prosus Group Stack Overflow is adapting to the AI era by licensing its trusted Q&A corpus, expanding into discussions and enterprise tools, and reinforcing its role as a reliable source as dev…
Knowledge is Eventually Consistent // Devin Stein // #335 [not-audio_url] [/not-audio_url]

Duration: 55:14
Knowledge is Eventually Consistent // MLOps Podcast #335 with Devin Stein, CEO of Dosu.Grateful to @Databricks and @hyperbolic-labs for supporting our podcast and helping us keep great conversations going.Join the Commun…
LinkedIn Recommender System Predictive ML vs LLMs [not-audio_url] [/not-audio_url]

Duration: 47:39
Demetrios chats with Arpita Vats about how LLMs are shaking up recommender systems. Instead of relying on hand-crafted features and rigid user clusters, LLMs can read between the lines—spotting patterns in user behavior…
The Hidden Bottlenecks Slowing Down AI Agents [not-audio_url] [/not-audio_url]

Duration: 47:59
Demetrios chats with Paul van der Boor and Bruce Martens from Prosus about the real bottlenecks in AI agent development—not tools, but evaluation and feedback. They unpack when to build vs. buy, the tradeoffs of external…
9 Commandments for Building AI Agents [not-audio_url] [/not-audio_url]

Duration: 1:20:33
Building AI agents that actually get things done is harder than it looks. Demetrios, Paul, and Dmitri break down what makes agents effective—from smart planning and memory to treating tools, systems, and even people as c…
Enterprise AI Adoption Challenges [not-audio_url] [/not-audio_url]

Duration: 1:05:00
Building AI Agents that work is no small feat.In Agents in Production [Podcast Limited Series] - Episode Six, Paul van der Boor and Sean Kenny share how they scaled AI across 100+ companies with Toqan—a tool born from a…
Real-time Feature Generation at Lyft // Rakesh Kumar // #334 [not-audio_url] [/not-audio_url]

Duration: 58:04
Real-time Feature Generation at Lyft // MLOps Podcast #334 with Rakesh Kumar, Senior Staff Software Engineer at Lyft.Join the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/Y…
AI Agent Development Tradeoffs You NEED to Know [not-audio_url] [/not-audio_url]

Duration: 57:06
Sherwood Callaway, tech lead at 11X, joins us to talk about building digital workers—specifically Alice (an AI sales rep) and Julian (a voice agent)—that are shaking up sales outreach by automating complex, messy tasks.H…
From the Legal Trenches to Tech // Nick Coleman // #332 [not-audio_url] [/not-audio_url]

Duration: 35:51
From the Legal Trenches to Tech // MLOps Podcast #332 with Nick Coleman, Attorney/Founder of LexMed.Join the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/YTNewsletter// Abs…