Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber

Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber

Author: Demetrios November 21, 2025 Duration: 44:55

Jeff Huber is the CEO of ​Chroma, working on context engineering and building reliable retrieval infrastructure for AI systems.


Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber // MLOps Podcast #348.


Join the Community:

https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter


// Abstract

Jeff Huber drops some hard truths about “context rot” — the slow decay of AI memory that’s quietly breaking your favorite models. From retrieval chaos to the hidden limits of context windows, he and Demetrios Brinkmann unpack why most AI systems forget what matters and how Chroma is rethinking the entire retrieval stack. It’s a bold look at whether smarter AI means cleaner context — or just better ways to hide the mess.


// Bio

Jeff Huber is the CEO and cofounder of Chroma. Chroma has raised $20M from top investors in Silicon Valley and builds modern search infrastructure for AI.


// Related Links

Website: https://www.trychroma.com/


~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Jeff on LinkedIn: /jeffchuber/


Timestamps:

[00:00] AI intelligence context clarity

[00:37] Context rot explanation

[03:02] Benchmarking context windows

[05:09] Breaking down search eras

[10:50] Agent task memory issues

[17:21] Semantic search limitations

[22:54] Context hygiene in AI

[30:15] Chroma on-device functionality

[38:23] Vision for precision systems

[43:07] ML model deployment challenges

[44:17] Wrap up


Hosted by Demetrios, MLOps.community is a space for honest, meandering talks about the real work of making artificial intelligence systems actually work. This isn't about hype or theoretical papers; it's about the messy, practical, and often surprising journey of taking models from a notebook into a live environment. You'll hear from engineers and practitioners who are in the trenches, discussing the tools, the frustrations, and the occasional breakthroughs that define the day-to-day. The conversations are deliberately relaxed, covering everything from traditional machine learning pipelines to the new world of large language models and even the intangible "vibes" of team culture and process. Each episode peels back a layer on what "production" really means, whether that involves deploying a predictive service, managing an agentic system, or maintaining reliability as everything scales. Tuning into this podcast feels like grabbing a coffee with colleagues who aren't afraid to dig into the technical nitty-gritty while keeping the tone conversational and accessible. It's for anyone who builds, manages, or is just curious about the operational backbone that allows AI to deliver value, offering a grounded perspective often missing from the broader conversation.
Author: Language: en-us Episodes: 100

MLOps.community
Podcast Episodes
Reliable Voice Agents [not-audio_url] [/not-audio_url]

Duration: 38:21
Brooke Hopkins is the CEO of Coval, a company making voice agents more reliable. Reliable Voice Agents // MLOps Podcast #347 with Brooke Hopkins, Founder of Coval.Join the Community: https://go.mlops.community/YTJoinInGe…
The Future of AI Operations: Insights from PwC AI Managed Services [not-audio_url] [/not-audio_url]

Duration: 41:27
Rani Radhakrishnan is a Principal at PwC US, leading work on AI-managed services, autonomous agents, and data-driven transformation for enterprises.The Future of AI Operations: Insights from PwC AI Managed Services // ML…
GPU Uptime with VAST Data CTO [not-audio_url] [/not-audio_url]

Duration: 1:33:45
Andy Pernsteiner is the Field CTO at VAST Data, working on large-scale AI infrastructure, serverless compute near data, and the rollout of VAST’s AI Operating System.The GPU Uptime Battle // MLOps Podcast #346 with Andy…
Thousands of Fine-Tuned Models [not-audio_url] [/not-audio_url]

Duration: 37:47
Jaipal Singh Goud is the CTO at Prem AI, working on model customization and privacy-preserving compute. This episode was recorded at the Plan B studios in Lugano, Switzerland. For more information, visit https://pow.spac…
The Semantic Layer and AI Agents // David Jayatillake // #343 [not-audio_url] [/not-audio_url]

Duration: 50:37
The Semantic Layer and AI Agents // MLOps Podcast #343 with David Jayatillake, VP of AI at Cube.dev.Join the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/YTNewsletter// Abs…
Building an Agentic AI Memory Framework [not-audio_url] [/not-audio_url]

Duration: 50:34
What if AI could actually remember like humans do?Biswaroop Bhattacharjee joins Demetrios Brinkmann to challenge how we think about memory in AI. From building Cortex—a system inspired by human cognition—to exploring whe…
Best AI Hackathon Project Ever? [Bite Size Episode] [not-audio_url] [/not-audio_url]

Duration: 9:08
AI Conversations Powered by Prosus Group Unicorn Mafia won the recent hackathon at Raise Summit and explained to me what they built, including all the tech they used under the hood to make their AI agents work. Winners:C…