Making AI Reliable is the Greatest Challenge of the 2020s // Alon Bochman // #312

Making AI Reliable is the Greatest Challenge of the 2020s // Alon Bochman // #312

Author: Demetrios May 6, 2025 Duration: 1:01:37

Making AI Reliable is the Greatest Challenge of the 2020s // MLOps Podcast #312 with Alon Bochman, CEO of RagMetrics.


Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter


Huge shout-out to RagMetrics for sponsoring this episode!


// Abstract

Demetrios talks with Alon Bochman, CEO of RagMetrics, about testing in machine learning systems. Alon stresses the value of empirical evaluation over influencer advice, highlights the need for evolving benchmarks, and shares how to effectively involve subject matter experts without technical barriers. They also discuss using LLMs as judges and measuring their alignment with human evaluators.


// Bio

Alon is a product leader with a fintech and adtech background, ex-Google, ex-Microsoft. Co-founded and sold a software company to Thomson Reuters for $30M, grew an AI consulting practice from 0 to over $ 1 Bn in 4 years. 20-year AI veteran, winner of three medals in model-building competitions. In a prior life, he was a top-performing hedge fund portfolio manager. Alon lives near NYC with his wife and two daughters. He is an avid reader, runner, and tennis player, an amateur piano player, and a retired chess player.


// Related Links

Website: ragmetrics.ai


~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]


Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Alon on LinkedIn: /alonbochman



Timestamps:

[00:00] Alon's preferred coffee

[00:15] Takeaways

[00:47] Testing Multi-Agent Systems

[05:55] Tracking ML Experiments

[12:28] AI Eval Redundancy Balance

[17:07] Handcrafted vs LLM Eval Tradeoffs

[28:15] LLM Judging Mechanisms

[36:03] AI and Human Judgment

[38:55] Document Evaluation with LLM

[42:08] Subject Matter Expertise in Co-Pilots

[46:33] LLMs as Judges

[51:40] LLM Evaluation Best Practices

[55:26] LM Judge Evaluation Criteria

[58:15] Visualizing AI Outputs

[1:01:16] Wrap up


Hosted by Demetrios, MLOps.community is a space for honest, meandering talks about the real work of making artificial intelligence systems actually work. This isn't about hype or theoretical papers; it's about the messy, practical, and often surprising journey of taking models from a notebook into a live environment. You'll hear from engineers and practitioners who are in the trenches, discussing the tools, the frustrations, and the occasional breakthroughs that define the day-to-day. The conversations are deliberately relaxed, covering everything from traditional machine learning pipelines to the new world of large language models and even the intangible "vibes" of team culture and process. Each episode peels back a layer on what "production" really means, whether that involves deploying a predictive service, managing an agentic system, or maintaining reliability as everything scales. Tuning into this podcast feels like grabbing a coffee with colleagues who aren't afraid to dig into the technical nitty-gritty while keeping the tone conversational and accessible. It's for anyone who builds, manages, or is just curious about the operational backbone that allows AI to deliver value, offering a grounded perspective often missing from the broader conversation.
Author: Language: en-us Episodes: 100

MLOps.community
Podcast Episodes
Rethinking Notebooks Powered by AI [not-audio_url] [/not-audio_url]

Duration: 26:13
Vincent Warmerdam is a Founding Engineer at marimo, working on reinventing Python notebooks as reactive, reproducible, interactive, and Git-friendly environments for data workflows and AI prototyping. He helps build the…
Physical AI: Teaching Machines to Understand the Real World [not-audio_url] [/not-audio_url]

Duration: 52:03
Nick Gillian is the Co-Founder and CTO at Archetype AI, working on physical AI foundation models that understand and reason over real-world sensor data.Physical AI: Teaching Machines to Understand the Real World // MLOps…
A Playground for AI/ML Engineers [not-audio_url] [/not-audio_url]

Duration: 54:41
Paulo Vasconcellos is the Principal Data Scientist for Generative AI Products at Hotmart, working on AI-powered creator and learning experiences, including intelligent tutoring, content automation, and multilingual local…
Conversation with the MLflow Maintainers [not-audio_url] [/not-audio_url]

Duration: 58:23
Corey Zumar is a Product Manager at Databricks, working on MLflow and LLM evaluation, tracing, and lifecycle tooling for generative AI.Jules Damji is a Lead Developer Advocate at Databricks, working on Spark, lakehouse t…
Leadership on AI [not-audio_url] [/not-audio_url]

Duration: 47:24
Euro Beinat is the Global Head of AI and Data Science at Prosus Group, working on scaling AI-driven tools and agent-based systems across Prosus’s global portfolio, deploying internal assistants like Toqan and generative…
Computers that Think and Take Actions for You [not-audio_url] [/not-audio_url]

Duration: 45:08
Zengyi Qin is the Founder of the OpenAGI Foundation, working on computer-use models and open, agent-centric AI infrastructure.Computers that Think and Take Actions for You, Zengy Qin // MLOps Podcast #355Join the Communi…