Measuring LLMs with Jodie Burchell

Measuring LLMs with Jodie Burchell

Author: Carl Franklin and Richard Campbell April 3, 2025 Duration: 1:00:44
How do you measure the quality of a large language model? Carl and Richard talk to Dr. Jodie Burchell about her work measuring large language models for accuracy, reliability, and consistency. Jodie talks about the variety of benchmarks that exist for LLMs and the problems they have. A broader conversation about quality digs into the idea that LLMs should be targeted to the particular topic area they are being used for - often, smaller is better! Building a good test suite for your LLM is challenging but can increase your confidence that the tool will work as expected.

Hosted by Carl Franklin and Richard Campbell, .NET Rocks! is a long-running conversation with the people building the future of software. This isn't a dry lecture; it's a lively, technical deep dive where two seasoned developers explore the vast ecosystem around Microsoft .NET, Azure, and modern development practices with a diverse roster of expert guests. Each episode feels like you're pulling up a chair in a room full of brilliant minds, listening to unfiltered discussions about real-world coding challenges, architectural patterns, and the tools that shape our daily work. You'll hear practical advice, war stories from the trenches, and forward-looking insights that go far beyond the documentation. Tuning into this podcast means connecting with a community of professionals who are as passionate about the craft as you are, offering perspectives that can transform how you approach your next project. Whether you're deep into C# or just curious about cloud-native development, these conversations provide a valuable blend of knowledge, humor, and genuine enthusiasm for technology.
Author: Language: English Episodes: 1000

.NET Rocks!
Podcast Episodes
Node in the 2020s with Rob Richardson [not-audio_url] [/not-audio_url]

Duration: 55:43
It's 2021 - how has NodeJS evolved? Carl and Richard talk to Rob Richardson about NodeJS and much more. Rob dives into the current evolutionary process of JavaScript with the ECMAScript standards and the impact of transp…
CUPID with Dan North [not-audio_url] [/not-audio_url]

Duration: 1:16:56
Do the SOLID principles still make sense? Carl and Richard talk to Dan North about SOLID, starting with a five-minute PubConf talk that Dan did about how SOLID was wrong. Meant to be humorous (it was PubConf after all),…
Open Source in the Enterprise with Rocky Lhotka [not-audio_url] [/not-audio_url]

Duration: 1:03:46
Are you using open-source libraries in your enterprise applications or products? What are the consequences? Carl and Richard talk to Rocky Lhotka about his work with organizations coming to grips with the use of open-sou…
Event Sourcing with Jeremy Miller [not-audio_url] [/not-audio_url]

Duration: 56:36
What is event sourcing, and why should you care? Carl and Richard talk to Jeremy Miller about the latest version of Marten, the Document store for PostgreSQL - and how there is an increased focus in this version of event…
Debugging Ransomware and Other Stories with Paula Januszkiewicz [not-audio_url] [/not-audio_url]

Duration: 58:15
Ransomware is serious - are you protected? Carl and Richard talk to Paula Januszkiewicz as part of the Techorama online event about her work fighting ransomware. Paula talks about how successful the bad guys have been th…
Build 2021 Announcements with Scott Hunter [not-audio_url] [/not-audio_url]

Duration: 57:00
What's being announced at Build? Carl and Richard talk to Scott Hunter about the latest from the Virtual Build conference. With .NET 5 out last November, and .NET 6 coming next November, this is a time for interim featur…
Oracle for Startups with Jason Williamson [not-audio_url] [/not-audio_url]

Duration: 48:59
Oracle for Startups? Carl and Richard talk to Jason Williamson about Oracle's programs to support startups using Oracle's Cloud Infrastructure (OCI). Jason talks about how startups can get credit on OCI, migrate workload…
Visual Studio 2022 with Simon, Anthony and Andy [not-audio_url] [/not-audio_url]

Duration: 56:19
What's new in Visual Studio 2022? Carl and Richard talk to Simon Calvert, Anthony Cangialosi, and Andy Sterland about the upcoming version of Studio. First on the docket is 64-bit: While Studio has been able to build 64-…
When Investors Come Calling with Paul Stovell [not-audio_url] [/not-audio_url]

Duration: 52:42
What happens when your company brings on investors? Carl and Richard talk to Paul Stovell about the recent investment made by Insight Partners into Octopus Deploy. Paul talks about how a great investment group brings imp…
Smarter Testing using Verify with Simon Cropp [not-audio_url] [/not-audio_url]

Duration: 58:42
How do you do smarter testing? Carl and Richard talk to Simon Cropp about Verify, Simon's open source project to add snapshot testing capabilities to many of your favorite testing stacks. Simon talks about how snapshots…