Measuring LLMs with Jodie Burchell

Measuring LLMs with Jodie Burchell

Author: Carl Franklin and Richard Campbell April 3, 2025 Duration: 1:00:44
How do you measure the quality of a large language model? Carl and Richard talk to Dr. Jodie Burchell about her work measuring large language models for accuracy, reliability, and consistency. Jodie talks about the variety of benchmarks that exist for LLMs and the problems they have. A broader conversation about quality digs into the idea that LLMs should be targeted to the particular topic area they are being used for - often, smaller is better! Building a good test suite for your LLM is challenging but can increase your confidence that the tool will work as expected.

Hosted by Carl Franklin and Richard Campbell, .NET Rocks! is a long-running conversation with the people building the future of software. This isn't a dry lecture; it's a lively, technical deep dive where two seasoned developers explore the vast ecosystem around Microsoft .NET, Azure, and modern development practices with a diverse roster of expert guests. Each episode feels like you're pulling up a chair in a room full of brilliant minds, listening to unfiltered discussions about real-world coding challenges, architectural patterns, and the tools that shape our daily work. You'll hear practical advice, war stories from the trenches, and forward-looking insights that go far beyond the documentation. Tuning into this podcast means connecting with a community of professionals who are as passionate about the craft as you are, offering perspectives that can transform how you approach your next project. Whether you're deep into C# or just curious about cloud-native development, these conversations provide a valuable blend of knowledge, humor, and genuine enthusiasm for technology.
Author: Language: English Episodes: 1000

.NET Rocks!
Podcast Episodes
DevOps Readiness Assessment with Jez Humble and Nicole Forsgren [not-audio_url] [/not-audio_url]

Duration: 52:48
Where is your DevOps at? Carl and Richard talk to Jez Humble and Nicole Forsgren about DORA, that is the DevOps Readiness Assessment. DORA helps you understand where your organization is at in the spectrum of DevOps, fro…
Antibiotic Geek Out [not-audio_url] [/not-audio_url]

Duration: 1:05:37
Most of us have taken antibiotics at some point in our lives, but how do they work? Time for a Geek Out! Richard talks through some fundamental cell biology to be able to set the stage for what antibiotics are, where the…
C# 7 and Visual Studio 2017 with Kathleen Dollard [not-audio_url] [/not-audio_url]

Duration: 1:02:03
C# 7 and Visual Studio 2017 are shipped - now what? Carl and Richard talk to Kathleen Dollard about her experiences working with folks that are trying to dig into the latest version of Microsoft development tools. Kathle…
DevOps Pipelines with Donovan Brown [not-audio_url] [/not-audio_url]

Duration: 1:01:53
The whirlwind of DevOps is back! Carl and Richard talk to Donovan Brown about his on-going mission to get everyone into a DevOps practice - in fact, he talks about getting to a place where continuous delivery and monitor…
Multi-tenant Apps Using cloudscribe with Joe Audette [not-audio_url] [/not-audio_url]

Duration: 1:00:49
What does it take to build great multi-tenant apps on the cloud? Carl and Richard talk to Joe Audette about his set of open source tools called cloudscribe to make multi-tenant apps easier to implement. The conversation…
MVVM Cross with Martijn van Dijk [not-audio_url] [/not-audio_url]

Duration: 42:16
MVVMCross has grown up! Carl and Richard chat with Martjin van Dijk about his work on MVVMCross. Past guest Stuart Lodge started MVVMCross way back in 2012 when it was focused on building MVVM apps in Silverlight, but to…
VSTS Analytics with Steven Borg [not-audio_url] [/not-audio_url]

Duration: 59:38
What can analytics around development do for you? Carl and Richard talk to Steven Borg about his work with Visual Studio Team Services (VSTS), especially around understanding developer productivity. The conversation star…
Chaiscript with Jason Turner [not-audio_url] [/not-audio_url]

Duration: 49:46
Scripting with C++? Of course! Carl and Richard talk to Jason Turner about ChaiScript, his open source project that simplifies adding C++ as a scripting language to any application, but especially other C++ applications.…
Visual Studio for Mac with Mikayla Hutchinson [not-audio_url] [/not-audio_url]

Duration: 44:33
Visual Studio on a Mac! Carl and Richard talk to Mikayla Hutchinson who's been with Xamarin since the very beginning, about her work on the preview of Visual Studio for Mac. The product comes from MonoDevelop, which is X…