Measuring LLMs with Jodie Burchell

Measuring LLMs with Jodie Burchell

Author: Carl Franklin and Richard Campbell April 3, 2025 Duration: 1:00:44
How do you measure the quality of a large language model? Carl and Richard talk to Dr. Jodie Burchell about her work measuring large language models for accuracy, reliability, and consistency. Jodie talks about the variety of benchmarks that exist for LLMs and the problems they have. A broader conversation about quality digs into the idea that LLMs should be targeted to the particular topic area they are being used for - often, smaller is better! Building a good test suite for your LLM is challenging but can increase your confidence that the tool will work as expected.

Hosted by Carl Franklin and Richard Campbell, .NET Rocks! is a long-running conversation with the people building the future of software. This isn't a dry lecture; it's a lively, technical deep dive where two seasoned developers explore the vast ecosystem around Microsoft .NET, Azure, and modern development practices with a diverse roster of expert guests. Each episode feels like you're pulling up a chair in a room full of brilliant minds, listening to unfiltered discussions about real-world coding challenges, architectural patterns, and the tools that shape our daily work. You'll hear practical advice, war stories from the trenches, and forward-looking insights that go far beyond the documentation. Tuning into this podcast means connecting with a community of professionals who are as passionate about the craft as you are, offering perspectives that can transform how you approach your next project. Whether you're deep into C# or just curious about cloud-native development, these conversations provide a valuable blend of knowledge, humor, and genuine enthusiasm for technology.
Author: Language: English Episodes: 1000

.NET Rocks!
Podcast Episodes
Angular 2 Docs with Ward Bell [not-audio_url] [/not-audio_url]

Duration: 56:43
How can you be successful with a product without good documentation? You can't! Carl and Richard talk to Ward Bell, who is serving as editor in chief for Angular docs. After complaining about the quality problems with th…
Xamarin Update with James Montemagno [not-audio_url] [/not-audio_url]

Duration: 59:09
Time for a Xamarin update - things are moving fast! Carl and Richard talk to James Montemagno, now a Microsoft employee since the Xamarin acquisition, about the on-going evolution of the Xamarin tools for building mobile…
SpaceX Interplanetary Transport System Geek Out [not-audio_url] [/not-audio_url]

Duration: 58:37
On September 27, 2016, Elon Musk held a press conference that was more like a rock concert to an excited crowd at the International Astronautical Congress in Guadalajara, Mexico. At the event, he announced the Interplane…
Migrating Legacy Apps to Docker with Elton Stoneman [not-audio_url] [/not-audio_url]

Duration: 51:38
What does it take to move an existing application to Docker? Carl and Richard talk to Elton Stoneman about his experiences migrating applications to Docker. The power of containers is obvious, with the ability to run com…
Serverless Architecture with Ben Godwin [not-audio_url] [/not-audio_url]

Duration: 52:10
Serverless is the new hot buzzword - but what does it really mean? Carl and Richard talk to Ben Godwin about his work building serverless applications - no servers, but lots of services! Ben talks about Amazon Lambda, wh…
Growing a .NET Meetup Group with Blake Helms and Robb Schiefer [not-audio_url] [/not-audio_url]

Duration: 53:38
Are user groups obsolete? Carl and Richard talk to Blake Helms and Robb Schiefer about their experiences starting and growing a .NET Meetup Group in Birmingham, Alabama. Modernizing on the user group with Meetup doesn't…
Distributed Caching with Iqbal Khan [not-audio_url] [/not-audio_url]

Duration: 50:28
What role does distributed caching play in applications today? Carl and Richard sit down with Iqbal Khan to talk about nCache, an open source product built to do distributed caching in the .NET world. The conversation st…
Thinking Voice Control with Austin Dimmer [not-audio_url] [/not-audio_url]

Duration: 58:10
Has voice control come of age? Carl and Richard talk to Austin Dimmer about his efforts to build a great voice control system - including for Visual Studio! The conversation digs into the complexity of recognizing a dive…
PHP using PeachPie with Benjamin Fistein and Jakub Míšek [not-audio_url] [/not-audio_url]

Duration: 47:22
Compiled PHP on .NET! Carl and Richard talk to Benjamin Fistein and Jakub Míšek about Peachpie, and open source project to implement PHP on the .NET Core. While the project isn't complete yet (you can help - it's open so…