Measuring LLMs with Jodie Burchell

Measuring LLMs with Jodie Burchell

Author: Carl Franklin and Richard Campbell April 3, 2025 Duration: 1:00:44
How do you measure the quality of a large language model? Carl and Richard talk to Dr. Jodie Burchell about her work measuring large language models for accuracy, reliability, and consistency. Jodie talks about the variety of benchmarks that exist for LLMs and the problems they have. A broader conversation about quality digs into the idea that LLMs should be targeted to the particular topic area they are being used for - often, smaller is better! Building a good test suite for your LLM is challenging but can increase your confidence that the tool will work as expected.

Hosted by Carl Franklin and Richard Campbell, .NET Rocks! is a long-running conversation with the people building the future of software. This isn't a dry lecture; it's a lively, technical deep dive where two seasoned developers explore the vast ecosystem around Microsoft .NET, Azure, and modern development practices with a diverse roster of expert guests. Each episode feels like you're pulling up a chair in a room full of brilliant minds, listening to unfiltered discussions about real-world coding challenges, architectural patterns, and the tools that shape our daily work. You'll hear practical advice, war stories from the trenches, and forward-looking insights that go far beyond the documentation. Tuning into this podcast means connecting with a community of professionals who are as passionate about the craft as you are, offering perspectives that can transform how you approach your next project. Whether you're deep into C# or just curious about cloud-native development, these conversations provide a valuable blend of knowledge, humor, and genuine enthusiasm for technology.
Author: Language: English Episodes: 1000

.NET Rocks!
Podcast Episodes
Hacking, SQL Injection, Ransomware and More with Troy Hunt [not-audio_url] [/not-audio_url]

Duration: 58:11
That scary guy is back! Carl and Richard talk to Troy Hunt about the latest state of affairs in the hacking world. Yes, SQL Injection is still a thing, and the hacks are actually getting bigger - entire voting population…
Thinking Android with Joshua Vergara [not-audio_url] [/not-audio_url]

Duration: 59:36
How do you think about Android? Carl and Richard talk to Josh Vergara, Android-fan, non-developer and head of Android Authority about his experiences around Android phones and tablets. Josh talks about the various flavor…
The Evolution of Services with Juval Lowy [not-audio_url] [/not-audio_url]

Duration: 57:11
So is every class a service? While at DevIntersection in Orlando, Carl and Richard talk to Juval Lowy about how his statement nearly ten years ago has in some ways come true. Juval talks about how services evolved back i…
Octopus 3 with Damian Brady [not-audio_url] [/not-audio_url]

Duration: 1:00:51
How do you deploy your applications? While at DevIntersection, Carl and Richard chatted with Damian Brady from Octopus about the latest version of Octopus Deploy. Damian talks about all the changes that have come in Octo…
Talking Core with Scott Hunter [not-audio_url] [/not-audio_url]

Duration: 1:02:09
Scott Hunter is back and managing the whole .NET platform! While at DevIntersection in Orlando, Carl and Richard sat down with Scott to talk about his new role as director of the entire .NET platform. That includes all t…
Mobile DevOps Pipeline with Donovan Brown [not-audio_url] [/not-audio_url]

Duration: 51:55
How do you manage the building, monitoring and maintenance of mobile apps? Carl and Richard talk to Donovan Brown about how all the pieces have come together in the Microsoft stack to make creating, testing, deploying, m…
Universal Apps on XBox One with Chris Gomez [not-audio_url] [/not-audio_url]

Duration: 56:07
Universal Apps are becoming more universal - arriving on the XBox One! Carl and Richard talk to Chris Gomez about the announcements at the Microsoft Build event around building software for the XBox One. Now, any develop…
Fixing the Web with Douglas Crockford [not-audio_url] [/not-audio_url]

Duration: 50:44
The Web is broken - time to fix it! While at DevIntersection in Orlando, Carl and Richard sat down with Douglas Crockford to talk about the problems the web has and what can be done about them. Doug rightfully focuses on…
InfoSec for Developers with Kim Carter [not-audio_url] [/not-audio_url]

Duration: 55:43
What do developers need to know about information security? Carl and Richard talk to Kim Carter about his experiences helping developers secure their web sites. Kim has written a series of books on the subject to help ge…
Supersonic Aircraft Geek Out [not-audio_url] [/not-audio_url]

Duration: 1:05:03
Concorde is gone, what will replace it? Time for a Geek Out! Richard talks about the aeronautical evolution that led to supersonic airliners, Concorde being the big one that flew from 1976 to 2003. What went wrong? Why d…