Docker Model Runner

Docker Model Runner

Author: Bret Fisher April 21, 2025 Duration: 13:06

Docker launched "Docker Model Runner" to run LLMs through llama.cpp with a single "docker model" command. In this episode Bret details examples and some useful use cases for using this way to run LLMs. He breaks down the internals. How it works, when you should use it or not use it; and, how to get started using Open WebUI for a private ChatGPT-like experience.

😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-on labs to harden your repos and your workflows from common supply chain attacks. I'll cover how attackers are getting in, and then we'll lock down a sample repo so you know what needs to be done to protect your code. You'll leave with a deep understanding of risks and mitigations as well as a list of helpful tools to keep your repos safe, including my new "gasa" tool for scanning your repos and orgs.


★Topics★
Model Runner Docs
Hub Models
OCI Artifacts
Open WebUI
My Open WebUI Compose file

Creators & Guests

  • (00:00) - Intro
  • (00:46) - Model Runner Elevator Pitch
  • (01:28) - Enabling Docker Model Runner
  • (04:28) - Self Promotion! Is that an ad? For me?
  • (05:03) - Downloading Models
  • (07:11) - Architectrure of Model Runner
  • (10:49) - ORAS
  • (11:09) - What's next for Model Runner?
  • (12:13) - Troubleshooting

You can also support this podcast by subscribing to my YouTube channel and my monthly newsletter at bret.news!

Grab the best coupons for my GitHub, Agents, Docker, and Kubernetes courses.
Join my cloud native DevOps community on Discord.
Grab some merch at Bret's Loot Box
Homepage bretfisher.com

★ Support this podcast ★

Bret Fisher hosts DevOps and Docker Talk: Cloud Native Interviews and Tooling alongside Nirmal Mehta, pulling conversations directly from their live audience sessions. This isn't a theoretical lecture series; it's grounded in the practical, often messy realities of building and running modern systems. Each episode digs into the specific tools and philosophies shaping infrastructure today, from the foundational elements like Docker and container orchestration with Kubernetes or Swarm to the broader cultural shifts of DevOps, SRE, and platform engineering. You'll hear discussions that span the entire software lifecycle, touching on the integration of security with DevSecOps, the automation promises of GitOps, and what "Cloud Native" truly means for development teams. The dialogue stays focused on actionable insights and real-world use cases, offering a genuine sense of what works and what doesn't in the field. For anyone navigating the container and cloud ecosystem, this podcast serves as a regular, insightful check-in with practitioners and experts, hosted by someone deeply embedded in the community. The natural, interview-driven format makes complex topics accessible, ensuring listeners come away with more than just buzzwords-they get context and clarity.
Author: Language: English Episodes: 50

DevOps and Docker Talk: Cloud Native Interviews and Tooling
Podcast Episodes
Aikido: Is a Single DevSecOps Tool Possible? [not-audio_url] [/not-audio_url]

Duration: 1:01:56
Or watch the video version on YouTube. Bret is joined by Willem Delbare and Roeland Delrue to discuss Aikido, a security tool consolidation platform designed specifically for smaller teams and solo DevOps practitioners.…
DevOps Cert Prep using AI, with the KodeKloud Team [not-audio_url] [/not-audio_url]

Duration: 1:12:23
Bret is joined by Mumshad Mannambeth and Vijin Palazhi of KodeKloud for Q&A on what we should be studying and certifying for in 2025. 😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-on l…
KubeCon Engineering Takeaways [not-audio_url] [/not-audio_url]

Duration: 37:15
Bret and Nirmal recorded this special offline episode at KubeCon North America in Salt Lake City. We hung out at the AWS booth to break down the major trends and developments from the conference. 😇 My new GitHub Security…
Personal AI with Ken Collins [not-audio_url] [/not-audio_url]

Duration: 57:58
Bret and Nirmal Mehta are joined by Ken Collins to dig into using AI for more than coding, and if we can build an AI assistant that knows us.😇 My new GitHub Security workshop has launched! A free 2-hour workshop with han…
State of Kubernetes UIs [not-audio_url] [/not-audio_url]

Duration: 17:08
Bret explores the spectrum of user interfaces and tools available for managing Kubernetes clusters as of Autumn 2024. 😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-on labs to harden yo…
Inspektor Gadget [not-audio_url] [/not-audio_url]

Duration: 40:19
Bret and Nirmal are joined by Chris Kühl and Jose Blanquicet, the maintainers of Inspektor Gadget, the new eBPF-focused multitool, to see what it's all about.😇 My new GitHub Security workshop has launched! A free 2-hour…
MLOps for DevOps People [not-audio_url] [/not-audio_url]

Duration: 48:29
Bret and Nirmal are joined by Maria Vechtomova, a MLOps Tech Lead and co-founder of Marvelous MLOps, to discuss the obvious and not-so obvious differences between a MLOps Engineer and traditional DevOps jobs.😇 My new Git…
Traefik 3.0: What's New? [not-audio_url] [/not-audio_url]

Duration: 54:15
Bret and Nirmal were joined by Emile Vauge, CTO of Traefik Labs to talk all about Traefik 3.0.😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-on labs to harden your repos and your workfl…
Debug Containers with Mintoolkit [not-audio_url] [/not-audio_url]

Duration: 1:15:34
Bret is joined by DockerSlim (now mintoolkit) founder Kyle Quest, to show off how to slim down your existing images with various options. 😇 My new GitHub Security workshop has launched! A free 2-hour workshop with hands-…
Observability Cost-Savings and eBPF Goodness with Groundcover [not-audio_url] [/not-audio_url]

Duration: 55:42
Bret is joined by Shahar Azulay, Groundcover CEO and Co-Founder, to discuss their new approach to fully observe K8s and its workloads with a "hybrid observability architecture."😇 My new GitHub Security workshop has launc…