SE Radio 703: Sahaj Garg on Low Latency AI

SE Radio 703: Sahaj Garg on Low Latency AI

Author: team@se-radio.net (SE-Radio Team) January 14, 2026 Duration: 54:47

In this episode, Sahaj Garg, CTO of wispr.ai, joins SE Radio host Robert Blumen to talk about the challenges of building low-latency AI applications. They discuss latency's effect on consumer behavior as well as interactive applications. The conversation explores how to measure latency and how scale impacts it. Then Sahaj and Robert shift to themes around AI, including whether "AI" means LLMs or something broader, as they look at latency requirements and challenges around subtypes of AI applications. The final part of the episode explores techniques for managing latency in AI: speed vs accuracy trade-offs; speed vs cost; latency vs cost; choosing the right model; reducing quantization; distillation; and guessing + validating.

Brought to you by IEEE Computer Society and IEEE Software magazine.


For developers who build the world's most critical systems, Software Engineering Radio offers deep, substantive conversations that move beyond the hype cycle. This isn't about quick tips or news flashes; it's a dedicated audio library for career engineers seeking to solidify their foundational knowledge and explore advanced concepts. Each episode is crafted as an enduring resource, featuring either a comprehensive tutorial breaking down a specific technology or methodology, or a detailed interview with a leading practitioner shaping the field. You'll hear focused discussions on everything from low-level systems architecture and programming language design to team dynamics and project management, all through the lens of professional software creation. The content is exclusively produced for this podcast, ensuring thoughtful, in-depth analysis you won't find simply repackaged from conference talks. If your work demands a rigorous understanding of the craft, this is the podcast for you.
Author: Language: en-us Episodes: 100

Software Engineering Radio - the podcast for professional software developers
Podcast Episodes
SE Radio 666: Eran Yahav on the Tabnine AI Coding Assistant [not-audio_url] [/not-audio_url]

Duration: 1:02:05
Eran Yahav, Professor of Computer Science at Technion, Israel, and CTO of Tabnine, speaks with host Gregory M. Kapfhammer about the Tabnine AI coding assistant. They discuss how the design and implementation allows softw…
SE Radio 663: Tyler Flint on Managing External APIs [not-audio_url] [/not-audio_url]

Duration: 52:27
Tyler Flint, CEO of qpoint.io, joins host Robert Blumen for a conversation about managing external vendor dependencies, including several best practices for adoption. They start with a look at internal versus external se…
SE Radio 662: Vlad Khononov on Balancing Coupling in Software Design [not-audio_url] [/not-audio_url]

Duration: 56:19
Software architect and author Vlad Khononov joins host Jeff Doolittle for a discussion on balancing coupling in software design. They start by examining coupling and its relationship to complexity and modularity. Vlad ex…
SE Radio 661: Sunil Mallya on Small Language Models [not-audio_url] [/not-audio_url]

Duration: 59:28
Sunil Mallya, co-founder and CTO of Flip AI, discusses small language models with host Brijesh Ammanath. They begin by considering the technical distinctions between SLMs and large language models. LLMs excel in generati…
SE Radio 660: Pete Warden on TinyML [not-audio_url] [/not-audio_url]

Duration: 55:04
Pete Warden, CEO of Useful Sensors and a founding member of the TensorFlow team at Google, discusses TinyML, the technology enabling machine learning on low-power, small-footprint devices. This innovation opens up applic…
SE Radio 659: Brenden Matthews on Idiomatic Rust [not-audio_url] [/not-audio_url]

Duration: 53:54
Brenden Matthews, a seasoned software engineer, entrepreneur, and author of the Idiomatic Rust and Code Like a Pro in Rust books (both from Manning), speaks with SE Radio host Gavin Henry about Idiomatic Rust. They start…
SE Radio 658: Tanya Janca on Secure Coding [not-audio_url] [/not-audio_url]

Duration: 1:11:40
Tanya Janca, author of Alice and Bob Learn Secure Coding, discusses secure coding and secure software development life cycle with SE Radio host Brijesh Ammanath. This session explores how integrating security into every…
SE Radio 657: Hong Minhee on ActivityPub and the Fediverse [not-audio_url] [/not-audio_url]

Duration: 40:09
Hong Minhee, an open source developer and creator of the Fedify ActivityPub library, discusses the ActivityPub protocol and the fediverse with SE Radio's Jeremy Jung. They explore ActivityPub use cases, including microbl…