AI Agents and Long Context Windows with Mark Huang

AI Agents and Long Context Windows with Mark Huang

Author: Software Huddle June 18, 2024 Duration: 50:24
Today we have Mark Huang on the show. Mark has previously held roles in Data Science and ML at companies like Box and Splunk and is now the co-founder and chief architect of Gradient, an enterprise AI platform to build and deploy autonomous assistants. In our chat, we get into some of the stuff he’s seeing around autonomous AI agents and why people are so excited about that space. Mark and his team has also recently been working on a project to extend the Llama-3 context window. They were able to extend the model from 8K tokens all the way to 1 million through a technique called theta-scaling. He walks us through the details of this project and how longer context windows will impact the types of use cases we can serve with LLMs. Follow Mark: https://x.com/markatgradient Follow Sean: https://x.com/seanfalconer

Every week on Software Huddle, Alex DeBrie and Sean Falconer sit down with a different expert from across the tech landscape. The conversations are less about quick tips and more about substantive discussions, digging into the real challenges and decisions behind building software, launching products, and navigating the industry's constant shifts. You'll hear from practitioners who have been in the trenches, offering perspectives that blend deep technical knowledge with hard-won business and entrepreneurial experience. Alex brings his specialized expertise as the author of The DynamoDB Book and an AWS Data Hero, while Sean contributes a unique viewpoint shaped by over two decades as an engineer, founder, and marketing executive, recognized as a Snowflake Data Superhero. Together, they create a space where complex topics in software development and technology trends become accessible and genuinely engaging. This podcast is for anyone who wants to move beyond surface-level news and understand the "why" behind the tools and strategies shaping our digital world. Tune in for a thoughtful huddle that feels more like a candid conversation between colleagues than a formal interview.
Author: Language: en-us Episodes: 79

Software Huddle
Podcast Episodes
The Real Work of Data Engineering with Joe Reis [not-audio_url] [/not-audio_url]

Duration: 59:00
Today, we have Joe Reis on the show. Joe is the co author of the book, Fundamentals of Data Engineering, probably the best and most comprehensive book on data engineering you could think to read. We talk about the cultur…
Tech layoffs, Sora by OpenAI, Gemini 1.5, Apple Vision Pro & more [not-audio_url] [/not-audio_url]

Duration: 1:01:12
Our special episode is back! Join Sean, Alex & Vino in this fun conversation. 00:00 Introduction 10:08 Sora by OpenAi 16:11 Google Gemini 1.5 22:05 Mixture-of-Experts 38:02 Nvidia’s Valuation 40:19 Apple Vision Pro 49:05…
Just use Postgres with Craig Kerstiens [not-audio_url] [/not-audio_url]

Duration: 1:16:42
Today's episode is with Craig Kerstiens, Craig has been in the Postgres space for a long time. First at Heroku, doing Heroku Postgres. Then at Citus, doing Distributed Postgres. Now at Crunchy Data, he's Chief Product Of…
From Academia to Startup Founder and Successful Exit with Jean Yang [not-audio_url] [/not-audio_url]

Duration: 59:14
Today on the show, we have the founder and CEO of Akita Software and now head of product at Postman, Dr. Professor Jean Yang. Jean has a super interesting background, a former computer science professor at Carnegie Mello…
Durable Async/Await with Stephan Ewen of Restate [not-audio_url] [/not-audio_url]

Duration: 1:04:56
Today's guest is a legend in the distributed systems community. Stephan Ewan was one of the creators of Apache Flink, a stream processing engine that took off with the rise of Apache Kafka. Stephan is now working on core…
AI Incubation and Investing with Rak Garg from Bain Capital Ventures [not-audio_url] [/not-audio_url]

Duration: 53:12
Today's guest is Bain Capital partner Rak Garg. Rak is a super smart guy that's worked as an ML researcher. Then he was in product at Atlassian before moving over to the venture capital side of the world. In this episode…
Finding Product Market Fit with Cassidy Williams of Contenda [not-audio_url] [/not-audio_url]

Duration: 55:09
Today, we have Cassidy Williams, CTO of Contenda. Contenda unbelievably started as a sticker distribution platform that pivoted into a product that converts podcasts and videos into various other forms of written content…
Holiday Special! [not-audio_url] [/not-audio_url]

Duration: 1:22:25
In this special end of the year clips episode of Software Huddle, we took some time to highlight some of our favorite clips from our interviews since we launched the show back in August. Software Huddle: https://twitter.…