GPT Reviews
Microsoft is investing $3.2 billion in Sweden for cloud and AI infrastructure, deploying 20,000 advanced graphics processing units and training 250,000 Swedes with AI skills over three years.
"Grokfast" is a new algorithm that accelerates generalization under the grokking phenomenon in machine learning by amplifying the slow-varying component of gradients, improving performance on tasks like image classification.
"Zipper" is a multi-tower decoder architecture that uses cross-attention to flexibly compose multimodal generative models from independently pre-trained unimodal decoders, showcasing superior performance in tasks like speech-to-text generation.
"MetRag" is a new framework for retrieval augmented generation that combines similarity and utility-oriented models, using an LLM as a task adaptive summarizer to generate knowledge-augmented text and outperforming existing models on knowledge-intensive tasks like finance and medicine.
Contact:ย ย sergi@earkind.com
Timestamps:
00:34 Introduction
01:49ย Microsoft to invest $3.2 bln in Swedish cloud, AI
03:42ย State Space Duality (Mamba-2) Part I - The Model
04:47ย Sam Altman, Lately
06:08 Fake sponsor
08:39ย Grokfast: Accelerated Grokking by Amplifying Slow Gradients
10:11ย Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities
11:38ย Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts
13:52 Outro