Fleet: Optimizing LLM Inference on Chiplet GPUs

In this AI Research Roundup episode, Alex discusses the paper: 'Fleet: Hierarchical Task-based Abstraction for Megakernels on Multi-Die GPUs' Fleet introduces a new multi-level task model designed specifically for modern chiplet-based GPUs. By mapping computation directly to memory scopes, it resolves the mismatch between flat programming models and hierarchical hardware. The core innovation is the Chiplet-task abstraction, which coordinates work through shared L2 caches to reduce redundant memory traffic. When tested on AMD Instinct MI350 hardware with Qwen3-8B, it significantly reduced decode latency compared to vLLM. This approach improves cache utilization and performance for memory-bound workloads like LLM inference. Paper URL: https://arxiv.org/pdf/2604.15379 #AI #MachineLearning #DeepLearning #LLMInference #GPUArchitecture #Chiplets #AMDInstinct #ParallelComputing

Видео Fleet: Optimizing LLM Inference on Chiplet GPUs канала AI Research Roundup