Why SVD Breaks LLMs (And How to Fix It)

Get the Code: https://github.com/JosefAlbers/rcrlm

Papers and Methods Covered:
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models (Zhihang Yuan, Yuzhang Shang, Yue Song, Dawei Yang, Qiang Wu, Yan Yan, Guangyu Sun): https://arxiv.org/abs/2312.05821v5
SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression (Xin Wang, Yu Zheng, Zhongwei Wan, Mi Zhang): https://arxiv.org/abs/2403.07378
SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compression (Xin Wang, Samiul Alam, Zhongwei Wan, Hui Shen, Mi Zhang): https://arxiv.org/abs/2503.12340
Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression (Jingcun Wang, Yu-Guang Chen, Ing-Chao Lin, Bing Li, Grace Li Zhang): https://arxiv.org/abs/2410.03765
Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives (Qinsi Wang, Jinghan Ke, Masayoshi Tomizuka, Yiran Chen, Kurt Keutzer, Chenfeng Xu): https://arxiv.org/abs/2502.02723
SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression (ICLR 2026 Conference Submission777 Authors): https://openreview.net/forum?id=KMAYsQO8pU

Видео Why SVD Breaks LLMs (And How to Fix It) канала Josef Albers