Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs (ACL 2025 Main)

ArXiv - https://arxiv.org/abs/2506.05629
Authors - Ananth Muppidi, Abhilash Nandy, Sambaran Bandyopadhyay

In this video, we dive into our latest research (to be presented in ACL 2025) on Input Dependent Soft Prompting with a Self-Attention Mechanism (ID-SPAM) — a breakthrough technique designed to make fine-tuning large language models (LLMs) more efficient and adaptive.

Traditional fine-tuning methods are computationally intensive and rigid. Our approach challenges that by dynamically generating soft prompts based on input tokens, using self-attention to identify which parts of the input deserve the most focus. The result? ✨ Fewer trainable parameters 🚀 Stronger zero-shot domain transfer 📈 Better performance on diverse NLP tasks

Whether you're a researcher, developer, or AI enthusiast, this video breaks down how ID-SPAM works, why it matters, and how it compares to other state-of-the-art methods like LoRA and fixed soft prompting.

#ACL2025

Видео Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs (ACL 2025 Main) канала Abhilash Nandy