LlamaIndex Webinar: NUDGE Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval

Fine-tuning your embedding model is an underrated way of increasing RAG performance.

​In this webinar, we hosted the authors of NUDGE (Sepanta Zeighami et al.) - a new non-parametric approach to embedding fine-tuning.

​Existing embedding optimization approaches either fine-tune the base model directly (which has downside that you need to reindex all your data with the new model), or train adaptor models that transform the output of the pre-trained model during inference.

​In contrast, NUDGE directly modifies the data embedding records themselves within a constrained bound - this does not drastically change the embeddings from pre-training but helps "nudge" the embeddings into a space that's better suited for the given use case at hand.

​Benefits:

​✅ NUDGE runs in minutes over millions of data records vs. spending hours for embedding model finetuning

​✅ You don’t need to know about the original embedding model at all

​✅ No added compute during inference time.

​Checkout LlamaIndex integration: https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/finetuning/embeddings/finetune_corpus_embedding.ipynb

​Paper: https://www.arxiv.org/pdf/2409.02343

Видео LlamaIndex Webinar: NUDGE Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval канала LlamaIndex