Building a RAG system? There’s no one embedding model to rule them all
When building a Retrieval-Augmented Generation (RAG) system, choosing an embedding model is a critical decision to achieve high-quality results. Embedding models transform text into dense vector representations in a high-dimensional space, aiming to represent similar concepts as vectors that are close to each other. A RAG system stores many “chunks” in a large database and relies on embedding similarity to choose which chunks are most relevant to responding to a particular query / LLM context.
However, the best notion of similarity can vary dramatically depending on the domain and task of a RAG system. For instance, in a general context, "weather forecast" and "snowflake inbound" might be considered related concepts. But in a tech startup environment, "snowflake inbound" could refer to data architecture for Snowflake’s data cloud – quite a different semantic relationship!
This variability has major implications for choosing an embedding model: it’s difficult to know which embedding will best encode similarity for any particular RAG system. In fact, even when we carefully choose embeddings for a RAG task we can’t know how well it actually performs until evaluating the effect of embedding similarity choices in a full end-to-end RAG system.
