Do Vector Databases Lose Accuracy at Scale?
New research finds significant accuracy loss at just 10,000 pages when using vector search for Retrieval Augmented Generation (RAG)
Vector databases, a key technology in building retrieval augmented generation or RAG applications, has a scaling problem that few are talking about.
According to new research by EyeLevel.ai, an AI tools company, the precision of vector similarity search degrades in as few as 10,000 pages, reaching a 12% performance hit by the 100,000-page mark.
The research also tested EyeLevel’s enterprise-grade RAG platform which does not use vectors. EyeLevel lost only 2% accuracy at scale.
The findings suggest that while vector databases have become highly popular tools to build RAG and LLM-based applications, developers may face unexpected challenges as they shift from testing to production and attempt to scale their applications.
The work was performed by Daniel Warfield, a data scientist and RAG engineer and Dr. Benjamin Fletcher, PhD, a computer scientist and former senior engineer at IBM Watson. Both men work for EyeLevel.ai. The data, code and methods of this test will beopen sourced and available shortly. Others are invited to run the data and corroborate or challenge these findings.
