At Mutable.ai, we want to make it much easier for developers to understand and build software, with a long-term mission to accelerate entire organiza

Leapfrogging traditional vector-based RAG with language maps

submited by

Style Pass

2024-07-18 06:30:16

At Mutable.ai, we want to make it much easier for developers to understand and build software, with a long-term mission to accelerate entire organizations with AI.

It might seem simple to plug in your codebase into a state-of-the-art LLM, but LLMs have two limitations that make human-level assistance with code difficult:

They currently have context windows that are too small to accommodate most codebases, let alone your entire organization's codebases.

We built a chat sometime a year ago based on keyword retrieval and vector embeddings. No matter how hard we tried, including training our own dedicated embedding model, we could not get the chat to get us good performance.

As you can see, the answers were oddly specific and seemed to pull in the wrong context consistently, especially from tests. We could, of course, take countermeasures, but it felt like a losing battle.

So we went back to step 1, let’s understand the code, let’s do our homework, and for us, that meant actually putting an understanding of the codebase down in a document — a Wikipedia-style article — called Auto Wiki. The wiki features diagrams and citations to your codebase.