Tutorial: Model-Based Evaluation of RAG applications
Retrieval Augmented Generation (RAG) is a common way of building Generative AI applications that have access to custom knowledge bases.
In this example, we'll show an example that has a retrieval step to get documents. By tracking this, you can debug your app and see what documents were pulled into the LLM context. We'll also show how to evaluate it using an LLM judge.
Check out the RAG++ course for a more advanced dive into practical RAG techniques for engineers, where you'll learn production-ready solutions from Weights & Biases, Cohere and Weaviate to optimize performance, cut costs, and enhance the accuracy and relevance of your applications.
First, we compute the embeddings for our articles. You would typically do this once with your articles and put the embeddings & metadata in a database, but here we're doing it every time we run our script for simplicity.
Next, we wrap our retrieval function get_most_relevant_document with a weave.op() decorator and we create our Model class. We call weave.init('rag-qa') to begin tracking all the inputs and outputs of our functions for later inspection.
