Predictions for the Future of RAG¶

In the next 6 to 8 months, RAG will be used primarily for report generation. We'll see a shift from using RAG agents as question-answering systems to using them more as report-generation systems. This is because the value you can get from a report is much greater than the current RAG systems in use. I'll explain this by discussing what I've learned as a consultant about understanding value and then how I think companies should describe the value they deliver through RAG.

So why are reports better than RAG? Simply put, RAG systems suck because the value you derive is time saved from finding an answer. This is a one-dimensional value, and it's very hard to sell any value beyond that. Meanwhile, a report is a higher-value product because it is a decision-making tool that enables better resource allocation.

If I have one employee I'm paying hourly, they can use a RAG app to run a query, and then they can deliver an answer. This is a perfectly acceptable way of using RAG in one-dimensional static scenarios, such as asking single questions. However, when a research team wants to do interviews (question-answer queries), the deliverable isn't an answer to a set of questions. Instead, it's a report. So, the RAG app can save the time of 8 employees making 50 dollars an hour, whereas the report will cost $20,000. If the report is helping an executive allocate a 5million dollar budget, the price might charge becomes a much smaller portion of that investment? This is true even if the process to generate the report is just a RAG application in a for loop.

Leave a Comment