How Dosu Used LangSmith to Achieve a 30% Accuracy Improvement with No Prompt Engineering
Editor's Note: the following is authored by Devin Stein, CEO of Dosu. In this blog we walk through how Dosu uses LangSmith to improve the performance of their application - with NO prompt engineering. Rather, they collected feedback from their users, transformed that into few shot examples, and then fed that back into their application.
This is a relatively simple and general technique that can lead to automatic performance improvements. We've written up a LangSmith cookbook to let anyone get started with continual in-context learning for classification! If learning from videos is more your style, check out our YouTube walkthrough here.
Prompt engineering is the easiest and most common approach to help LLMs learn, but Dosu takes a different approach. Our team is not using prompt engineering, and we see significantly better results.
Dosu is an engineering teammate that acts as the first line of defense for ad-hoc engineering requests, protecting engineers from unnecessary interruptions and unblocking GTM teams. We intentionally use the word “teammate” rather than copilot or assistant because, like a teammate, Dosu should learn the nuances and workflows specific to your organization.
