Using Dspy To Detect Document Boundaries
DSPy is becomming increasingly popular (at least in my bubble on X) - and, in my opinion, for good reason! It provides a sense of control and composability that becomes addictive once you get a few reps and understand how it all fits together. It allows you to inject LLMs into your program’s control flow, providing useful leverage.
Today we’ll walk through a super simple application to solve a real-world problem with DSPy and demonstrate a few of its useful capabilities, though we’re just scratching the surface here:
Document processing workflows often involve complex, multi-section files where identifying logical boundaries between different components is important. Whether you’re dealing with contracts that contain exhibits, reports with appendices, or order forms with attached terms and conditions, knowing where one section ends and another begins is key to improving downstream processing accuracy.
While classifying single pages or multiple pages is straightforward for vision-language models today, how you use the models matters - the real world is messy. For example, a 15-page PDF might contain a 5-page main document, a 3-page appendix, and a 7-page exhibit - and treating them as a single unit could lead to poor extraction results downstream.
