It  was one of those Wednesday evenings that blend into night without you noticing. The office had emptied hours ago, but I remained at my desk, illum

Building a Responsive `Textual` Chat UI with Long-Running Processes

submited by
Style Pass
2024-11-24 14:30:12

It was one of those Wednesday evenings that blend into night without you noticing. The office had emptied hours ago, but I remained at my desk, illuminated by the soft glow of my terminal and an almost empty cup of coffee. The task seemed simple when it landed in my inbox that morning: “Create a terminal-based chat interface for our data processing team.”

Our data scientists had been using a powerful but slow natural language processing system. While it produced excellent results, it was synchronous, took anywhere from 2 to 10 seconds to process each message, and provided no feedback during processing. The team needed a more user-friendly way to interact with it, preferably through a terminal interface since that’s where they spent most of their time.

My first attempt was straightforward — create a simple input/output terminal UI. Type a message, wait for the response. But during the demo with Sarah, one of our senior data scientists, her feedback was clear: “It feels… frozen. I can’t tell if it’s working or stuck. And why can’t I see what’s happening while it thinks?”

Leave a Comment