Are prompts the new unit of work in applications?
In the early days of internet applications, the unit of work was either: a function call, a database query, an HTTP request. The latter being formally structured as APIs and we saw the rise of things like API design and microservices architecture. Services talked to each other through structured interfaces and well-defined contracts. Infrastructure patterns evolved around this model. Layer 4 handled traffic at the socket level, Layer 7 handled application-level routing based on paths, headers, and cookies.
But is something fundamental chanaging with AI? In AI-native systems, the primary unit of work is no longer a structured API call. It’s a prompt. A prompt isn't just data—it’s an open-ended instruction expressed in natural language. It doesn’t follow a spec. It’s not typed or versioned in the way an API might be. It might ask for a summary, a chart, a line of code, a meal plan, or all of the above. And it’s often sent to a generic endpoint like /v1/chat/completions with a POST payload that looks identical regardless of intent.
Routing at Layer 7 made sense when applications were deterministic. /cart/checkout clearly meant one thing. But in an LLM-based system, the meaning is embedded in free-form language: “Can you turn this bullet list into a short product pitch?”
