Designing data products
Working backwards from the end goal is a core principle of software development, and we’ve found it to be highly effective in modelling data products. In this article we'll explore a step-by-step, methodical approach to identifying data products that avoids overdesign while providing just enough clarity for teams to begin implementation. Starting with a use case, we work backward to define data products and establish their boundaries. By overlaying additional use cases, we generalise the data products to prevent overfitting. Finally, we assign domain ownership and define service level objectives to guide the architecture and implementation.
Kiran is a principal engineer at Thoughtworks with a focus on data. An avid practitioner of extreme programming, he is passionate about microservices, platform modernization, data engineering, and data strategy. As a senior leader in the Data and AI service line, he helps large, strategic clients in leveraging data to achieve business success
One of the earliest questions organisations need to answer when adopting data mesh is: “Which data products should we build first, and how do we identify them?” Questions like “What are the boundaries of data product?”, “How big or small should it be?”, and “Which domain do they belong to?” often arise. We’ve seen many organisations get stuck in this phase, engaging in elaborate design exercises that last for months and involve endless meetings.
