The Operational Analytics Loop: From Raw Data to Models to Apps, and Back Again
Over the next decade or so, we’ll see an incredible transformation in how companies collect, process, transform and use data. Though it’s tired to trot out Marc Andreessen’s “software will eat the world” quote, I have always believed in the corollary: “Software practices will eat the business.” This is starting with data practices.
To understand modern DataOps, we can look to DevOps principles. DevOps completely changed engineering practices and philosophies like continuous integration and continuous deployment to close the gap between operations and development. Today, we’re starting to see DataOps wash over the analytics world, pushing it toward the same type of repeatability, flexibility, and speed in data operations and processes. However, DataOps doesn’t solve for one vital element: The request-and-wait cycle.
Unfortunately, DataOps hasn’t been able to fully parallel the success of DevOps. Where DevOps leverages a continuous, functional loop of code deployment between teams, DataOps has always struggled with a gap between the data stakeholders who use data (e.g. RevOps) and the data team that delivers it.
