Why Google Treats SQL Like Code and You Should Too
F or the past 2 years as a vendor working at Google, I’ve been observing the way Data Engineers at Google treat SQL the same way Software Engineers treat code. This winning mentality can be integrated into the data strategy of any company of any size. I’m going to walk through the ways that Google benefits from treating SQL like code and provide specific ways that all organizations can benefit from these principles.
Just like object-oriented code, SQL can be time-intensive to write, laborious to debug, difficult to understand, can cause versioning challenges, and must be maintainable. Speaking from the perspective of just one division at Google; SQL is also used to create data pipelines. When data pipelines fail, it must be easy and accessible to debug and fix data pipelines. With all of these factors in mind, it’s clear that code centralization must be a core component of data strategy.
With the added benefit of code management tooling, it’s easy to understand who has been changing or maintaining a given SQL script, and trace that author’s changes to other relevant queries. It makes it fast and easy to find failing commits, and revert changes or apply relevant fixes. Upon committing SQL code, the code is instantly deployed to the development environment. Development pipelines can then be executed and failures identified and fixed instantly. Each night, a UAT release occurs, and the code is promoted in order to ensure that the code would likely pass a production release. Each week, a production release of all SQL committed throughout the week occurs, and at this stage, has very few failures.
