How DBT Helps Data Engineers Work Like Software Engineers
In a previous article, we introduced DBT, an open source tool that data engineers can use to transform data directly within their warehouses.
One of the cited benefits of DBT is how it enables the modern practices of Software Developers to be applied to Data Engineering. But what exactly does this mean, and how does it benefit the data team and wider organisation?
Many ETL tools that are used by data teams are GUI based. The ETL code is implemented within the tool, often by clicking and dragging connections between database tables.
DBT breaks out of these proprietary GUIs and turns these transformations into readable source code using a simple "domain specific language". This code can then be edited in any text editor or IDE in the same way a software developer can openly choose how to develop their Java or Javascript code.
Breaking outside of a proprietary tool into human readable source code is great for openness and collaboration, and underlies many of the benefits we discuss below.
