How to structure a data team to climb the pyramid of Data Science
Before joining Airbyte, I worked at a few companies building Machine Learning teams. What I loved the most from my days as a Data Scientist was helping business teams extract insights from data to attain business impacts, the “sexiest job of the 21st century” as it was coined in 2012. I participated in expanding teams from various “data maturity” stages, and like many others, I struggled and often got disappointed with what happens when you hire a Data Scientist without a Data Engineer.
Then, with 2020 and the world going on lockdowns, I felt it was not a great time to start a brand new management role if I couldn’t have as many face-to-face interactions. Instead, I joined Airbyte as a software engineer individual contributor. Putting my data science endeavors on hold, I helped build an open-source data integration tool from its early stage with a very small team of talented engineers. I viewed this as an opportunity to lay solid technical and strategic foundations for what I believe would contribute to shaping my view of an ideal data organization, which I am now going to share with you.
At the same time, I would also like to discuss how adding data tools such as Airbyte to your data stack contributes to redefining the roles of Data Engineers, Analysts, Scientists, etc, and the dynamics of the data team with the rest of the company.
