Why do people want to be analytics engineers?

First, data was everywhere. Publications like the Economist—required reading for conceited twenty-four year old econ majors, insofar as subscribing to it and piling unread issues on your coffee table counts as reading—routinely published breathless charts and articles about how the amount of data in the world doubles every two years. Second, all this data could predict the future. Specifically, it could predict when your teenage daughter was pregnant before you, her parent, knew about it—and this was just the beginning. 1 Third, the oracles behind these predictions held the “sexiest job of the twenty-first century.” In the wake of the Great Recession, banking, consulting, and going to law school were out; learning Chinese and becoming a data scientist was in. Fourth, the reality of being a data scientist didn’t match the image. They spent up to eighty percent of their time cleaning data. The job, it turns out, wasn’t sexy; it was mostly “headaches” and “mundane labor.” 

This final number—eighty percent!—was burned in Silicon Valley’s collective brain. Though it may have been exaggerated, it captured a deep frustration in the industry: Cleaning data is a tedious chore that prevents data scientists from working on exploration and analysis, the “ parts of the job that they enjoy most.” As one data scientist told the New York Times, this drudgery got in the way of the “cool, sexy things that got you into the field in the first place.” Internet famous people wrote blog posts lamenting it. Other internet famous people started companies to fix it. And some people quit their jobs because of it. 

Leave a Comment