Whatever happened to Big Data?
There was a time not so long ago when “Big Data” was the hottest thing since sliced bread. Developers talked about it. Managers wanted it. Presenters made memes about it. Children asked their parents for Hadoop clusters for Christmas (okay, maybe not quite that far).
Big Data was going to change the way everything worked. We were about to solve every financial, medical, scientific, and social problem known to humankind. All it would take was a great big pile of data and some way to process it all.
But somewhere along the line, the big data revolution just sort of petered out, and today you barely hear anything about big data. Today, even saying “big data” is like exhibiting the millennial pause – you date yourself as soon as you open your mouth.
Big data very much followed the typical hype cycle. But given the pronouncements of curing cancer and solving world hunger via improved food distribution, big data’s hype peak wasn’t simply constrained to the ill-conceived open offices of countless developers. Rather, it was plastered all over the internet, newspapers, and the blogosphere (remember that?). Products whose only commonality with Hadoop was that it required more than one server were suddenly “big data systems,” perfect for serving the entire needs of the internet at web scale.
