Joining Forces for an Arrow-Native Future

submited by
Style Pass
2021-08-05 18:00:08

Too often people say "let’s do something together" in passing, and don’t. There's the occasional inter-project collaboration, but rarely will people take that next step. There are countless reasons why this happens, and aligning goals is challenging to say the least. But after spending the last several years working separately on related problems in the data ecosystem, we realized our best hope to make lasting progress was to build a stronger, unified foundation. We needed to do something radically different.

Wes helped start the Apache Arrow project in 2015, and since then has continued to build a developer community to achieve Arrow’s dual goals. The first goal is to be an efficient, language-independent open standard for columnar data interchange. The second goal is to be a portable, high-performance computing foundation for doing analytics on that columnar data. To pursue these goals, Wes formed Ursa Labs in 2018 and Ursa Computing in 2020.

In parallel, Josh and colleagues at NVIDIA foresaw the potential of GPUs to accelerate analytics workloads. In 2017, they created the GPU Open Analytics Initiative and later RAPIDS, which has demonstrated the potential of accelerated high-performance columnar analytics. Josh and the cuDF developers collaborated extensively with BlazingSQL to bring GPU-accelerated Arrow analytics not only to the Python community, but to modern SQL workloads as well.

Leave a Comment