*ฅ^•ﻌ•^ฅ* ✨✨  HWisnu's blog  ✨✨ о ฅ^•ﻌ•^ฅ

My main background is a hedge fund professional, so I deal with finance data all the time and so far the Pandas library has been an indispensable tool in my workflow and my most used Python library.

Then came along Polars (written in Rust, btw!) which shook the ground of Python ecosystem due to its speed and efficiency, you can check some of Polars benchmark here.

I have around +/- 30 thousand lines of Pandas code, so you can understand why I've been hesitant to rewrite them to Polars, despite my enthusiasm for speed and optimization. The sheer scale of the task has led to repeated delays, as I weigh the potential benefits of a faster and more efficient library against the significant effort required to refactor my existing code.

Pandas is written in C and Cython, which means the main engine is King C...there got to be a way to optimize Pandas and leverage the C engine!

Here comes FireDucks, the answer to my prayer: a speed demon Pandas library!. It was launched on October 2023 by a team of programmers from NEC Corporation which have 30+ years of experience developing supercomputers, read the announcement here.

Leave a Comment