Speeding up Python with Rust, what works and what doesn't ! - Ketan Singh
Not so long ago I encountered lzstring python package which is itself a port from Javascript’s lz-string library. What’s the purpose you ask? Javascipt package was written to compress relatively large amounts of data and store it in localStorage. Depending upon kind of text, its redundancies and compression level of the algorithm I found that it can compress text very well. For Some of the texts I was able to compress text to its 25% of the original size. We can go into details of lzma and brilliance of Igor Pavlov here but that’s not the point of his post, maybe another time. Coming back to python package, which can do pretty much same thing but was quite slow. So I ran some benchmark to get baseline performance numbers using timeit. Its not the best tool out there to benchmark but it will do the job.
How do we speed it up? Most obvious answer few years ago was to use C or C++ to rewrite this particular module. It still is in some cases but extending python with Rust in my opinion blows both C and C++ out of the water. Why? If you ask some Rust evangelist they could probably speak on this topic for entire day. In my opinion it has the following advantage
