TL;DR: I published a parallel processing library for Rust that works differently than rayon and is especially useful in cases where rayon falls a bit

Dawid Ciężarkiewicz aka `dpc`

submited by
Style Pass
2022-01-14 10:30:10

TL;DR: I published a parallel processing library for Rust that works differently than rayon and is especially useful in cases where rayon falls a bit short. I'll talk a little bit about it, and show you when it can be helpful.

Rayon is a data-parallelism library for Rust. It is extremely lightweight and makes it easy to convert a sequential computation into a parallel one.

In a nutshell, rayon is a “change your .iter() into .par_iter() and see your data being processed on multiple threads and enjoy your program getting faster.

As much as rayon is usually very useful, quite often I have been hitting one of its shortcomings: it is designed to use a collection as a starting point of parallelism. If you look at documentation for its IntoParallelIterator the types that support turning into parallel iterators are pretty much only collections. But what do you do when you have an iterator and you would like to start processing it using multiple threads?

Well, you could always .collect() it into a collection and then process using rayon. But very often that defeats the whole purpose!

Leave a Comment