Generators - Without boats, dreams dry up

submited by

Style Pass

2023-03-26 18:30:02

One of the main emphases of my recent posts has been that I believe shipping generators would solve a lot of user problems by making it easy to write imperative iterative code, and especially to make that iterative code interact well with asynchrony and fallibility as well. One thing that frustrates me about the situation is that generators have been nearly ready to ship for years now, but very little visible progress has been made. In particular, the core compiler transform to take a generator and produce a state machine already exists, because it’s exactly how async functions are implemented.

I want to collect all of the decisions that need to be made before generators can ship, and also my opinions about those questions. In 2019, I wrote two posts about generators, and in 2020 I created a library called propane, implementing my idea of how to solve some of the remaining design questions. I also had a chat in early 2021 with a member of the Rust project who intended to take over the work on generators, but I never saw any public work as a result of that conversation. Unfortunately, this post will be a re-hash of my posts from 2019 and 2020 in many ways, because not much has changed.

I want to reiterate a point I’ve made before, which is that the feature I want to see shipped in Rust soon is an imperative syntax for writing functions that compile to iterators. This is what I call “generators;” the more general feature which can be supported by the same compiler transform - state machines that act as “resumable functions,” I prefer to call “stackless coroutines;” the big difference is that stackless coroutines can both return and receive values as they are paused and resumed, and so they are more flexible. But this also means they aren’t strictly suitable for iteration, and they have other remaining design issues to resolve. I don’t think stackless coroutines are a bad feature, but I don’t think they should be given the same priority as the narrower generators-for-iterators feature, and I think the syntax should be different between these features in the same way its different for async functions.