Follow me down the optimization rabbit hole

submited by
Style Pass
2024-04-16 11:30:13

This article is about FOSS (free and open-source software). The expression "The Last Mile" refers to supply chain or delivery services that get the products or packages into people's hands. A larger company or service may produce it and ship it across the world, but it is usually up to a local company/carrier to bring it the last mile. I see FOSS as having a "Last Mile" problem. The problem is that it's missing proper incentives to maintain/finish/optimize projects which become important to a large set of computer users.  Occasionally I benefit from this problem - I optimize other people's software for a living; some of that software starts out as an open source project.

A few years ago I optimized a FOSS tool (ETCPACK - Ericsson Texture Compressor) for a large Silicon Valley company. This tool is used for compressing RGB images into a lossy texture that GPUs can directly decompress. My optimizations gave a significant speedup and I asked my supervisor to upstream the changes so that others could benefit. The response was typical, but not surprising - "The legal department won't approve of this, so forget about it". I've been in similar situations with the same decision when working with other companies. In this case, I felt quite strongly that this was a shortsighted move because there was no competitive advantage to be lost by sharing the code, it simply makes artists and programmer's jobs go faster.

A great majority of open source projects start out in a similar humble way - the author had a need or desire to write some software and shared it with the community. The reasons for sharing it can be altruistic or not. This is intrinsic motivation - when you want to get something done because "reasons". For me, I sometimes want to test an idea for making code execute faster, or I need something for a personal project (e.g. a LCD display library). Maybe the existing solutions don't quite fit or I want to do it "My Way ™️". All of these reasons are perfectly valid and some great FOSS projects started this way. What comes next? Well, the project will get supported (or exist) as long as the original author wants it to. Perhaps other people join in, fork the project, and make improvements. Great, this happened with libjpeg. The JPEG (joint photographers expert group) released some reference code in the early 1990s to get people started and unfortunately no one improved it (at least publicly) for a long time. Someone was eventually inspired to create the libjpeg-turbo project. It may have been for personal (intrinsic) reasons or maybe a company seeded the money to do the work. This is a major flaw in the whole system - most people on the Earth operate in a capitalist society and need money as a (extrinsic) motivation to work. The avenues for getting paid to work on open source are mostly non-existent.

Leave a Comment