Branchless UTF-8 Encoding ↑ Writing ↑ cceckman.com
I know how to decode UTF-8 using bitmath and some LUTs (see https://github.com/skeeto/branchless-utf8), but if I want to to go from a codepoint to UTF-8, is there a way to do it without branches?
To start with, is there a way to write this C function, which returns the number of bytes needed to store the UTF-8 bytes for the codepoint without any branches? Or would I need a huge look-up-table?
Very handwavy idea: encode a 32 bit code point into utf8 but store the result in a 32bit word again. Count the number of leading / trailing zeroes to figure out how many bytes are necessary. Write four bytes into the output buffer but only advance your position in the output by the number of bytes you really need.
The number of leading zeros will range from 12 to 321 – a very reasonable size for a lookup table. From there, we could look up other parameters by length (no more than 4).
I fired off a draft into the chat, then came back to test (and fix) it in the evening. When I got the tests passing, it looked like this:
