Creating invariant floating-point accumulators

submited by

Style Pass

2024-09-01 17:00:05

One of the requirements we have for astcenc is ensuring that the output of the codec is consistent across instruction sets. Users quite like having the choice of NEON, SSE4.1, or AVX2 SIMD when their machine supports it – faster compression is always good – but no dev team wants a game build that looks different just because a different machine was used to build it.

With the upcoming 2.5 release we’ve decided to aim for invariance by default, even though it has a slight performance hit, because it just makes life easier for downstream game developers. This week threw up an interesting set of case studies for where invariance goes wrong …

Before I go off a ramble about floating-point maths, the important learning point of this blog is that by the end of it you’ll know how to write an invariant vector-length independent accumulator implementation, and some of the common pitfalls that occur along the way …

The root cause of invariance problems is floating-point arithmetic. Due to the dynamic precision of floating-point numbers, the accuracy of the number represented by a sequence of processing operations depends on the values of the numbers involved. Changing the order of operations changes the value of the intermediate numbers, which changes the precision, which changes the accumulated error, which ultimately changes the final result.