Problems with testing Python pre-releases and pip caching — sethmlarson.dev

submited by

Style Pass

2022-01-13 15:00:05

Some time ago the urllib3 team noticed that our Python 3.11 test suite started failing with a strange error. On GitHub Actions all we could see was:

Notice that our test execution was exited with a return code of -11 which corresponds to signal SIGSEGV also known as a segmentation fault. This means that somewhere in C-land an improper memory location was being accessed, not good! Without a better traceback we wouldn't be able to know where the error is occuring.

Quentin and I both attempted to recreate the error locally with mixed results. I wasn't able to reproduce the error but Quentin was able to reproduce the error with tracebacks that were happening in both the ssl and selectors modules:

After we could reproduce the error reliably, Quentin started git bisecting CPython to find the exact commit that introduced the issue. The error was narrowed down to this single commit and after seeing that the PR mentioned that "there should be zero change in behavior" thus an issue was opened on the Python issue tracker (BPO) to report what we'd been experiencing.

Within this issue Eric Snow (who was the author of the commit in question) answered quickly and was able to reproduce the error, however he mentioned that the commit changed CPython's Application Binary Interface (ABI) and after slightly rearranging the PyThreadState struct was able to make the error go away. Then Eric asked this critical question: