While developing mirrord, which heavily relies on injecting itself into other people’s binaries, we ran into some challenges posed by macOS’s SIP (System Integrity Protection). This post details how we ultimately overcame these challenges, and we hope it can be of help to other people hoping to learn about SIP, as we’ve learned the hard way that there’s very little written about this subject on the internet.
mirrord lets you run a local process in the context of a cloud service, which means we can test our code on our staging cluster without actually deploying it there. This leads to shorter feedback loops (you don’t have to wait on long CI processes to test your code in staging conditions) and a more stable staging environment (since untested services aren’t being deployed there). There is a detailed overview of mirrord in this blog post.
Apple introduced SIP in 2015 to prevent tampering with system binaries because those usually have high permissions and entitlements. mirrord works by injecting its library (.dylib or .so) into the local process, the one you want to “mirror” into the cloud. In order to inject itself into binaries, it uses LD_PRELOAD on Linux and DYLD_INSERT_LIBRARIES on macOS. One of the features of SIP is that it disallows the use of DYLD_INSERT_LIBRARIES with protected binaries. Bummer - we can’t use mirrord to locally run e.g. bash or ls in a cloud context. we can’t run mirrord exec bash or mirrord exec ls. We’ll have to find a way to get around SIP!