testtrim: The Testing Tool That Couldn't Test Itself (Until Now)
Today, we’re going to deep-dive into the kind of thing you can only “invest” time on if you’re a single engineer working on a project with no supervision. I just finished a crazy complicated development effort in my project, testtrim, and all I want to do is talk about how surprised I am that it actually worked.
Then, when any future software changes occur, it can use that analysis to select which tests need to be run to cover the software change.
If a test does either of those things, it requires some special handling to determine when the test needs to be run again. For example, if a file referenced by a test (eg. test_data/Fibonacci_sequence.txt) changes in the future, it would make a lot of sense to run the referencing test (eg. test_fibonacci_sequence).
The tool that I always reach for when I need to do syscall tracing is called strace. It’s a superpower to be able to use strace effectively, but it’s super easy to start with. You take a command that you want to run, and you throw strace at the beginning, and you get a record of all the syscalls that the command made:
