How to troubleshoot memory problems in Python
Finding out that an application is running out of memory is one of the worst realizations a developer can have. Memory problems are hard to diagnose and fix in general, but I’d argue it’s even harder in Python. Python’s automatic garbage collection makes it easy to get up and going with the language, but it’s so good at being out of the way that when it doesn’t work as expected, developers can be at a loss for how to identify and fix the problem. In this blog post, I will show how we diagnosed and fixed a memory problem in EvalML, the open-source AutoML library developed by Alteryx Innovation Labs. There is no magic recipe for solving memory problems, but my hope is that developers, specifically Python developers, can learn about tools and best practices they can leverage when they run into this kind of problem in the future. After reading this blog post, you should walk away with the following:
The EvalML team runs a suite of performance tests before releasing a new version of our package to catch any performance regressions. These performance tests involve running our AutoML algorithm on a variety of datasets, measuring the scores our algorithm achieves as well as the runtime, and comparing those metrics to our previously released version. One day I was running the tests, and suddenly the application crashed. What happened?
