RAM leaks is very common problem which happens in a big amount of external libraries. They might work perfectly, have a lot of downloads on NPM and st

How to simply workaround RAM-leaking libraries like Puppeteer. Universal way to fix RAM leaks once and forever

submited by
Style Pass
2022-06-24 12:00:10

RAM leaks is very common problem which happens in a big amount of external libraries. They might work perfectly, have a lot of downloads on NPM and stars on the GitHub. Nevertheless, all of these does not really save them from RAM leaks on a long run. Especially if they have some underlying native bindings. Library can be unreplaceable in own niche, have very simple API, and satisfy you in all other aspects but consume more and more RAM when you use it in long-running process.

For example, one of the most popular cases here are all libraries which are based on headless Chromium. One of them is very popular Nodejs package called puppeteer which is a great tool for crawling, user-imitation and testing tasks. This library is API-controlled wrapper over Chromium which allows you to load page, walk through selectors, make some clicks on site or enter some values in inputs on page. But when you use it in a long-running process your server monitoring tool starts reporting "RAM is almost full". Having lack of RAM on server is a terrible thing because it activates operation system's out-of-memory killer who starts killing any processes randomly causing service downtime or even touching system processes which destroy server connectivity until full reboot.

Another challenge is to find which code or library caused the leak in your app. What you might get is a huge amount of RAM reported by htop utility after couple of days of your process uptime and you will have no clues what caused it. There are reliable methods to understand it but we will not cover them in this post. Basically, assumptions and suspicions based on developer's experience in combination with code experiments work here as well.

Leave a Comment