Improving Instagram notification management with machine learning and causal inference
On Instagram, notifications play an important role in providing efficient communication channels between Instagram and our users. As the types of notifications have increased, a need has arisen to provide people with personalized notification experiences to help avoid them receiving excess notifications or ones they may not find to be important.
At Meta, we have been applying statistics and machine learning (ML) for notification personalization and management on Instagram. Today, we would like to share an example of how we used causal inference and ML to control sending for daily digest push notifications.
A daily digest push notification about stories is a type of notification that lists a digest of stories that are shared and ready for someone to view. When such a notification is delivered to someone’s device, they may click on the notification to view the content. Traditionally, an ML model called a click-through rate (CTR) model is used to predict how likely someone is to click on a notification. CTR models have been working well in many applications across the industry. The predicted click probability is used as a proxy to indicate the notification’s quality to the user. If the predicted click probability is too low, the notification will be dropped in the middle of its sending flow and the user won’t receive the notification because it has been deemed low quality.
CTR model-based filtering worked well for daily digest notification in the sense that the actual average click rate when using the CTR model was significantly higher than without the model. However, we also noticed that using the CTR model meant a large portion of the daily digest notifications were sent to users who are relatively active in terms of using Instagram. For many highly active Instagram users, even without sending these daily digest notifications, they would be able to view the corresponding stories in an organic manner. This actually opens up an opportunity to provide a better user experience by sending fewer notifications to active users who are likely to view the stories listed in the notifications organically.
