Real-time Mask and Gear Compliance Check for Swiggy Delivery Partners
When Lockdown 1.0 was upon us in 2020 in India, the question on the mind of every Swiggy engineer or Data Scientist was, “How can we help our Delivery Partners (DP) and ensure their safety during this pandemic?”. To answer that question, Swiggy’s AI team proposed doing real-time mask detection using Computer Vision, which would encourage Delivery partners to wear masks to protect themselves and our customers during Swiggy deliveries. Soon, our Product team set forth a plan for implementing a self-audit check for mask compliance for our Delivery partners.
And within three weeks of the conceptualization of the idea, working together with our engineering teams, we were able to productionize a model for real-time mask detection. Following the successful launch of the mask compliance check, we were also able to include Gear compliance checks, namely Bag and T-shirts, to ensure our Delivery Partners are wearing Swiggy-authorized attire and carrying Swiggy bags.
What we have at hand is a classical object detection problem — For a given selfie of a person, we are to check if a person is wearing a mask (classify if it is a face with a mask or not, along with the type of mask) and to localize the mask (where is the mask in the image). This translates to identifying as one of the categories given below, along with localizing the region of a face (with or without mask) as bounding box coordinates (top_left, bottom_right)
