Photos (on iOS, iPadOS, and macOS) is an integral way for people to browse, search, and relive life's moments with their friends and family. Phot

Recognizing People in Photos Through Private On-Device Machine Learning

submited by
Style Pass
2021-07-23 21:00:04

Photos (on iOS, iPadOS, and macOS) is an integral way for people to browse, search, and relive life's moments with their friends and family. Photos uses a number of machine learning algorithms, running privately on-device, to help curate and organize images, Live Photos, and videos. An algorithm foundational to this goal recognizes people from their visual appearance.

Photos relies on identity information in a number of ways. As shown in Figure 1A, a user can scroll up on an image, tap on the circle representing the person that has been recognized in that image, and then pivot to browse their library to see images containing that person. A user can also directly access the People Album, shown in Figure 1B, to browse images and confirm the correct person is tagged in their images. A user can then manually add names to people in their photos and find someone by typing the person’s name in the search bar, as shown in Figure 1C.

Photos can also learn from identity information to build a private, on-device knowledge graph that identifies interesting patterns in a user’s library, such as important groups of people, frequent places, past trips, events, the last time a user took an image of a certain person, and more. The knowledge graph powers the beloved Memories feature in Photos, which creates engaging video vignettes centered around different themes in a user’s library. Memories uses popular themes based on important people in a user’s life, such as a memory for “Together,” as shown in Figure 1D.

Leave a Comment