AI and Augmented Reality (AR) in Pinterest

The power of browsing in search of specific or not so specific visuals of people or items on the lookout for inspiration and ideas is no longer placed merely in the hands of Google photos, not with a platform like Pinterest having successfully marked its place in this area. 

 

Established in 2010 and presenting itself as a visual discovery engine, Pinterest offers a platform and mobile application in which its massive scale of active users can browse through photos, posting or rather “pinning” them and create customized digital inspiration boards. From images regarding recipes, home decor, renowned figures, or any other category, this platform’s got it all.  

 

Users in the present scenario expect instant data and a flawless and personalized experience on any platform they visit, without having to go through too many clicks, and Pinterest is all set on meeting these expectations and satisfying its user base.  

 

The platform’s headway towards object-recognition technology kickstarted with the acquisition of VisualGraph in 2014

 

Have you ever wondered how Pinterest is able to showcase, say, clothing recommendations on the basis of style or recommend home decor in accordance with a particular taste of the user? This is all undertaken through deep learning, which is a subset of AI and machine learning. This subset adopts neural networks for simulating the brain faster for data analytics and to instruct computer models. Comprehending the purpose of a particular search allows the platform’s deep learning models to serve personalized outcomes. 

 

With its intense degree of focus on personalization, Pinterest adopts technologies like AI and Augmented Reality for interpreting the wide sections of data and to narrow down and customize the search results individually for all their users. In this blog, we’ll broadly highlight how Pinterest adopts and utilizes these two technologies. 

 

“We’re constantly experimenting with applications that people can imagine using today,” There are AI projects focused on things like self-driving cars, but there’s also the everyday, accessible AI that helps people live better lives now.”

- Vijay Narayanan, the company’s Head of Discovery and Content. 

 

 

AI in Pinterest

 

AI is being adopted by the platform for luring an increasing number of users to purchase items that they observe on Pinterest’s website or their mobile app and for expanding their user base, as an attempt to gain more advertising revenue. 

 

Speaking of advertising, you can also take a look at our blog on IoT in Advertising. 

 

Skin Tone AI

 

From the year 2018, the platform has been allowing filtered searches to their users in accordance with skin tones, to help their users discover examples of faces resembling their complexions. This is one of the features of the platform where AI is being adopted. This skin tone is adopted for a range of features, allowing people to make use of the platform to discover custom looks for themselves. 

 

Recently in their fresh skin tone technology version, the platform has steered beyond its initial dependence on face recognition AI. This has enabled the platform to avert any biases taken up by the algorithms for detecting faces, for instance in the case of dealing with darker toned skin faces, which is generally a commonly faced issue. This has also allowed the platform’s skin tone systems to be adopted for discovering close up shots of items like accessories or clothing which also include skin. 

 

The platform states that their AI technology can detect over 2.5 billion objects across pins of fashion and home, such as sunglasses, tattoos, wedding dresses, or natural hairstyles. Several of these objects allow shopping i.e the user can click on the item or any such similar suggestion and buy it on the website of the brand. 

 

For instance, users on the lookout for inspiration for recipes for, suppose, sweet potato, can come across a photo of a sweet potato black bean casserole and pin it onto their board. Pinterest allows the user to zoom in on any specific portion of the image, for instance, a fork, using a feature of the platform termed as Lens, which is powered by AI.  The technology can inadvertently detect the object zoomed in and then facilitate links of likewise dishes being served with forks, which can be explored. This technology is also active for the photos uploaded by the users themselves. 

 

 

How does Pinterest use AI? 

 

1. Detecting Visual Similarities

 

Pinterest has been adopting machine learning for its task of detecting visual patterns and matching them to other pictures. The technology is being leveraged for processing over a

 

hundred million image searches each month, aiding users in locating content that resembles the pictures they pinned before. For instance, if we pin a picture of a panda print pillow, Pinterest will show us the decor of other animal prints of other users. 

 

2. Categorizing and managing images

 

If a chair having a modernly designed model has been pinned by the user, the platform can provide recommendations of other items from the same contemporary era. This is done through metadata, like the pinboard name or by the data of the website where the picture has been shared. This metadata allows the platform to comprehend the pictures such as through the names of pinboards and the websites where images have been posted, this helps the platform understand what the pictures suggest.
 

---

How Pinterest uses AI

---

 

3. Focusing on individual taste and habits

 

Although platforms like Facebook give priority to the content they gain from the contacts and friends of the user, unlike them, Pinterest hands more limelight to the individual taste and habits of the user, the content and the time of the posts pinned by them, allowing the platform to site to provide more personalized suggestions. 

 

 

4. Giving importance to the local taste

 

Being a global platform, over half of Pinterest’s users are situated outside of the U.S. The platform’s recommendation system has been trained to recommend well-known content belonging to the local region of the users, content created in their native language. 

 

 

5. Venturing beyond images

 

Pinterest doesn’t just focus on interpreting the content of a picture, it also analyses the captions of pinned content as well as keeps an eye on the items getting pinned to the same virtual boards. For instance, this enables the platform to link a particular purse alongside a certain dress that has often been pinned beside it, even if the two may not look alike.

 

                                                                           

6. Reducing self-harm

 

On 2019’s international World Mental Health Day, Pinterest announced in a blog post that for the past year, the platform has been adopting machine learning techniques for detecting and inadvertently shrouding content that demonstrates, rationalizes, or promotes self-injury. By adopting this technology, the platform claims that it has accomplished a considerable percent of cut down in reports of self-harm content on the part of users, and also states that it is able to abolish harmful content at a 3 times faster pace as compared to before. 

 

Additionally, in the same year, Pinterest cleared its platform by cutting down more than 4,600 search terms and phrases relating to self-harm. The platform ensured that if such removed content is searched by any of the users, they come across links for confidential and free support from expert and trusted resources, both on their pinboards as well as on their homepages. 

 

 

Augmented reality in Pinterest

 

Pinterest’s Augmented Reality feature is powered by Lens, which is the platform’s owned visual search technology. This feature allows users to search up “lipstick” or “lipstick shades” on the search bar and choose the “try on” option that would open their phone’s front-facing camera from where the users can go through and experiment with around 25 varying lipstick shades. Upon discovering a shade they prefer the user has the option of saving the pin to get back to it later.  This feature is an advantage for both the Pinterest users who adopt the platform for shopping as well as the platform advertisers who are engaging in the program. 

 

Every shade has a brand that is associated with it and this allows the users to do the shopping for the look instantly. The saved pins which adopt this Try On feature allow the users to go back for checking the brand and then they can buy the product at any point in time. The initial partners who joined hands with the platform for this feature include beauty brands such as Estée Lauder, Sephora, bareMinerals, Neutrogena, and L’Oreal.

 

Recently Pinterest has added to its Try-On platform by launching an AR eyeshadow. This builds ahead on the platform’s facial AR technology which it introduced back in 2019. 

 

 

The new AI competitor

 

Recently in the month of June 2020, Google launched Keen, its curation tool which would serve as a competition for Pinterest.  Keen was marketed as an approach to "share your passions" with people, which implies that we can develop special small boards on the basis of our interests.  For instance, if someone likes dogs, they can create a do-themed board (termed as a Keen) which Google would automatically fill up with relevant content pertaining to the interests of the user.

 

Contrary to Pinterest, Keen is operated via Google's AI and search engine technology. When a user makes a Keen, they are asked by the site to offer a few search prompts which the site can seek for fetching them content, while also having the auto-generate option in case the user wishes to let Keen do the work. 

 

Conclusion

 

From deploying Artificial Intelligence in its operations to wandering deeper in the field of Augmented Reality Pinterest has consistently been focusing on improving and updating its customer experience with the purpose of offering its customers a personalized and advanced platform to use and browse images.