Some of you would have tried this on an ecommerce site – checking to see how a lipstick shade or a hair colour will actually look on you. Amazon, for instance, has both a live video option, and a picture option. In both, you can try out different shades virtually, and see if it suits your complexion. Myntra has similar virtual reality options when someone shops for cosmetics, apparel and shoes.
Most such features are based on AI models, and are helping ecommerce companies make the customer experience exciting, and reduce returns of shipped products.
Rahul Jain, AI consulting leader at Tiger Analytics, says such models involve using a combination of computer vision, deep learning, machine learning, and 3D modelling algorithms. He says the system is initially trained with publicly available data on sites like Google and Meta. This will be data on things like skin colour, body sizes, differentiation among brands. The user’s captured image or real-time video, he says, is typically processed using algorithms like Google TensorFlow pose detection for body scanning and measurement to identify key points/regions of the body.
“Mathematical models render the customer with the digital version of the product. Finally, deep learning models (such as GAN-based or Diffusion-based approaches) enhance the scenes and context based on product attributes (texture, colour, fit, etc) before streaming the image back to the device, and enabling customers to visualise how the product fits or looks on them,” he says.
Raghu Krishnananda, chief technology & product officer at Myntra, says their AI system also has an algorithm that links a person’s measurements with the measurements of each brand on the portal, after which suggestions are given. “The AI system can find the right measurement despite different brands having different measurements of a particular size,” he says.
Understanding language
Ecommerce uses AI for a variety of other purposes too. If you find the homepage of your shopping app to be different from your friend’s, that’s because the AI systems are tracking users’ browsing history on the application, as also looking at users’ profile details – such as age and location – to provide customised recommendations.
Dealing with vernacular languages and Indian speaking quirks is another major use case. Kishore Thota, Director of Shopping Experience, India & Emerging Markets, Amazon, says their AI algorithm allows customers to type in English the way they understand it, and if the spellings aren’t correct, the technology will modify the spellings based on the words that sound phonetically similar.
“We’re constantly improving our search algorithms to help customers find the products they’re looking for. Our algorithms take into account query semantics, they identify phrases and brand mentions, correct spell errors, support transliterated and colloquial usages, and classify queries into product categories,” he says.
The platform has trained its AI model to understand eight Indian languages. There’s also a facility for users to talk to the system and the algorithm at the backend will interpret to provide suggestions – for instance, users can just say, ‘Find me red shoes’. On the same lines, Flipkart has developed AI models to understand vernacular languages written in the English script. “We found that people are more comfortable with their local languages. For example, someone might say ‘Bachchon ke liye dress’ and not dresses for kids,” Mayur Datar, chief data scientist at Flipkart, says.
Datar is also looking to replace the written word to explain what a user is shopping for, using AI. “Let’s say your favourite star has worn something. You like what that person is wearing but you might not want that exact dress. You can post a picture of it and get suggestions for clothes on the same lines,” Datar says.
Completing the look
These platforms are also creating systems that can look at you and suggest what might be best for you. Flipkart has a feature that analyses people’s skin tones and points out blemishes on the skin and suggests cosmetics accordingly.
Amazon has an outfit builder based on AI that assists customers in discovering fashion items and suggesting complementary products to complete their look. Flipkart is building a virtual assistant called Flippi, which hopes to replicate a physical store’s environment where a salesman is constantly in touch with the customer. Datar says Flippi will provide product assistance not only from the look perspective, but also the return or exchange policy associated with it.
We asked Flippi how we could buy an iPhone, and it provided all the options of iPhones on Flipkart. It then asked us what colour we preferred – no prompting required there. Then it asked us if we wanted to try Samsung, OnePlus or Xiaomi phone options, and prompted us about our price range and RAM preferences.
