Amazon app can scan clothes photo with ai and gives product suggestions for webshop
Amazon announces StyleSnaps, a feature in the Amazon app that allows users to scan a captured photo, after which artificial intelligence offers suggestions with links to Amazon products that resemble the photographed item of clothing.
StyleSnap allows users to select a screenshot or a self-made photo and have it scanned by the tool. After the photo has been scanned at a number of points, which takes about one or a few seconds, similar items from the Amazon web store that match the garment in the photo in appearance are displayed.
Amazon states that building this feature was not an easy task, partly because according to the company, photos taken can be unpredictable because the perspectives, poses and locations can be very varied. In order to be able to identify the photographed garments, Amazon has applied computer vision and deep learning, for example to classify the items into categories such as ‘flannel shirts’ or a certain type of dress.
Multiple layers were used to enable the applied neural network to identify a large number of classes. For example, the angles and colors are mainly identified in the first layers, while the network recognizes patterns such as denim or floral clothing in the middle layers. After the algorithm goes through all the layers, it should be able to fairly accurately identify the fit and size from the garment in the image.
According to Amazon, this circumvents the vanishing gradient problem. This concerns the situation in which the signal of the training data of the neural network is spread too far over the different layers, so that there can no longer be any question of identification. In its own words, Amazon has used residual networks to solve this problem; this means that shortcuts are applied to make the training signal skip certain layers in the network.
That would help the network learn basic aspects first, after which it can focus on more complex aspects. This would allow the network to learn new concepts, but at the same time also remember things it has learned in the past. According to Amazon, the latter is crucial for StyleSnap, because it involves effectively processing large volumes of data
For now, the addition only works on the American part of the app. Amazon also previously released the Echo Look, a speaker cum clothing style advisor. Stylesnap also seems to be a further development of the ‘search with camera’ function that has been present in the Amazon app for some time. In addition, a photo can be taken directly from the app of any product, after which suggestions of Amazon products are shown in the same way.