AR try-on feature

KIT208 Assignment 4 - Design of Interface Application » Devlog

Share this post:

Introduction/Overview

In the manner of purchasing goods, every people want to select the best item considering their current environment. Our team devised this interface solution so that the users make a better choice.

Description of the application

The technological advancements in the retail industry have successfully added practicality to its service to the customers as they are able to purchase their desired products without actually going to the place, which is said to be a great feature, particularly in the retail sector of many fashion brands. However, there are still some drawbacks in the interface part when it comes to online shopping as it is common to find misunderstanding that leads to customer disappointment, as such the customer is not fully aware of the product’s quality or how it looks on them for goods, specifically targeted for fashion products.

This application allows users to use of this interface as they did in traditional online shopping malls. If they show their figure or picture on the camera, the 3D model avatar will appear. All they have to do is make choices with a convenient interface. While making decisions, they can also compare their choices very straightforwardly through given interactions.

Thus, the goal of the interface solution is intended as a feature of online shopping to let the customer get more detailed information about the product before purchasing, such as its price and material or how it will look on them, specifically on fashion items. Therefore, the solution solves some of the issues with online shopping to increase customer satisfaction.

Description of the interface solution

By comparing the ongoing application of online retailing services to on-site shopping, usually, the customers can only get the product’s description via text-based information; even though some retailers have already included the use of videos to showcase what the product actually looks like. Nevertheless, it still does not solve the ongoing issue of customer disappointment after receiving the goods as scams might happen in this type of situation, especially in wear-able goods. In the context of fashion goods, the addition of videos might not be that helpful in deciding if the product actually suits the customers in specific ways they might have wanted. Thus, it has caused frequent complaints and bad reviews due to the customer’s disappointment.

<Reference of video usage in online marketplace, such as eBay and Tokopedia>

The interface solution for this is inspired by the existing use of AR technology in the retail industry and has been successfully implemented by Warby Parker, an online prescription glasses retailer, as it can be seen in the picture where they detect the face of the customers and add the product to the customer’s face in a filter-like technology. This offers the customer the ability to visualise how the glasses will suit their face as it is crucial for glasses to match the facial features of its user.

By gathering problems that come up with the online-retailing trend, the most suitable and existing interface solution is to make use of facial recognition technology as it offers the ability to mimic a mirror’s functionality when used. However, due to resource limitations, we have chosen augmented reality (AR) as a media of the interface solution as it offers more similar features to the existing interface solution. One being a resemblance to the face recognition technology in the actual solution as AR is able to detect the customer’s face as image targets, which substitutes the actual facial recognition technology. Furthermore, not everyone has proper virtual reality (VR) equipment if the solution is being implemented in VR instead of AR; therefore, it would be harder to achieve the application’s aim if the solution is being implemented using VR instead of AR.

<Reference of typical online shopping features from Glue store>

As for the other clothing products, such as shirts and jackets we are replacing the use of models in online websites as mentioned by using AR to solve the current issue with dissatisfaction due to poor visualisation before purchasing it; as customers might purchase a particular product by basing it on how good it looks like on a model but not on themselves, besides reading the text-based information. Hence, using AR is appropriate as it forms an unbiased opinion when using 3D models that resemble the customer, using the camera to point to the customer as the primary subject in the initial interface solution.

Although our planned interface solution is to include different varieties of fashion items, for instance, shirts, jackets, and trousers; our main focus for the interface solution is fashion items that can be worn in the customer’s head or face as facial fashion items, mainly eyeglasses since they tend to have longer consumption use. Due to its consumption duration, customers will be more considerate when choosing eyeglasses before purchasing. Thus, the interface solution suggests a detailed function for eyeglasses, with the addition of comparison function for different types of glasses to lower miscommunication and misunderstanding of the product that leads to customer dissatisfaction. However, the initially planned interface solution is to extend the application’s functionality to clothes with the main focus of eyeglasses and other facial fashion items.

Interaction Design

First of all, we aimed to let users make the most considerate decision for their facial accessories by comparing the different 3D avatars of themselves with chosen items. This would effectively shorten the consuming time that users make choices and get the customised product to their taste.

<The AR application storyboard>

We have discussed what the application should look like to give convenient interactions to users. Meanwhile, we found some examples of avatar customization in the game “The Sims”. That game screenshot showed correspondence with what we have planned, and gave some inspiration about how the interactions would look like, giving users various options as well as entertainment from directly visualising what they choose to the 3D figure.

<Reference of 3D model customization in the game “The Sims>

In addition to making use of the game interface, we decided to combine it with existing online websites, as they are used by the public due to the current trend. When we consider why the way most shopping websites use is well-settled, it would be because of some business aspect. Suggesting users some interaction boxes with the most compact information would be beneficial and attract the most of them is already confirmed by many specialties.

Therefore, we basically applied that way which will be interacted by mouse with screen canvas, for the options for colour, frame variety, and rotation. Also, it was sure that when we attach those buttons on the world canvas, which would be virtual buttons on the image target, they will be hard to adjust because they will not only move along with the image target. Moreover, by using Vuforia’s virtual button, we have faced some difficulty in getting the virtual button pressed when the application is run, which makes the application to be buggy. Additionally, the number of options we are going to provide would be too many to contain them into the world canvas.

Different from the basic buttons suggested above, we will only use world canvas for detailed position adjustment. We thought that virtual buttons for that give straightforward information about the exact actions of the user. Virtual buttons next to each axis of the avatar will let users be aware of where they are in, which switches the weakness that we proposed above into strengths.

There are several applications and websites that allow you to place a pair of glasses on your face, but Warby Parker goes one step further by using augmented reality to render the glasses in a real-time, 3D preview on your face. With the help of Apple's Face ID and augmented reality technology, Warby Parker has updated its iPhone app to allow users to visually try on glasses before making a purchase.

<Reference of feature in existing AR application from the brand, “Warby Paker>

It was a challenge to differentiate our application from Warby Parker’s existing one because it seems sufficient enough to cover our fundamental aim, giving users information. The difference between them starts from the biggest one, the reflection of the user's face. While Warby Parker’s application shows a mirrored figure exactly the same as our face, our process includes facial recognition and creating the 3D figure, which could be more inaccurate.

However, when the idea that whether users can look at themselves in an objective point of view is factored in, we strongly suggest that ours would be more helpful. Above all, with our application, users can even view the whole side of the face with the rotation interaction.

To firmly stand our decision, we are planning to place two 3D face figures on the target image, one of them shows the previous feature and another shows the current selection, which can help users effectively compare the products. We believe that there exists distinct advantages for both (our application and Warby Parker’s), users will choose one according to their needs.

Initial Technical Development

Firstly, we used the “Vuforia Engine'' that allows us to interact with 3D models with AR interfaces. Thanks to this software, 3D objects could appear when the camera targets the subject image and we could control the object with interaction buttons.

To realise our actual goal, the camera should recognize the real figure of the user and create a 3D model as accurate as possible. However, we made the camera target the user’s picture instead of the user’s real figure as we have limited resources to implement it. Fortunately, we could find an asset named “Avatar Maker” that recognizes the picture and creates a decent 3D model.

For the interaction buttons, As we explained in the interaction design part, we applied different interaction buttons to each canvas. Basic buttons will be located on the screen space canvas, which could be easily interacted with the mouse, such as face rotation, frame selection, and changing the colour. Users only need to click the mouse on the buttons to their preference, then the item on the 3D face would be changed. We put component scripts on each button to function suitable actions.

In a different manner, we employed virtual buttons which are provided by the Vuforia Engine to on the world scape canvas. With the buttons which are attached to each side of the 3D modelled face, users can adjust the position of the item in a detailed manner; moving the item into each axis like what we can do with Unity tools.

In addition, we would especially like to emphasise this technique that we have chosen to make our application’s direction clean. We are going to add a “Comparison” button, which shows an additional 3D face model that has the copies of the current choice. It will let users be aware of which item goes the best with them, comparing from an objective point of view.

<Additional function that our application provides>

Initial 3D Models

To make our interface solution into a reality, the first 3D model that is used in the application is the face model of the image target. It is one of the most crucial 3D models in the application as it majorly resembles facial recognition of a person, but instead we have chosen to make use of the image targets to resemble it. The asset for the face’s 3D model was taken from the Unity Asset Store as the asset generates an accurate 3D version of the image target to resemble the facial recognition feature from the initial interface solution.

<An unity asset that produces a 3D avatar from the picture>

Secondly, the next 3D models included in the AR application are different shapes of glasses: square-shaped and oval-shaped, to add variety to the user’s choice. The 3D glasses models that are used in the interface solution were the ones that are specifically editable so that the script can be implemented to change specific features of the glasses itself; such as: the colour of the frame and the lenses’ colour.

<Series of glasses for initial version>

We also planned to use 3D models for the upper part of the body to visualise fashion items, such as shirts and jackets in our interface solution. Although, it might be a challenge to find suitable 3D models of clothes that can be worn to the 3D model of the upper body part, thus, we are still looking for suitable 3D models.

Conclusion

In the future of fashion, virtual try-on technology in particular will be quite important. These technologies essentially transform the physical fitting room into a digital one, enabling customers to try on clothing via augmented reality without going to a physical store. Utilising a camera to capture the consumer, augmented reality trying-on solutions map a realistic virtual depiction of the object over the customer's body in real-time. Consumers may browse for clothing much more easily and conveniently with the AR try-on feature. The current fashion industry will need augmented reality clothing purchasing technologies like AR try on because consumers are wanting technology-driven experiences that increase convenience, promote sustainability, and aid in informing their purchases.