AR Shopping App with Minigame For Retail Industry

Augmented reality shopping app for a large retailer

Industries
Technologies / Platforms

The mobile application enables users to learn about promotions; learn about the products sold in the store (general product info, price, promotion);

see each store’s opening hours and features; provide feedback, and entertain via earning in-game bonuses to exchange them for a discount.

Engagement model

Time & Materials

Duration

Ongoing since October 2020

Solution

iOS app, Android app, Web admin

Project team

1 PM, 1 BA, 1 QA, 2 Backend Developers, 1 iOS Developer, 1 Android Developer, 1 Unity Developer

Tech stack / Platforms

About the client

The client is one of the largest chains of convenience stores that opened its first store in 1999. Now the chain has over 15000 stores.

Client’s request

When it comes to improving in-store customer experience, modern technologies together with a customer-centric approach can work wonders.

According to statistics, 61% of shoppers prefer to buy in stores that have AR over those without it, and 72% of shoppers that used AR in their retail journey purchase goods they didn’t plan to buy. And 55% of respondents admit that AR makes the shopping experience more exciting.

So our client contacted us to craft a new kind of interaction between retail and its customers in order to create a new competitive advantage in the market and enlarge overall turnover by increasing purchase conversion.

Target audience

There is no specific audience segment since every existing and potential customer with a smartphone falls under the definition. However, the shopping app is intended for tech-savvy people and toddlers’ and preschoolers’ parents mainly.

Project challenges

AR Objects' and Goods Recognition

To enable an AR app to recognize objects precisely, you have to manage a large amount of data proficiently. At some point, we faced difficulties with accurate objects and goods recognition so we built a custom recognition model in Python.

Store's Logo Recognition

The logo of our client’s stores doesn’t contain enough control points so it was hard to recognize for smartphones’ cameras. To improve the logo recognition process, we trained our own model for object recognition using CoreML for the iOS platform and TensorFlow for Android.

Solution Overview

To start using the app, users have to download it and create an account. After registration, users can:

  • see each store’s opening hours and features;
  • learn about promotions and discounts;
  • get information about the product’s title, weight, price, and macronutrient ratio;
  • leave feedback about the stores’ work and service;
  • earn in-game bonuses and exchange them for a real discount.

The main idea behind the app was to craft a more engaging in-store customer experience. This was achieved by implementing AR functionality and a minigame.

• Augmented Reality

In the app, AR is the basis and is responsible for the following features:

– Access to information about each store’s working hours, special offers, and discounts via recognition of the store’s logo.

– Pointing the smartphone’s camera on a product package provides information about this product like its title, weight, price, and full food composition data including protein, fat, and carbs ratio.

– Access to information about each store’s working hours, special offers, and discounts via recognition of the store’s logo.

– Pointing the smartphone’s camera on a product package provides information about this product like its title, weight, price, and full food composition data including protein, fat, and carbs ratio.

• Minigame

Unity-based minigame was crafted especially for the app and is available only when the user is in the store. To play, users have to launch the game and find the place inside the store where the major game object is generated.

The game is simple yet engaging so both adults and kids whose parents are picking up groceries can enjoy it. But the most pleasant thing about the game is that it enables customers to accomplish two goals at once: it helps them and their kids pass the time in line and provides a real discount in case they win.

Project approach

In this project, we follow the Agile methodology and Scrum framework. The approach is quite regular – we work by iterations, each lasts two weeks. After every iteration, we share the build with the client, conduct a sprint review, and get down to another iteration.

We also run regular sessions with a UX expert provided by the client. Between these sessions, the expert conducts research and tests the app involving real users, so we constantly get valuable feedback from end-users which helps us to see the app from their perspective and generate ideas on improving it.

Results & Future plans

In 4 months we delivered the application to the client. Currently, we are focused on extending the AR functionality in the app. Our next step is to allow the app to recognize not just single objects (like a product or logo), but also groups of objects (like shopping shelves) and more complex objects in stores (like checkout areas). After that, we will implement the feature of highlighting the goods on the shelves that participate in promotions. The release is planned for the early summer.

At Itexus, we are convinced that the potential of AR is underrated for such domains as retail. We are looking forward to learning how the app we’ve crafted would change the overall in-store customer experience in our client’s region and help to attract new customers and increase the average check.

Contact Form

Drop us a line and we’ll get back to you shortly.

For Quick Inquiries

Offices

United States

8, The Green, STE road, Dover, DE 19901

Poland

Żurawia 6/12/lok 766, 00-503 Warszawa, Poland

Name
Company name
Phone number
Text
x