Cutting product submission time 4x by replacing manual entry with OCR

Industry
FoodTech iOS & Android
Timeline
Jan – Mar 2022
Skills
User Research Interaction Design Prototyping
My role
Sole designer in the squad at Blix Group (which also runs Blix), working closely with the Product Owner from research to launch.

Zdrowe Zakupy helps shoppers make better food choices by scanning product barcodes. I redesigned how users add missing products: three photos read by OCR instead of typing out long ingredient lists.

Highlights
  1. The average time of submitting a new product dropped from 1.5 min to 0.36 min.
  2. 95% of submissions showed the right product with correct data and were approved without changes.
  3. No-product screens: 20% → 15% of scans in 30 days.
Intro

Zdrowe Zakupy is an app with about 350,000 users that helps shoppers make more conscious shopping decisions based on their own dietary preferences.

I worked there in a small team and was responsible for many features: new content formats, paid brand collaborations and improvements to everyday use cases.

The problem

Every fifth scan showed no product

Grocery stores change and add products all the time, so users were seeing an empty “new product” page on roughly every fifth scan.

The app is only as good as its database. If a product isn’t there, we can’t show a score, and that happens exactly when someone is in the shop deciding what to buy.

The old flow asked users to type the whole ingredient list by hand. For a product with 20+ ingredients it took a few minutes, in a shop, with a basket in one hand. People made mistakes, and wrong data ended up in the database.

Before: a flow to add new products manually. A “You found a new product!” screen, an empty form asking for name, category and ingredients, the same form filled in by hand for Nutella, and a confirmation that the product was added.
The old flow: users typed the name, category and every ingredient by hand.
Research

Forming the hypothesis

Missing products were the most common complaint in app store reviews.

We looked into the qualitative data and talked to a few power users to understand the problem better.

Four familiar products the app knows, and a fifth, a new soda, crossed out because it is missing
Four-panel storyboard: 1. Maya is doing her shopping as usual. 2. She sees a new soda from her favourite brand. 3. She wonders how healthy it is and scans it. 4. Unfortunately it’s not in the database.
Five app store reviews, from one to three stars: “You need more products”, “Why new products are never added”, “I don’t add products anyway because they don’t go straight to the application”, “2 stars because of missing products”, and “I used to add new things only when I wanted to know the score. But there were mistakes anyhow, so I stopped doing it.”
New products missing from the database · A storyboard of a scan that ends without a score · App store reviews about missing products
Interviews

Key research findings

It turned out that users wanted the score right away, not a “new product” screen. They were fine with adding a product themselves from time to time, but they wanted the score back while they were still in the shop.

  1. People don’t read while shopping

    Screens with more than one line of instruction were skipped. The flow had to explain itself with visuals.

  2. Everyone wanted to take a photo, not type

    Every person we talked to asked “can’t I just take a photo?”. That’s how they already save things on their phones.

  3. No feedback, no motivation

    After submitting, nothing happened. People didn’t know if it went through, so they stopped trying.

  4. 45 seconds was the limit

    If it took longer, users put the phone away. In a shop nobody has time.

Key takeaways
  1. Missing products meant bad reviews. Most low ratings were about a product that wasn’t in the app.
  2. People add a product to see its score. If it doesn’t show up in the app quickly, they don’t do it again.
  3. Typing caused mistakes. Users who tried the manual flow and still saw errors gave up.
Hypothesis

If users can add a product with three photos instead of typing, and get the score while they are still in the shop, they will add missing products themselves.

Solution

Three photos instead of typing

In the new flow users take three photos instead of typing: the front of the product, the ingredient list and the nutrition label. OCR on the backend reads the data. Users don’t type a single character.

Solution overview: three photos go into the OCR tool, a reviewer approves the entry, and it lands in the database
How the solution works: instead of manual input, the user uploads 3 photos of the product. The images go straight into our OCR tool that recognises the text, then a human reviewer approves the entry before it lands in the database.
Results

The new flow in four steps

  1. Scan detects a missing product

    The user is asked to add 3 photos to add the product to the database.

  2. Three guided photos

    Product front, ingredient list, nutrition label. Visual overlays show exactly what to frame.

  3. Submit and wait until the magic happens

    Photos go to the backend. OCR extracts everything. Under 30 seconds total.

  4. Push notification closes the loop

    When the score is ready, the user gets notified. Their scan now works.

Adding a missing product with three photos.
Outcome

What we learned after 30 days from launch

In the first 30 days we approved about 800 new products

The share of scans ending on the “no product” screen dropped by 5 percentage points, from 20% to 15%.

95% of submissions were approved without changes

I was one of the reviewers, so I saw the results directly: 95% of submissions showed the right product with correct data. We were scared that users would send inappropriate photos, but it didn’t happen.

The average time of submitting a new product dropped from 1.5 min to 0.36 min

Even if 3 photos sounds like a lot, it’s still smoother than the previous flow.

Looking ahead

Next steps

Fully automate the flow

Some users complained that they had to wait several days for the approval. In the long run three photos should be enough to get a score, with no manual step in between.

Remove cropping step from every photo

Some users said the process was still long and they’d like it to be even faster, so we decided to remove some of the steps.

Onboarding process

About 30% of users didn’t notice that this feature had changed, so we should think about an onboarding process.