The objective of the test is to find out a better design solution for ratings and reviews submission flow through gathering feedback from users based on the questionnaire. Meanwhile, observe the user’s reaction, interaction and behavior on each prototype during the interview session.

ServicesUser TestingGenresE-commerce (Online Food Ordering)Year2019ToolsSketch, Marvel


OBJECTIVES

We want to know which design pattern is more engaging for users to give ratings and reviews.
We want to know the points of comparison between the two design options. 
We want to validate which submission flow is faster and easier to get feedback from users that are in line with our early phase business goal of collecting data.

PROCESS

1. Invite 5 people who work internally in honestbee who is not in the loop of ratings and reviews squad. 

2. Take individual interview way for each session. Briefing the usability test purpose for the interviewee.

3. Give quick introduce of A and B option to the interviewee.

4. Interview the participant from the pre-question list to understand their online shopping and reviewing behaviors from the user before testing.

5. Show prototype A to the interviewee, ask her or him to finish the submission.

6. Show prototype B to the interviewee, ask her or him to finish the submission. 

7. Get the interviewee to rate both designs based on the questionnaire.

MY ROLE


  • Planning, Recruiting
  • Goal Setting
  • Purposed A/B design options
  • Usability testing
  • Data analyzed, Reported
  • Took action based on the results

Design iteration

Possible design ideation when things took shape. 

Clickable Prototypes

Purposed A/B design options with prototypes to find out which is a better solution for ratings and reviews submission flow. 

Design option (A) — One-page scrolling 

One-page scrolling with every question needed to be filled out in only one page. As a reviewer, I am able to rate the deliverer, store, individual dish only on the same screen.

Design option (B) — Progressive Disclosure 

Reviewers give ratings and reviews step by step, page by page. In this UI treatment user can only read one question in one page. Once a reviewer tap on ‘next’ call-to-action button she or he is able to reach the next page to see the following questions.

Questionnaire

Pre-questions

1. How often do you leave reviews online? When was the last time? Which product and what is it about?

2. How was your experience? What were your difficulties?

3. Give some examples of good incentive that makes you want to leave reviews.

Questions

* Observe user interaction with prototypes. Follow-up questions:

* Tell me what you see on the screen?

* What did you expect to happen there? (If they tap on something wrongly)

* Did you notice this feature? (If they did not tap on something). What do you think it should be?

* How did you find the overall experience of using this? (Layout, Navigation)

* How easy was it to leave a review? Use score like 1-10, 10 is the easiest

* What did you like and dislike?

* Did you encounter any difficulties? What was that?

* Would you do it regularly? Why?

* Which of these two prototypes do you find better? Why?

Take-Away Points

Thanks to the amazing note taker. I was able to summarize findings to see the comparison points that indicate which is a better design for users from the interview minutes.

Participants preferred version A to B. In option A (6 out of 8 people preferred) In option B (2 out of 8 people preferred)

Reviewing behaviors

* Users tend to leave a review if a bad experience happened or they ran into some issues with orders, for example, messy packaging, late delivery, wrong order. If it was a good experience, the user didn’t leave reviews.

* Price and reviews from others are two primary points that drive users’ decision for placing an order. Also, most users looking at reviews every time trying to shop online.

* Users would leave reviews for incentives such as cash back, promotional code and vouchers.

* Users easy to leave a review without thinking while in-app notification prompted an order completed.

* Most users tend to contact support first instead leave negative reviews for ordering.

Tappable feedback

* In prototype A, people tend to skip tappable feedback section without tapping on any feedback options.

* In prototype B, tappable feedback section are noticeable for users. They trying tap and interact with tappable feedback section in this version.

* As some options don’t convey exactly how she feels. Would rather type out her own personal feedback.

* Users are likely to tap multiple feedback buttons.

Need help button

* If she has a 1-2 star experience with a driver, she doesn’t want to feel forced to contact CS as she would have to wait. Just wants to leave feedback and have them resolve it later.

* Also noted that she would not click the CS button because it would take her out of the review page. Suggested having the CS chat button on the last page after she had submitted her review.

* Help button was too prominent, making it seem urgent.

* It’s helpful and easy to contact customer service.

Rewards and incentive

* Participants tend to leave review spontaneously if there is a monetary reward as incentives. If there is some coupon, voucher, promotional code when complete fill out ratings and reviews request as submitting. 5 out of 7 people are in favor of leaving ratings and reviews for reward in this case.

* Participants show fewer interests in earning social currency in such food delivery and online shopping service.

Clickable prototype A

* More efficient and easier to use than B.

* Would like to finish rate and review in this version at once without thinking.

* Would want to leave a comment for the driver.

* Some users missed the review section. Felt that it should be larger and higher because it’s the most important page element.

* Prototype A gives an overview, she knows what is to be expected – it wouldn’t accidentally give the driver feedback in the food section.

* Can double check in a glance before submitting.

* Less draggy. * Know what to expect.

Clickable Prototype B

* Too many steps. Tend to drop out and leave in the middle.

* Felt like stopping midway as it was too draggy, and had too many pages.

* Didn’t know how many pages left.

* Focus on one thing at a time with a clean layout.

* Rather skip in this version, seems much effort to do if there is no incentive for next purchase.

* Would finish all steps with monetary rewards.

* Easy to understand as a tutorial in app.