Usability Evaluation Study
About
Eater is a popular website that covers the dining and food culture of various cities worldwide. The website provides restaurant reviews, food and drink news, features on local chefs and trends, and guides to dining out.
Eater Chicago is the Eater website's section dedicated to Chicago.
By visiting the Eater Chicago website, readers can explore articles and recommendations about new restaurant openings, chef profiles, food events, recipes and other food-related topics specific to Chicago.
The site often includes detailed reviews of popular restaurants, along with information on their menus, ambience, and overall dining experience.
Current State of the Website
Goals & Objectives
-
Assess the overall effectiveness of Eater Chicago for different types of users performing common tasks.
-
Identify obstacles to finding restaurant recommendations on the site.
-
Assessing the website to see if it satisfies user expectations and standards.
Research Questions
-
What are the major usability flaws that restrict users from completing common tasks?
-
How does the organization and structure of the website match users’ expectations?
-
How easily and successfully can users find restaurant recommendations based on their unique criteria?
-
How satisfied are the users with the presented recommendations?
Preliminary Research
Heuristic Evaluation
Cognitive Walkthrough
Identified major issues related to:
Identified learnability issues like:
-
Consistency and Standards
-
Endless scrolling to find relevant information
-
Aesthetic and Minimalistic Design
-
Lack of navigational cues
-
Missing filter options
Usability Testing Method
Conducted using within subjects design.
​
Recruited 6 participants.
​
Conducted individual study that lasted for 45 minutes each.
​
Asked to perform 4 key tasks in counter balance order.
Participant Characteristics
Eligible participants for this study must be:
-
18+ years of age
​
-
Residents of Chicago
​
-
Visiting restaurants/bars frequently (at least twice a month)
​
-
Actively looking for restaurant/bar recommendations
-
Based on criteria such as location, budget, cuisine, ambience, etc.
-
​
-
Looking to try new recipes at home
Task List
The following tasks were given to all the six participants:
-
Find restaurant recommendations based on location
-
Find restaurant recommendations for a certain cuisine
-
Find restaurant recommendations based on ambience and budget​
-
Find a particular recipe on the website
Evaluation Measures
The following measures were observed and evaluated for each task to determine Eater Chicago’s current usability standard:
-
Percentage of participants performing the task successfully
​
-
Time taken to complete the task
​
-
Number of articles browsed
​
-
Perceived difficulty of the task
​
-
User satisfaction with provided recommendations
​
-
Suggestions by participants for improving site navigation
​
-
Most and least favorite features of the website
​
-
Comments on the website’s visual layout
Top Findings
"This website feels disorganized, clunky, and confusing”
-
All participants completed Task #2 successfully.
-
Task #1 & Task #4 saw reasonable success (5 out of 6)
-
Task #3 saw the most failures (2 out of 6).
-
This indicates that the website does work as intended.
-
Participants’ satisfaction with the results was low.
-
Most settled for a recommendation found in the 1st or second relevant article they browsed.
-
The lack of filters meant they were settling for an acceptable but less-than-ideal recommendation.
-
Participants found Task #1 and Task #3 the hardest to complete.
-
For Task #1, they were shown irrelevant information.
-
For Task #3 they had no means to search by multiple criteria simultaneously.
-
Task #2 was the easiest as the most relevant article was at the top of the list of results.
Participant Opinions
"I would recommend it more for interesting content to read rather than a primary tool for finding recommendations, especially when pressed for time."
What they did not like
What they liked
-
Confusing navigation bar
​
-
Glitches that redirected to the main Eater website
​
-
Lack tabs led to the use of ‘search’ for everything
​
-
Lack of filters resulting in irrelevant information and endless scrolling
​
-
Tasks took longer than estimated
​
-
Experience was marred and interrupted by ads
-
Quick links to recommended restaurant websites and social media handle
​
-
Interactive map along with recommendations as a means to browse options.
​
-
Appreciated the passion for food displayed by the articles’ writing
​
-
Felt reassured that the writers articles were Chicago based
Recommendations
Potential design solutions can be:
-
Having relevant search criteria in the main navigation header such as the existing ‘neighborhoods’, along with ‘budget’, ‘ambiance’, ‘reservations’, and ‘timings’ (lunch, dinner, and brunch specials).
​
-
The ability to filter results by these criteria should also be an included feature.
​
-
Providing important information for each featured restaurant such as address, timings, estimated cost for two, etc. can help users make more informed and thus satisfying decisions.
​
-
Update the search algorithm to find articles based on keywords entered.
​
-
Exclude mentions of restaurants in cities other than Chicago.
Going Forward
-
Devise a test plan for an updated version of Eater Chicago that has incorporated desired features and addressed common participant complaints.
​
-
This round of testing will focus on the effectiveness of the implemented design recommendations with user satisfaction being a key performance indicator.
​
-
A larger subject pool will also be recruited to assure consistent and good quality data.
​
-
While the tasks used in this test can be repeated, additional tasks can be devised to test user retention using the number of articles browsed as a key performance indicator.