Underdogs beat Expedia in usability showdown Demo UX Cases

Underdogs beat Expedia in usability showdown

The international travel site Expedia (Alexa Rank 816) gets defeated by its competitors Hotwire, Priceline, and Travelocity on basic usability tasks. Expedia performed the worst in a usability showdown between the four major international travel sites. A total of 148 people participated in this usability test and tried to perform three basic tasks on one of the four websites.

Alexa rank for Expedia, Hotwire, Travelocity, and Priceline

Alexa rank for Expedia, Hotwire, Travelocity, and Priceline

Book a room in Amsterdam

The 148 participants tried to book a hotel room in Amsterdam in a certain price range on one of the four big travel websites. They had to find a way to search for a hotel room (task 1), limit their search on price (task 2), and book a room in the correct price range (task 3). The task performance was measured by the success rate and the time per task.

Expedia homepage

Expedia, Hotwire, Travelocity, and Priceline

Task 1: Where do you click to search a hotel?

The first task was to find a way to search for a hotel room on the homepage. Clicks on the right search fields or on one of the buttons leading to hotel sections have been measured as successful.

Sample results task 1 (Expedia)

134 participants out of 148 (91%) successfully performed this task. All four travel sites performed quite good on this first task. The diagram below shows the percentage of task success on task 1 for each of the four travel sites.

Another interesting metric is the time users need to answer each task. The following diagram displays the average time for task 1 (in seconds). On average it took users about 10 seconds to find a way to book a hotel room on the homepage of Travelocity. The homepage of Hotwire seems to be well organized for : it took users less than 7 seconds to find a way to search for a hotel.

Task 2: Where do you click to limit your search on price?

The participants were looking for a hotel in a price range of 80 to 120 dollars. The participants have been asked to limit the search results on price.

Sample results task 2 (Priceline)

The average success rate of this task was 76 percent. Only 60 percent of the participants found a way to sort the search results on price at Expedia. The sorting features on Hotwire and Travelocity were found by more than 80 percent of the participants.

It took participants only a little bit more than 7 seconds to find a way to sort their search results on price at Travelocity. Expedia users needed more than 20 seconds to answer this task. Hotwire and Priceline users needed on average around 13 to 14 seconds for this task.

Task 3: Where do you click to book a hotel?

The final task for this test was to book a hotel in the right price range from the page with search results. Only 32 out of 148 participants clicked on a link leading to a booking page of hotel in the right price range. Many participants did not select a hotel in the right price range or clicked on an element that didn’t link to a booking page. Priceline definitely outperformed it’s competitors on this task with a success rate of almost 70 percent.

Participants searching for a way to book a room at Expedia answered the task in about 13 seconds. The success rate on task 3 for Priceline was way better than its competitors and participants answered in less than 6 seconds.

Overall performance

Expedia gets beaten by its smaller competitors in this usability test. Priceline has the highest success rates and Hotwire seems to be the runner up. The differences in the total time for all tasks are small for Priceline, Hotwire, and Travelocity. Users need more time to perform a basic task (booking a hotel in Amsterdam) at Expedia.

Tested with Usabilla

This usability test was conducted with the online usability tool Usabilla. This online usability tool allows you to collect direct and valuable feedback from your users. You can use these lean and mean usability tests in any stage of the design process to collect feedback on a webpage, mockup, wireframe, or image. You create an online test in a few minutes, using any image as source. Users share their opinion or perform tasks by adding points and notes directly on the screenshot, mockup or image. Sign up for a free account to try it yourself.

7 comments

  1. Sean Power

    Neat information. Though I like the concept, it’s really important to stress that the most important thing is how usability affects conversions.

    That’s where the heart of the issue should be. Though task completion by speed can be an indicator of better conversions, each website has a different point at which one experiences diminishing returns by further optimizing this KPI.

  2. JeromeR

    This makes Expedia look bad, but only in some respects. This study also points to a problem with usability research on people who aren’t real users doing tasks provided by a scenario. The scenario requires a booking in a certain price range. It’s possible that the people who “failed” at this task are reaching a booking page for a higher-proced hotel. If they convert (complete the sale), then Expedia hasn’t failed at all.

    This study would be more meaningful if we knew the conversion rates of real customers.

  3. Paul

    @Sean
    You’re completely right. The combination of analytics, A/B tests and a usability study would be more valuable in this case. Unfortunately we didn’t had access to the servers of these four companies ;) . In this case we focused on the task success rate and time per task (both indicators for performance) to compare the travel sites on three basic tasks. Because the limitations of the test I’ve been careful with the conclusions: in this test, on these basic tasks, Expedia gets beaten by its competitors.

    @JeromeR
    So true. If we assume that Expedia was the only one out of these four to successfully upsell its visitors they did an awesome job ;) .
    If we stick to the success rate of these three tasks Expedia has the lowest task success on all tasks and gets outperformed on task time on 2 out of three tasks. I suspect this could is an interesting indication for the usability of these four sites for users aware of price.

  4. Kari

    In the curernt ecocomic envirment it is refreshing to know that Expedia price sorting is not a predonmiate feature. My best guess is most users wanting to search by price tend to sort low to high which does not always give the guest the best overall value.

  5. Robin Allenson

    Although the tests are not aimed directly at increasing conversions, this is exactly the kind of data one could use to determine likely alternatives for A/B or multivariate testing.

  6. Last Minute

    Simply want to say your article is as surprising. The clarity for your put up is simply cool and i can suppose you are an expert in this subject. Well with your permission allow me to grab your feed to stay up to date with coming near near post. Thanks one million and please continue the gratifying work.

  7. hoteluri mamaia

    I’ve read several just right stuff here. Definitely worth bookmarking for revisiting. I wonder how much effort you set to create one of these fantastic informative web site.

Leave a Reply

Your email address will not be published. Required fields are marked *