Comments on: Underdogs beat Expedia in usability showdown http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/ Wed, 03 Mar 2010 11:40:50 +0100 http://wordpress.org/?v=2.8.6 hourly 1 By: Robin Allenson http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/comment-page-1/#comment-258 Robin Allenson Thu, 03 Dec 2009 07:08:37 +0000 http://blog.usabilla.com/?p=172#comment-258 Although the tests are not aimed directly at increasing conversions, this is exactly the kind of data one could use to determine likely alternatives for A/B or multivariate testing. Although the tests are not aimed directly at increasing conversions, this is exactly the kind of data one could use to determine likely alternatives for A/B or multivariate testing.

]]> By: Kari http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/comment-page-1/#comment-196 Kari Thu, 12 Nov 2009 23:56:32 +0000 http://blog.usabilla.com/?p=172#comment-196 In the curernt ecocomic envirment it is refreshing to know that Expedia price sorting is not a predonmiate feature. My best guess is most users wanting to search by price tend to sort low to high which does not always give the guest the best overall value. In the curernt ecocomic envirment it is refreshing to know that Expedia price sorting is not a predonmiate feature. My best guess is most users wanting to search by price tend to sort low to high which does not always give the guest the best overall value.

]]> By: Paul http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/comment-page-1/#comment-195 Paul Mon, 09 Nov 2009 07:51:03 +0000 http://blog.usabilla.com/?p=172#comment-195 @Sean You're completely right. The combination of analytics, A/B tests and a usability study would be more valuable in this case. Unfortunately we didn't had access to the servers of these four companies ;). In this case we focused on the task success rate and time per task (both indicators for performance) to compare the travel sites on three basic tasks. Because the limitations of the test I've been careful with the conclusions: in this test, on these basic tasks, Expedia gets beaten by its competitors. @JeromeR So true. If we assume that Expedia was the only one out of these four to successfully upsell its visitors they did an awesome job ;). If we stick to the success rate of these three tasks Expedia has the lowest task success on all tasks and gets outperformed on task time on 2 out of three tasks. I suspect this could is an interesting indication for the usability of these four sites for users aware of price. @Sean
You’re completely right. The combination of analytics, A/B tests and a usability study would be more valuable in this case. Unfortunately we didn’t had access to the servers of these four companies ;) . In this case we focused on the task success rate and time per task (both indicators for performance) to compare the travel sites on three basic tasks. Because the limitations of the test I’ve been careful with the conclusions: in this test, on these basic tasks, Expedia gets beaten by its competitors.

@JeromeR
So true. If we assume that Expedia was the only one out of these four to successfully upsell its visitors they did an awesome job ;) .
If we stick to the success rate of these three tasks Expedia has the lowest task success on all tasks and gets outperformed on task time on 2 out of three tasks. I suspect this could is an interesting indication for the usability of these four sites for users aware of price.

]]> By: JeromeR http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/comment-page-1/#comment-193 JeromeR Sun, 08 Nov 2009 21:54:36 +0000 http://blog.usabilla.com/?p=172#comment-193 This makes Expedia look bad, but only in some respects. This study also points to a problem with usability research on people who aren't real users doing tasks provided by a scenario. The scenario requires a booking in a certain price range. It's possible that the people who "failed" at this task are reaching a booking page for a higher-proced hotel. If they convert (complete the sale), then Expedia hasn't failed at all. This study would be more meaningful if we knew the conversion rates of real customers. This makes Expedia look bad, but only in some respects. This study also points to a problem with usability research on people who aren’t real users doing tasks provided by a scenario. The scenario requires a booking in a certain price range. It’s possible that the people who “failed” at this task are reaching a booking page for a higher-proced hotel. If they convert (complete the sale), then Expedia hasn’t failed at all.

This study would be more meaningful if we knew the conversion rates of real customers.

]]> By: Sean Power http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/comment-page-1/#comment-192 Sean Power Sun, 08 Nov 2009 15:17:35 +0000 http://blog.usabilla.com/?p=172#comment-192 Neat information. Though I like the concept, it's really important to stress that the most important thing is how usability affects conversions. That's where the heart of the issue should be. Though task completion by speed can be an indicator of better conversions, each website has a different point at which one experiences diminishing returns by further optimizing this KPI. Neat information. Though I like the concept, it’s really important to stress that the most important thing is how usability affects conversions.

That’s where the heart of the issue should be. Though task completion by speed can be an indicator of better conversions, each website has a different point at which one experiences diminishing returns by further optimizing this KPI.

]]>