What you can learn from 27 random sets of participants

What you can learn from 27 random sets of participants

on / by Paul Veugen

With Usabilla you combinate quantitative and qualitative research. The results from a larger group of participants are visualized with colorful heatmaps and measurable scatterplots. How many participants should you use for a typical test? It’s quite hard to give one single answer to this question. The number of participants really depends on the purpose of your test and the type of questions you’re using. To give you an indication about trends we’ve plotted 27 random samples of an open question in a large demo case with over 500 participants.

200 participants 05
Random sample of 200 participants

5 participants

005 participants 01 005 participants 02 005 participants 03 005 participants 04 005 participants 05

In these five random samples with 5 participants you don’t see any clear patterns. There are no obvious trends and the points don’t really cluster.

10 participants

010 participants 01 010 participants 02 010 participants 03 010 participants 04 010 participants 05

In these samples with 10 random participants some vague patterns are recognizable. There’s still no good overlap in the different samples, but you could identify some basic trends.

20 participants

020 participants 01 020 participants 02 020 participants 03 020 participants 04 020 participants 05

The samples with 20 participants show interesting trends. The results look quite the some. If you want to really quantify results there are still obvious differences between the individual samples.

50 participants

050 participants 01 050 participants 02 050 participants 03 050 participants 04 050 participants 05

There’s an interesting overlap in the random samples of 50 participants. With these samples sizes you could quantify results. You can still see some slight differences in specific (smaller) areas of the visualizations.

100 participants

100 participants 01 100 participants 02 100 participants 03 100 participants 04 100 participants 05

These random samples of 100 participants show clear overlap, also in more detailed areas of the image. You can clearly recognize detailed patterns (menu items, buttons, images, etc). With about 100 participants you can better quantify your results.

200 participants

200 participants 01 200 participants 04

These two random samples with 200 random participants each show identical results.

Trends versus Quantification

The number of participants you should use for a reliable test really depends on the purpose of your test and the type of questions you’re asking. As a rule of thumb you’re already able to measure basic trends with smaller data sets. If you’re looking for quantifiable data you should be focusing on more participants.

Questions? Contact us.

We use our blog for updates about our service, demo cases, and how-to’s. Subscribe to our RSS feed.

Article by

Paul Veugen

Founder / CEO @ Usabilla User Experience designer, entrepreneur and metrics junky.

Share your thoughts

Pin It on Pinterest