All A/Bout Testing: How Netflix Knows Which Images You’ll Click
It’s safe to say that Netflix has come a long way over the past few years. From its humble beginnings as a DVD distribution service almost 20 years ago, the digital streaming platform now has 86.7 million subscribers who collectively spend over 100 million hours per day engaging with content. And with that many eyes on their product, it’s imperative that the company nails their visuals.
As the saying goes, a picture is worth a thousand words, and Netflix knows that if they fail to catch a members attention within 90 seconds, they are likely to lose interest and move onto another activity (Netflix counts as an activity right? Asking for a friend…)
So, let’s delve into the specifics. When a member lands on the Netflix homepage, they are typically met with several details for each title including the display artwork, title, age rating, synopsis, and star rating. Through various studies, the Netflix testing team discovered that members would clock the artwork first, and then decide whether to continue to the other details. This can largely be explained by recent Neurological studies which prove that the human brain can process an image in as little as 13 milliseconds.
The power of imagery
With this in mind, Netflix’s Gopal Krishnan and his team asked themselves if click-through rate could in fact be improved within this first glance. So, they set up a series of A/B tests with the collective goal of increased member engagement as a direct result of artwork. Gopal explains on the company’s technical blog, “we sought the support of our Creative Services team who work on creating compelling pieces of artwork that convey the emotion of the entire title in a single image, while staying true to the spirit.”
The experiments began with Netflix Original, The Short Game, which tells the tale of several young kids competing with each other on the golf course. The default artwork did not make it clear that the film was centered around children, so the team created two alternate artworks and measured engagement (click-through, play duration etc.) for each variant. Sure enough, audience engagement increased; demonstrating that Netflix members are sensitive to these types of visual changes.
The company then began to experiment with a much larger set of titles across both ends of the popularity spectrum, from big blockbusters to smaller, niche categories. Gopal explains, “This test was constructed as a two part explore-exploit test. The “explore” test measured engagement of each candidate artwork for a set of titles. The “exploit” test served the most engaging artwork (from explore test) for future users and see if we can improve aggregate streaming hours.”
The results of the A/B tests were clear, showing members more relevant artwork actually encouraged them to watch titles they had not discovered earlier. For a more in-depth look at the process and challenges, you can read Gopal’s write-up here.
Throughout this testing phase, Netflix uncovered a handful of interesting trends surrounding artwork and imagery, and the influence they have on a members decision to press play. According to Nick Nelson, Global Manager of Creative Services, here are some of the biggest discoveries:
1. Emotions are an efficient way of conveying complex nuances
On top of the knowledge that humans respond extremely well to faces, Netflix found faces that portrayed complex emotions would outperform stoic or benign expressions. It seems a range of identifiable emotions make a story appear more compelling to viewers. To illustrate, have a look at which image ‘won’ the most engagement as the artwork for Unbreakable Kimmy Schmidt:
2. Nice guys often finish last
Using visible and recognizable characters resulted in more engagement throughout the testing process. But, what Netflix found particularly interesting was that members responded well to villainous characters, especially across both the kids and action genres. For Dragons: Race to the Edge, artwork that portrayed the baddies outperformed all others:
3. Less is more when it comes to cast size
Netflix’s testing also uncovered that engagement with a particular artwork would drop when it contained three or more people. According to Nelson, this discovery actually directly impacted his team’s creative decisions for Orange is the New Black, which is clear when you compare the main artworks for seasons one and two.
As Nelson explains, “while ensemble casts are fantastic for a huge billboard on the side of a highway, they are too complex at small sizes and ultimately, not as effective at helping our members decide if the title is right for them on smaller screens.”
These A/B tests may seem simple but they have had a huge impact for the streaming platform. According to Netflix statistics, they now see a 20-30% increase in viewings for titles with artwork that was chosen in this manner. Not to mention, your weekly Sunday evening struggle just got a little easier.
Images courtesy of Netflix.