case study

A/B testing better ways for new users to discover playlists

User testing revealed that newcomers had no idea how much our music library had to offer

promotional illustration with game characters playing music trivia
Illustration by Leonardo Rios Gómez

The Problem

SongPop is a free-to-play mobile music trivia game with a massive music library. I noticed during user tests that new players were often only playing one playlist over and over. They lamented that there wasn’t more music, but we had thousands of playlists available, and this is the reason for SongPop’s enduring success.

illustrated list of music categories
A team of music curators hand-picked SongPop’s playlists from hundreds of thousands of song clips

Turns out the layout was gating the music library content way too aggressively and people thought they had to pay to play more than one playlist, which was not true.

Selecting a playlist was a key part of the core game loop and new players were not figuring it out.

diagram of the small section of the screen that users looked at, ignoring everything else
Nobody scrolled down. Not one person. Zero!

The Solution

We were hemorrhaging new users. I proposed the parameters of an A/B test and created the designs. In less than a week, the test was up and running with new variations. I added a clear title to the screen and an extra-bright card element above the fold that housed a playlist recommendation. Some additional direct labelling helped players understand what all the cards and boxes mean.

four original screens of the playlist flow
Original flow: Start a game > pick a playlist (before improvements) > play > send > repeat
diagram outlining each of the screen improvements
These design improvements enabled both new and existing players to discover more music and watch more rewarded ads.

Results

The new layout was a success, it won the A/B test. We found a significant increase in the number of rewarded. ads that players elected to watch. Additional user testing revealed that new players figured out the mechanics of picking out alternative playlists sooner, and played longer sessions overall.


Let’s work together

I work across data, cartography, and product design. I’m a design system enthusiast, former game designer, and user experience expert.

I’m always open to chatting about new projects and opportunities.

Connect with me!