Challenge
Working with global teams in Los Angeles and Hangzhou, the challenge was to help bridge the cultural gap for game designers in China, help them understand the casual gamer audience, and work with them to design features that would appeal to a Western market. To accomplish this, my goals were to create personas for the game team, collect player feedback on a monthly basis, and deliver useful insights with direct impact on the business goals and player retention.
Approach
For UX research to be done properly, we needed a good balance of qualitative data and quantitative data.
Qualitative data provide a connection with real players so the game team could understand more subjective UX issues. Do players understand how to play? Do they know how to get around the game? Do they know how to get around the game?
Dedicated Tester Pool
It was critical that a tester pool of dedicated players was established quickly. In six months, using social media and email campaigns, we gathered and segmented 390 testers by age, gender, and player type. With the registered testers, we were able to run surveys, polls, and phone interviews within 2-3 days notice.
The tester demographic showed that there are 60/40 female to male ratio and usually 35-55. Player data, however, showed that the audience was closer to 50/50 and the larger target audience was 18-24. I realized that this age group probably did not respond or see our Facebook posts nor emails
So I decided to incorporate an incentivized recruitment campaign in-game to bring more 18-24 players into our tester pool. This tactic worked and we were able to test this age group more.
Playtesting
I am a BIG believer of early playtesting. The more eyes, the better. I also add some quantifiable element to the testing so we can better evaluate it. When asking players, I have them rate each component on a scale of 1 to 5.
Starting from the first line of defense, I encourage the UI/UX team to walk around and show early concepts and prototypes to internal employees, specifically people who are not familiar with the game. This should help identify any bugs or UX painpoints before putting the game in front of actual players.
Using first Playtest Cloud, then Lookback, we streamlined a test plan so that players could record their play session. From the videos, we could see where players were hitting pain points. We provided tasks to complete while playing. What I learned is that, in the comfort of their home and without a live moderator, they played the game as they normally would. The only request from the research team was for players to voice out their thoughts, actions, and questions. When possible, we also invited players to live focus groups and recorded their hand gestures with an overhead camera.
Post Launch Research
Post Launch Research
Research does not stop after launch. Often times, this is much easier to conduct since it has very little impact on release date. Several ways to conduct post-launch research are the following:
App reviews and Community
I love reading app reviews to uncover any player complaints and UX issues straight after the build release. Often players aren’t able to describe the problem but it helps to keep a pulse on players. Community is another place to uncover player pain points.
Player Interviews
One month after a launch of a new feature, we worked with community to recruit players to give us their honest feedback. Some of these players have been airing their complaints in Facebook, and we wanted to let them know that they were being heard.
QA and Customer Support Reports
After a game or large feature release, I pay close attention to QA and customer support tickets. Most often, I like to work with the QA team so they know what might be an UX issue and assign a ticket to me directly so I can investigate to determine what the problem is.
Once all UX issues have been uncovered, I will keep a UX issues backlog. When schedule permits, the game producer and I will work on getting them into the roadmap.
Quantitative Data
A/B Testing with Community
A/B tests worked well when the game team reached an impasse, and wanted a player vote to decide which direction to take. UX team worked on agreeing on what the variables and constants so the data was clean.
I worked with the Community team to run A/B testing with early concepts. In the community poll shown, players were more than willingly to partake in being part of creating the game.
By using polls, we were also able to directly message players in Facebook, and ask if they would be willingly to explain their chosen concept in a follow-up interview.
A/B Testing with Facebook Ads
With the UA team, we were also able to test variables such as colors, app icons, and messaging by partnering with the UA team. Using Facebook ads, the UA manager would then deliver a report of which concept performed the best with our target audience. Since these were blind testing, meaning viewers clicking without context, the tests had to be taken with a grain of salt. However, any player feedback helps to challenge design assumptions.
With the app icon testing, the red app icon performed the best. However, we interpreted as red being the reason why player engaged with it more compared to others. In the final design, we wanted to make sure the color red was in, but still communicate ‘UNO’ and ‘Sudoku’.
Engagement Heat Map
In a game redesign project, one of the most important tasks a UX designer can do is measure player engagement in the main screen. As a game ages, features are tacked on and remain on the HUD well past its prime. I went through this process on Zynga Poker and Hit It Rich.
With Mattel163, I partnered up with a data analyst and product manager to investigate how we could optimize the main screen of the UNO mobile app. I asked the data anaylst to provide click rates for all call to action items. Below are the estimated results.
Quick Play: 65% (Players accessing Classic Mode)
Daily: 45%
2vs2 mode: 15%
Room Mode: 10%
Rankings: 40%
Store: 15%
More: 25% (Players accessing Events)
Based on the engagement, the insights were the following:
1) Players wanted to play Classic Mode as they were used to growing up with UNO
2) 2vs2 UNO was a unfamiliar way of playing UNO and can be moved to a second level
3) Players liked Events but had to tap twice on ‘More’
4) Store was lost in the clutter of gold icons in the top bar
5) New players were not ready for advanced features: Fun Room, Go wild, and UNO Contest.
6) Daily Challenges had good engagement but also lost at the top
7) Sales was lost and needed more prominence
8) Rankings was very important but we needed the horizontal space to make all cards visible.
After the UI changes were made, new player retention went up by 20%. Players were able to get into gameplay much quicker, rather than having to swipe through cards. Store and sales promotions were much more visible. Overall, engagement went up significantly.
Outcomes
After three months of research, a UX researcher and I defined two player personas and their player journeys. Personas helped the game team define for whom were they designing a feature. I worked with the team to create UI and UX that was designed for different player types.
For new players, a new onboarding experience welcomed them to UNO via daily challenges. Through UX recommendations, we shortened the tutorial experience to bite-size chunks so players could start exploring the game on their own with less guidance.
Casual Players
For casual players, a less competitive game mode, UNO Story, was developed so they were not forced to engage with more competitive players. The game team followed up with social features such as Clubs and Game Chat.
Competitive Players
For competitive players, Leagues play and Go Wild features appealed to their competitive streak and being able to show off their game mastery via badges.