Project
Mattel163 UX Leadership
Role
Management, Interaction Design and UX Research
Team
Senior UI Designers (3), Animator, Concept Artist and UX Researcher
Date
October 2018-March 2020
Working with global teams in Los Angeles and Hangzhou, the challenge was to bridge the cultural gap for game designers in China, help them understand the casual gamer audience, and design features that would appeal to a Western market.
CHALLENGE
GOALS
Western Player Motivations and Behaviors
Create tools and frameworks to better understand Western player behaviors and motivations.
Playtest group and Data Collection
Recruit and grow a community of available testers and dedicated power users for interviews and playtests.
Weekly Insights
Deliver actionable insights that directly impact the business goals, improve player experience, and seed ideas for innovation roadmap.
process
For UX research to be done properly, we needed a good balance of qualitative data and quantitative data.
Qualitative data provide a connection with real players so the game team could understand more subjective UX issues. Do players understand how to play? Do they know how to get around the game?
With a UX researcher, we established several approaches to gather qualitative data: surveys, interviews, playtests, and data analysis.
Dedicated Tester Pool
It was critical that a tester pool of dedicated players was established quickly. In six months, using social media and email campaigns, we gathered and segmented 390 testers by age, gender, and player type. With the registered testers, we were able to run surveys, polls, and phone interviews within 2-3 days notice.
The tester demographic showed that there are 60/40 female-to-male ratio and usually 35-55. Player data, however, showed that the audience was closer to 50/50 and the larger target audience was 18-24. I realized that this age group probably did not respond to or see our Facebook posts or emails
So I decided to incorporate an incentivized recruitment campaign in-game to bring more 18-24 players into our tester pool. This tactic worked and we were able to test this age group more.
Playtesting
I am a BIG believer in early playtesting. The more eyes, the better. I also added some quantifiable elements to the testing so we can better evaluate it. When asking players, I have them rate each component on a scale of 1 to 5.
Starting from the first line of defense, I encourage the UI/UX team to walk around and show early concepts and prototypes to internal employees, specifically people who are not familiar with the game. This helps identify any bugs or UX painpoints before putting the game in front of actual players.
Using first Playtest Cloud, then Lookback, we streamlined a test plan so that players could record their play session. From the videos, we could see where players were hitting pain points. We provided tasks to complete while playing.
What I learned is that, in the comfort of their home and without a live moderator, they played the game as they normally would. The only request from the research team was for players to voice out their thoughts, actions, and questions. When possible, we also invited players to live focus groups and recorded their hand gestures with an overhead camera.
Post Launch Research
Post Launch Research
Research does not stop after launch. Often times, this is much easier to conduct since it has very little impact on release date. Several ways to conduct post-launch research are the following:
App reviews and Community
I love reading app reviews to uncover any player complaints and UX issues straight after the build release. Often players aren’t able to describe the problem but it helps to keep a pulse on players. Community is another place to uncover player pain points.
Player Interviews
One month after a launch of a new feature, we worked with community to recruit players to give us their honest feedback. Some of these players have been airing their complaints in Facebook, and we wanted to let them know that they were being heard.
QA and Customer Support Reports
After a game or large feature release, I pay close attention to QA and customer support tickets. Most often, I like to work with the QA team so they know what might be an UX issue and assign a ticket to me directly so I can investigate to determine what the problem is.
Once all UX issues have been uncovered, I will keep a UX issues backlog. When schedule permits, the game producer and I will work on getting them into the roadmap.
A/B Testing Creative
Community
A/B tests worked well when the game team required quick design validation. UX team decided on what variables to test.
By using polls, we were also able to directly message players in Facebook, and ask if they would be willingly to explain their chosen concept in a follow-up interview.
Facebook Ads
Partnering with the UA team also allowed us to test variables such as colors, app icons, and messaging. Using Facebook ads, the UA manager would then deliver a report on which concept performed the best with our target audience. Since these were blind tests, meaning viewers clicked without context, the tests had to be taken with a grain of salt. However, any player feedback helps to challenge design assumptions.
With the app icon testing, the red app icon performed the best. However, we interpreted as red being the reason why player engaged with it more compared to others. In the final design, we wanted to make sure the color red was in, but still communicate ‘UNO’ and ‘Sudoku’.
QUANTITATIVE DATA ANALYSIS
In a game redesign project, one of the most important tasks a UX designer can do is measure player engagement in the main screen. As a game ages, features are tacked on and remain on the HUD well past its prime.
With Mattel163, I partnered up with a data analyst to investigate how we could optimize the main screen of the UNO mobile app. I asked the data anaylst to provide click rates for all call to action items. Below are the estimated click rates for the homescreen.
Insights
We discovered low click rates of 30.2% for Quick Play and 7.0% for Store. After tapping Quick Play, players still had to decide between Classic mode or Quick Match Mode. Other modes such as Room Mode or Go Wild were too advanced for new players.
Design Impact
After the UI changes were made, new player retention went up by 20%. Click rate for Quick Play increased to 52%. Store and sales promotions were much more visible. Overall, engagement went up significantly.
Old Design
Due to the rotating carousel, players had a difficult time accessing the Quick Play. Store and Sales UI elements were obscured in the cluttered top bar.
Revised Design
The carousel was replaced with static cards and top HUD was decluttered to better surface the Store and Sales. For new players, we added level gates to other modes so they could be eased into the experience.
USER PERSONAS
After three months of research, a UX researcher and I defined two player personas and their player journeys. Personas helped the game team define for whom were they designing a feature. I worked with the team to create UI and UX that was designed for different player types.
For new players, a new onboarding experience welcomed them to UNO via daily challenges. Through UX recommendations, we shortened the tutorial experience to bite-size chunks so players could start exploring the game on their own with less guidance.
Casual Players
For casual players, a less competitive game mode, UNO Story, was developed so they were not forced to engage with more competitive players. The game team followed up with social features such as Clubs and Game Chat.
Competitive Players
For competitive players, Leagues play and Go Wild features appealed to their competitive streak and being able to show off their game mastery via badges.
OUTCOMES
Within a year, the UX team defined the UX research discipline by creating a strong testing community, monthly playtests and interviews, and delivering actionable insights to the development team.
With personas, the development team in China was able to grasp the motivations and play behaviors of the casual Western gamer. The team was able to observe how the Western player interacted with the game via Lookback.
The insights helped inform design decisions which improved the game’s success with player satisfaction and business metrics.