Guidelines for International Games Testing

Test your set up ahead of time to avoid any unwelcome surprises

Having the right team

When running playtesting in a country with a different native language to your own, you will want to rely on a local partner to help with recruitment of playtesting participants, and with the moderation and interpretation of the playtests. Even if language is not a barrier it can be invaluable to use a local partner. Why? They can provide valuable cultural context to what you’re observing and hearing from participants to enrich your insights. Local partners are able to give insights into cultural, language, and social conventions, among many other areas. At Sutherland, we are lucky to be members of the UXalliance, a global network of exceedingly experienced local experts to provide exactly this kind of support.

Interpreters are another important puzzle piece to get right. The technical setup alone can be complex and local researchers are likely to have experience in setting up audio channels to capture both local and interpreted audio. It’s also important to think ahead of time how you’ll be running your research sessions and where the interpreter will be needed and most valuable. For example, do you want the session introduction interpreted, the comments made during gameplay, or the interviews with players before, during or after the playtesting? Depending on the length of the sessions, you may require more than one interpreter as it’s a tiring role and your interpreter may need to tag team with another. Your local partner or interpretation agency can advise on the best approach.

Regular calls with your local research partner are vital

Communication

Regular and transparent communication with your local research partner is vital. Where possible, we will establish a Slack or Teams channel for sharing quick, important information and keep the momentum going on the project. Regular calls with the team are also key; once or twice a week in the lead up to the research sessions is ideal to cover topics such as recruitment updates, walking through research materials, and game builds.

Where budgets and timescales allow, consider travelling to the location as this has a number of advantages. Travelling ahead of the playtesting dates allows you to test and fine-tune the set up, and sit with the moderator to walk through the discussion guide, latest game build, and research materials. Being physically present allows you to be located in the same room as your participants when the playtesting is taking place. Seeing reactions, hearing any asides and backroom conversations between researchers, interpreters, etc is often invaluable and helps to identify and solidify patterns that you’re seeing during the playtest.

Logistics

We use the term ‘logistics’ as a catch-all term for all the complexity of conducting international playtesting. Depending on the specifics of the project, you will need to consider and manage; time zones, shipping, travel, cultural and language barriers, technology, and do much of this while working remotely. It’s no small task, and provides plenty of room for things to go wrong. Using an agency experienced in international research to manage all of this for you is one way to increase your success rate, but that’s not always an option.

‘We have all the time in the world…’

Working across different time zones can make communication tricky, but using email and Slack helps to bridge the gap. However, with large time zone differences, it’s even more important to ensure that regular remote meetings are maintained. When scheduling playtesting sessions, you’ll want to ensure that the times accommodate convenient waking hours for the participant, as well as any remote observers in other countries. There’s a fantastic site, WorldTimeBuddy, which really helps calculate and visualise time differences.

‘I travel the world and the seven seas…’.

Ensure you allow plenty of time for shipping any hardware and other research equipment between countries. Shipping and customs is often unpredictable and so allowing good time to resolve any issues is important. If you’re travelling to the playtesting site, then it’s often easier to take equipment with you – just be aware that a well-packed case full of equipment may attract the attention of customs officials!

‘Someone done changed the lock on my door…’

Just because a game can be accessed (e.g. via Steam) and run on a computer or dev kit in one country, does not necessarily mean that it can be accessed and run in another. Differences in network setups, firewalls, and even electricity supplies, can all impact whether equipment works as it should. Making sure that you test these things out as much as possible ahead of time will help avoid any nasty surprises on the day when time is not on your side.

Overall, the two key takeaways with regards to logistics is to build in plenty of time and to pilot multiple times to ensure your equipment and set up is working.

Consider providing maps of controls to games testing participants

Briefing

If your local partner doesn’t have playtesting experience, then the briefing becomes all the more important. They will need to know how moderating, observing and analysing a playtesting session differs from traditional UX research.

The best way of doing this is to be there yourself to support and guide the process and act as an observer, arriving at least a day ahead of the sessions. Even if budgets and timelines don’t allow for this some knowledge transfer is crucial:

Confidentiality. If the game that’s been evaluated is unreleased and confidential, you may need to collect recording devices (phones, etc) from participants ahead of the session. An example of other things to consider might be if windows need to be covered to prevent screens from being seen by people in neighbouring buildings.

Observation strategy. Does your local partner have sufficient numbers of observers/notetakers? If you’re running a simultaneous playtesting session, then aim for at least one observer for every 2 screens. Do the observers know what to look for? Are they sufficiently experienced with the game to understand what’s happening? Make sure they have time ahead of the session to play the game themselves.

Induction. How are participants going to be inducted into the session? What information do they need to know? Should maps of the controls be provided? Are there tutorials that players should complete first? If tutorials aren’t available, what else can be done instead? Have you built in time for any materials to be translated to the local language?

Emergency procedures. What happens if the game freezes, crashes, or another bug happens such as sound cutting out? Agree procedures ahead of time so that your local partner knows any debug shortcuts, or processes to get participants back on track.

In conclusion…

With careful planning, game testing is a fun, engaging process for researchers. We hope that these guidelines provide some insight into what to consider when preparing to run games testing on an international scale.

If you’d like to learn more about our experience in research in different geos then get in touch! Contact us below to speak to one of our playtesting experts.

Image credits: iStockPhoto (Hero Image), all others Sutherland Labs.