Longitudinal Playtests

When to conduct longitudinal playtests

Unlike single session playtests, longitudinal playtests – also called diary studies – follow players as they progress through your game over days or weeks. By tracking the same players over time, you'll understand if your progression systems work, if players stay engaged, and how their skills and strategies evolve.

Here are some common reasons why developers choose to run longitudinal studies:

• Your game has complex progression systems that unfold over time

• There's a deep narrative that can't be fully experienced in one sitting

• You're developing multiplayer features that need time to be properly tested

• You want to evaluate player retention and engagement over multiple sessions

• You need to understand how player strategies evolve as they become more experienced

If you're working on a game that's meant to be played over multiple sessions – like an RPG, or strategy game – longitudinal testing should definitely be part of your testing strategy. But before you jump into a longitudinal playtest, make sure you have the following as well:

• A relatively stable build of your game

• Features that require multiple play sessions to fully experience

• Tools for tracking player progress and gathering data, such as surveys

While longitudinal playtests provide valuable insights, they do have their limitations. They require more time, resources, and coordination than traditional playtests. You'll also need to decide how often you want players to check in and what kind of feedback you're looking for at different stages. The feedback you receive might be affected by external factors outside your control, like players' schedules or competing games. Additionally, some of your findings might be less relevant if your game has evolved significantly during the testing period.

To get the most out of your longitudinal playtest, start by setting clear goals and identifying specific metrics you want to track. Here are some other ways to ensure you set longitudinal playtest up for success:

• Use multiple feedback collection methods (For example, conducting surveys after the gameplay session and a focus group discussion after the playtest has concluded)

• Combine quantitative metrics with qualitative feedback

• Be responsive to playtester needs and concerns

At Lysto, we can help set up your longitudinal playtest and generate actionable insights. Here are some examples of successful longitudinal studies that we have conducted which provided in-depth insights into player experience: For more info, head over to the Case Studies section in our Game Central Hub!

Why add a survey to your longitudinal playtest

Surveys can provide a deeper understanding of how your players' engagement with the game evolves over the duration of the playtest. This will help you prioritize the actions that need to be undertaken to improve player experience. 

There are two routes you can take:

1. Run the same survey across all days of the playtest

2. Change up the surveys. Begin with a pre-playtest survey to assess the expectations players have from the game and the genre, conduct daily surveys across the playtest, and conclude with a longer end-of-test survey. This way, you can get a sense of the evolution of the player’s engagement with your game. 

If you’re unsure of which approach to take, reach out to us at playtesting@lysto.io.