A/B Testing

When to conduct A/B testing for your game

An A/B test enables you to compare two versions: either different versions or builds of your own game, or between your game and a competitor's game. Lysto's A/B testing option lets you see exactly how player experience changes with the build. 

Here's how we make it work:

Start with version A: First, we'll set up a playtest for your initial version. 

The first round works just like our standard playtests: players try your game, share their thoughts, and provide feedback through both gameplay recordings and surveys. Once these sessions are complete, you'll have a baseline of data to compare against version B, which could be either an alternate build of your game or the build of a competitor’s game.

Moving to version B: This gives you direct, player-to-player comparisons of the changes in the gaming experience. To prevent bias in the responses, the order in which the versions are shown to the players are alternated between different cohorts of players. 

Why this approach works: Players can provide a comparative review of the different versions of the same game, noting specific improvements or changes that caught their attention. Plus, since they're already familiar with the basic concept of your game, they can focus more on the differences and how that affects their gameplay. In the same vein, you can assess how players compare your game with a competitor’s game, so you have a better idea of what works and what needs improvement. 

After both tests are complete, you'll receive:

• Gameplay recordings showing how individual players approached each version

• Depending on whether you have compared two versions of your own game or if you compared your game against a competitor’s, you will receive survey responses that specifically address the comparative experience

• Clear insights into which version performs better for specific aspects of your game

This structured approach to A/B testing helps you make data-driven decisions about your game's development. Whether you're testing different tutorial approaches, UI layouts, or game mechanics, you'll get clear evidence of what resonates better with your target players.

Interested in conducting an A/B test? Reach out to us at playtesting@lysto.io.