I will attempt to try to organize my beliefs about playtesting in a chronological order over a series of posts. That will be more useful as a reference for when you run your own playtests. If you do use it, just remember that you are taking advice from someone who has never been published. Kids, don’t try this at home.
The key is balance. As a designer, I need to remember that I look at my designs with a certain set of assumptions about how players will behave. While those assumptions are true more often than not (I use my talent for economics quite a bit when creating mechanics, something that has proven invaluable), when they are wrong the results will be as disastrous as what happened on Saturday. By playing in all of the playtests myself, I was influencing my testers who all assumed that I knew “the best” way to play my own games. In essence, I was playing against several copies of myself. So I can’t participate in every test.
However, never playing myself is not an option, as Andy pointed out. There are events that happen to players while playing games of which I need to be personally aware. As a CCG playtester there is a term that gets thrown around when cards have a particularly large effect on an opponent: negative play experience. NPEs are where some event during a game creates such a negative emotional impact on a player that it overshadows the rest of the game experience. The emotions could be frustration at being unable to execute a particular strategy (or any at all in the case of infinite loops), annoyance at a luck factor (hence the terms “dice screw” or “mana/color screw”), or anger at another player for his or her seemingly unwarranted aggression or poor play that benefited a third party. As the designer it is important for me to experience these NPEs so that I can accurately assess their importance and determine a course of action to prevent them if I deem it necessary.
Whether or not you yourself are playing, always take notes. Make sure to record who is playing, what version you are on (which should later be combined with version notes for your game that track changes), and how long the game takes. That is the essential raw data, but there is much more you need to keep track of. Pay attention to non-verbal audio cues. Yelling, cheers, groans, and ponderous “hmmm”s should be noted with a short analysis of the player’s probable mood and what was special about the game state. Table talk need not be recreated word for word, but the subject of discussions concerning the game and strategy should be recorded. If players spend a non-trivial amount of time discussing what’s on television, or what movie they saw, or their plans for later that week, it is a sign that the game is not engaging enough or has too much downtime for the non-active players. Mark down the percentage of time these ancillary conversations take up. Players who seem disinterested should be watched for changes in mental involvement and the results taken down.
If players become anxious at someone else’s slow play, start recording the amount of time used by each player for each turn or phase. You don’t need to be precise, 30-second or 1-minute increments will do. Rules questions are almost inevitable during any playtest (indeed, most published games will end up causing numerous rules questions when played). Do not just answer these questions. Write down the question and the exact wording of your answer. If there are follow-ups, do not leave those out.
If you are playing yourself, taking notes will be much more difficult. Sometimes you’ll have no choice but to play because you need to test the game with N+1 players. When playing you will be far less naturally aware of the other players and their moods. Make an effort to stay aware, even to the detriment of your own result. Remember that you are not in this to win. Don’t lose on purpose if you can help it, as that will pollute the sample. But remember that winning is not your primary concern at the moment.
When you do play, try not to bog down the game with your note taking. Restrict it to when you are in personal downtime. Wait until after the game to record your own thoughts, because those are more memorable.
When the game is over, record the length of the game, both in terms of time and rounds/turns. Include the final scores, as they can help you decide on whether rubber-band mechanics (rules that have the effect of helping those who are in last place or at least far from first place) are necessary. If there were wild swings in player positions, add intermediate scores as well as a quick notation on any specific causes (events, luck, mechanics, and/or diplomacy) of those swings.
Part 1 - Create a Playtest TeamWhen Norm told me that the reason I was so surprised by what occurred during the Z-Man playtest of Black Market might be that in all my previous testing I had myself been playing the game, it was like a light went off in my head. I asked if I should just not allow myself to play any of my games, Andy said that that would be an equally monumental mistake.
Part 2 - Organize a Playtest Session
Part 3 - Introduce the Game
Part 4 - Play the Game
Part 5 - Get Feedback
Part 6 - Make Changes
The key is balance. As a designer, I need to remember that I look at my designs with a certain set of assumptions about how players will behave. While those assumptions are true more often than not (I use my talent for economics quite a bit when creating mechanics, something that has proven invaluable), when they are wrong the results will be as disastrous as what happened on Saturday. By playing in all of the playtests myself, I was influencing my testers who all assumed that I knew “the best” way to play my own games. In essence, I was playing against several copies of myself. So I can’t participate in every test.
However, never playing myself is not an option, as Andy pointed out. There are events that happen to players while playing games of which I need to be personally aware. As a CCG playtester there is a term that gets thrown around when cards have a particularly large effect on an opponent: negative play experience. NPEs are where some event during a game creates such a negative emotional impact on a player that it overshadows the rest of the game experience. The emotions could be frustration at being unable to execute a particular strategy (or any at all in the case of infinite loops), annoyance at a luck factor (hence the terms “dice screw” or “mana/color screw”), or anger at another player for his or her seemingly unwarranted aggression or poor play that benefited a third party. As the designer it is important for me to experience these NPEs so that I can accurately assess their importance and determine a course of action to prevent them if I deem it necessary.
Whether or not you yourself are playing, always take notes. Make sure to record who is playing, what version you are on (which should later be combined with version notes for your game that track changes), and how long the game takes. That is the essential raw data, but there is much more you need to keep track of. Pay attention to non-verbal audio cues. Yelling, cheers, groans, and ponderous “hmmm”s should be noted with a short analysis of the player’s probable mood and what was special about the game state. Table talk need not be recreated word for word, but the subject of discussions concerning the game and strategy should be recorded. If players spend a non-trivial amount of time discussing what’s on television, or what movie they saw, or their plans for later that week, it is a sign that the game is not engaging enough or has too much downtime for the non-active players. Mark down the percentage of time these ancillary conversations take up. Players who seem disinterested should be watched for changes in mental involvement and the results taken down.
If players become anxious at someone else’s slow play, start recording the amount of time used by each player for each turn or phase. You don’t need to be precise, 30-second or 1-minute increments will do. Rules questions are almost inevitable during any playtest (indeed, most published games will end up causing numerous rules questions when played). Do not just answer these questions. Write down the question and the exact wording of your answer. If there are follow-ups, do not leave those out.
If you are playing yourself, taking notes will be much more difficult. Sometimes you’ll have no choice but to play because you need to test the game with N+1 players. When playing you will be far less naturally aware of the other players and their moods. Make an effort to stay aware, even to the detriment of your own result. Remember that you are not in this to win. Don’t lose on purpose if you can help it, as that will pollute the sample. But remember that winning is not your primary concern at the moment.
When you do play, try not to bog down the game with your note taking. Restrict it to when you are in personal downtime. Wait until after the game to record your own thoughts, because those are more memorable.
When the game is over, record the length of the game, both in terms of time and rounds/turns. Include the final scores, as they can help you decide on whether rubber-band mechanics (rules that have the effect of helping those who are in last place or at least far from first place) are necessary. If there were wild swings in player positions, add intermediate scores as well as a quick notation on any specific causes (events, luck, mechanics, and/or diplomacy) of those swings.
Anyone out there actually reading this stuff should feel free to tell me what they like, don't like, want to see more of, or if I'm just plain wrong.
ReplyDelete