What Happens In Playtest Feedback Does Not Stay In Playtest Feedback
I’m now completing final revisions on the Dying Earth Revivification Folio, (In which the Game Master is gifted with an extended and inescapable stratagem, whereby foolishly hesitant players might be cleverly tricked into enjoying the unparalleled sublimities of the Dying Earth Roleplaying Game.) Once again I’ve been blessed by a wealth of superb playtest feedback
Anybody who gives solid feedback can expect some of it to appear as additional text in the finished manuscript-but not all of it. That's why it’s essential to get as much outside playtest feedback as possible-to sift issues unique to one group from those everyone might be expected to face.
Roleplaying experiences go well or poorly based on a myriad of factors, only some of which arise from the rules set or adventure a game group has kindly volunteered to playtest. Our responses to those experiences can also be subjective. We can also, as mentioned previously, suffer a disconnect between the pure experience itself and our attempts to subject it to later critical analysis. For this reason a purely impressionist account of what happened and how the group felt about it may prove more enlightening than a carefully constructed theoretical argument.
That’s where comparisons between responses become crucial. If half your testers like scenario Y better than scenario X, and the other half reverse that response, you’re probably in good shape. If everyone loves Y and got stuck trying to play X, you may be looking at profound changes to X. Or a subtle tweak that makes everything fall into place; such are the vagaries of RPG design.
I do still sometimes make changes based on the comments of a single group. This happens when I can see others having the same issue, and the fix for it is reasonably easy to perform, without blowing up the word count. As a designer, I always want more words to explain every contingency. As gamers, you don’t want to pay the price in dollars and back strain for an infinitely verbose edition.
When most groups have a problem with something, it needs changing no matter how much work and rewriting is called for. This happened with the first iteration of starship combat in Ashen Stars. Even then subjectivity rears its head: while a strong majority of players rated the complete overhaul a success, a few missed the old way.
For any new game, you’re going to get at least one group whose experience utterly implodes. Often reading between the lines of a feedback report, you can sense the influence of one or two dominant players who resisted the game from the get-go and unconsciously torpedoed it for everyone. These are less fun than the reports that come in as raves. However, they’re a hair more useful. They show you what text you need to add to manage expectations, to explain the key features of its approach so that those disposed to it can find the game, and those who want something else entirely can stay clear.
On the DERF, one eagle-eyed Vancian (Chris Dalgety, you know who you are) caught me writing a tagline that mentioned the moon. Aficionados of the Dying Earth well know that this celestial body long ago crashed unceremoniously from the sky.
That’s a fun example, which will have to stand in for the many contributions of playtesters past and present. When submitting feedback, your least glamorous or insightful observations may make the most difference in ensuring a quality product. Telling me what confused you is enormously helpful. The spotting of simple errors-the kind that blare out at the publisher and designer the moment they first flip through the published pages-may be the greatest service a test group can render.