RPT#264 – Conducting GM Surveys
A Conan: Atlantean Edition Review
Many issues ago, I asked for thoughts and impressions about the Conan RPG published by Mongoose. Recently, I received a copy of Conan: Atlantean Edition, and you can read my review of it here:
Overall, I think it’s a solid product with good GM usability and a good reflection of Robert Howard’s fiction.
I’ll be playtesting the Conan RPG soon and will let you know how things went.
NEW Sci-Fi Gaming Terrain Available NOW!
*** Free 64 oz. Gamer Mug with purchase! ***
ALSO, Message Boards, Photo Galleries and News
When I first started this e-zine in 1999, I felt that the task of game mastering was actually a collection of skills, knowledge, and experience, and that one could study, learn, and practice to become a better GM. I also felt that becoming a better GM would mean having even more fun at each game. I still think these things are true.
Improvement requires feedback though. What things should you continue doing because they add to game’s enjoyment? What things should you stop doing? What could you do better? A GM can only be self-aware to a limited extent during games because he’s so busy focusing on facilitating the game itself. So, it’s up to our players to supply us with the feedback we need and want.
Players often feel uncomfortable providing their GMs feedback, however. Players could be concerned with constructive criticism being misunderstood, taken personally, or scoffed at. Players might worry about offending you, being unable to come up with the right words to accurately express their thoughts and feelings, or just unwilling to start a potential confrontation.
One of the best tools to solicit player feedback is the survey. Surveys come in different styles and flavours you can tweak to everyone’s preferences, and they act as a sort of friendly middleman in the feedback process. They focus thought and present a less stressful way to provide feedback than a direct conversation, though I do feel direct conversation should be encouraged and performed in addition to surveys.
A GM could split surveys into four broad topics:
- Campaign: establishing playing and GMing styles and preferences so you can construct a compatible campaign framework.
- GM: how are you doing as a referee, storyteller, and in- game opposing force?
- Player: how well are the players contributing, getting along, and co-authoring the ongoing story?
- Character: fleshing out PCs according to the styles and preferences of the GM, player, and campaign.
The following article deals with #2: how are you doing behind the screen? Hopefully, the article will give you enough ideas and information so you can draft your own group survey to help you constantly improve your GMing over time so that everybody has more fun.
There are many varieties of survey and question types. Which is best for you and your group? To figure that out, I recommend this complete process for managing GM surveys:
- Build Your Wishlist
A recommended first step is to think about what you’d most like to learn about your own GMing and how you would like that reported to you. For example, if you ask a yes/no question, will “yes” or “no” give you the exact information you need to make a decision as to how you’ll change some aspect of your GMing to accommodate? Will asking about NPC personalities truly give you the feedback you want about challenge levels? Will asking if the players are having fun let you know whether you need to speed up, slow down, or leave unchanged session and story pacing?
Start with a wish list. If you could get honest feedback from your group about some aspect of your GMing, what would it be? Write down on scratch paper anything that comes to mind, or do a brain dump on your computer.
Once you have all your thoughts out, review them and organize them. During review, look for ambiguities or lack of clarity or specificity. For example, you might write down, “Are my sessions challenging enough?” Well, there are three stakeholder groups in every campaign: GM, players, and characters. So, in regards to the question, challenging enough for who? You might then break that question out into three, one for each stakeholder group.
Also, look for any broad categories that you can group information by, such as NPCs, session management and organization, fun factor, environments, and so on.
- Ask your players how they’d like to be surveyed
This step can be done at any time, but I recommend performing it as early on in the process as possible to prevent any wasted efforts on your part. Ask your group how they’d like to be surveyed. In rare cases, you might also ask your players _if_ they’d like to be surveyed. For example, with certain fragile groups, it might be better to get some more games under your belts before requesting feedback.
You might also consider drafting custom surveys for different players or groups of players. The end goal is to get feedback to help you GM better, and if certain players feel strongly about one type of survey versus another, you might need to use more than one format to accommodate everyone.
- Build the first draft of your survey
Once you know what feedback you’d like to receive, draft a survey that you think will get the answers you want in your preferred reporting format. At this point, you are just crafting questions, roughly grouped by any categories you’ve spotted to date.
You’ll likely find as you go through this step that more questions, wish list items, and categories will emerge. Feel free to iterate between Step A and C as much as you like. As noted below, you probably won’t be delivering all of your questions in a single survey anyway, so if you emerge from this process with a long list of questions, you’ll have lots of chances to chunk them up and deliver them in stages over time.
- Test your survey
Before presenting your survey to your players, try to get a couple of test runs in beforehand. Getting others to fill out your survey can reveal ambiguities, missed questions, and poor question types. It’s tough to survey non-players because they might not be able to understand or answer your questions, but it’s worth a shot.
- Gamer friends not in your campaign
- Previous players who you can ping via IM or e-mail
- Gamers on the Internet
After test surveys are complete, ask those who filled them out for feedback on the survey itself. Were the questions easy to understand, what was the PITA (Pain In The A**) factor of filling the survey out (too long, questions too complicated, etc.), was the format acceptable?
Also, compare the answers to your wishlist. Do you feel your survey will deliver the information you want? Tweak your survey as desired. Do as many tests as you like based on how busy you are and how many willing victims, er, testers you have.
- Take your own survey
Once you feel you have a final draft completed, take your own survey. Do this for two reasons:
- A final test to check for poorly worded questions, poor question formats, length, and so on. Compare how filling out the survey felt to how your players requested they be surveyed. Compare your answers to the information you wish to receive – will you get what you want from this particular survey configuration? If not, make changes.
- Fill out your survey honestly to rate yourself. This will give you a benchmark about how you feel you’re doing and what your strong and weak spots are. When the players are done their surveys, you can compare their answers to yours. This will give you a sense of how well you think you know the players and their preferences. If you’re close, then you can feel confident about various session decision-making. If you’re not close, then consider being more open-minded about what your group truly wants and survey more often until you come to know your players and yourself a bit better.
- After completing your own survey, make additional changes as needed.
- Deliver the survey to your players.
The permutations of survey and question types are huge. A reader knowledgeable in statistics is welcome to send in a Readers’ Tip about question and survey types, their statistical factors, and other information.
I personally categorize questions in three ways:
- True or False, Yes or No
These questions are usually fast to create and fast to answer, making them good candidates to include in your surveys. However, be warned that this type of question, more than any other, reaps answers of a quality directly related to the quality of the question. For example, if players mis-understand the question, all you’ll get is a single, one word answer, with no clues that they haven’t grasped what you were after.
Also, these questions are often inefficient when dealing with tricky, grey area, or granular topics. It can require several questions to fully flesh out a topic when a single, open ended question could accomplish the same.
For example, “Do you like my GMing?” will get you a yes or no. If the answer is no, you won’t find out why, unless you ask more questions in the survey.
Some topics are well-suited to this type of answer though, and as mentioned, the speed factor makes this a popular choice.
- Multiple Choice/Value Rating
Here, you make a statement and provide several answer options. This provides you with more detailed answers than just Yes/No, and lets you confine the scope of the answers so you can compare player responses easily, apples to apples.
This type of question is also popular with respondents because it’s usually fast to answer; however, they can be the most time-consuming to generate.
I quite like value ratings type of questions where you make a statement and then supply a spectrum of responses to see how strongly the player agrees or disagrees. These questions can help you plow through various topics quickly and thoroughly, though you still don’t get all the detail an open ended question can reveal.
How much did you enjoy the battle with the mercenaries last session?
1] It was awesome
2] 3] I enjoyed it somewhat
4] 5] It sucked
- Open Ended/Essay Type
Here’s the dreaded “he’s going to make me think and write a bunch of stuff” question that kills a lot of surveys. This type of question can get you the most detailed and informative answers, but busy players, or those who aren’t good at expressing themselves, might balk at answering. Also, answers received might be all “over the place” and hard to derive a consensus from.
It’s up to you to choose the type of question to get the information you want from your group. One trick to make your survey as appealing and easy to answer as possible is to add an open ended question _option_ to each of your non-open ended questions. Typically, you would ask “Why?” or add a “Comments:” box beneath each question. Remind players that these are optional, so that those who don’t want to answer don’t need to, but players who want to frame or qualify their answers can do so.
Once you have your questions nailed down, consider how you should deliver your survey. There are a few different options, and you want to pick a format that will get the highest response rate this time, and for future surveys. If you deliver a long-winded survey on paper, for example, you might get a grudging response this time and great protest next time.
Paper surveys are easy to deliver and return. They require literacy, but no other technical knowledge. Some players have limited computer or Internet access, so they might prefer paper.
A drawback of paper is that it can be lost or destroyed, and it provides fixed dimensions that open-ended answers must fit within, though players could always continue wordy answers on new sheets.
Paper might also be the most convenient format for GMs who like to keep all their campaign info in GM binders.
This is a convenient method because it allows players to respond when it’s convenient for them. Many folks type faster than they can write, as well. Sophisticated e-mails and programmer GMs can include forms within an e-mail (note potential spam-blocking though) for even faster/easier player response.
E-mails can be printed for GM binders, and responses are likely to be more legible than hand-written responses. On the downside, e-mail can feel less urgent or be less convenient to fill out for certain players. For example, paper surveys might be ideal for players who take the bus or have time to kill that’s away from a computer.
- Discussion board
If your group uses a forum to game or track campaign information, a survey thread might be the best format. Answers will not be private though, and players who answer their survey after others might become “tainted” by reading current survey responses. However, a forum provides a great way to organize responses and keep them handy, and it allows free-form comments and discussion outside of the survey.
- Discussion after sessions
This format allows a group to openly discuss various aspects of your GMing. If this scares you, then consider paper or e- mail formats. Also, some folks will feel uncomfortable expressing their thoughts face-to-face. On the other hand, asking for feedback after a session is fast and easy (with the caveats just noted), some players are better at verbally providing feedback, and everyone has the session fresh in their minds at the time of discussion.
Tip: focus on a specific question. Pre-plan a question or two before the session. Discussions can wander, so frame your question so that it generates specific feedback for you. You want to be specific so that you also don’t get a vague response, such as, “Yeah, the session was fun.”
- Comment book or sheets after session
Providing a comment book has already been detailed in the e-zine: Roleplaying Tips #180.
- In-game roleplay
This is, perhaps, one of the best ways to solicit player feedback. Have NPCs within your game ask your survey questions for you. This lets the players express themselves in-character, which can be a very effective method. It also has a side benefit of encouraging roleplay.
A big limitation is that you can’t chat much about meta-game issues, such as session length or the quality of the gaming environment. However, it’s terrific for discussing such things as challenge levels (“We’re getting the crap kicked out of us and it’s wearing us down–why do you ask, are you offering to help?”), quality of folks the PCs interact with (“Boy, the people in this village sure are boring.”) and so on.
- Survey tool
Google “free survey” and you’ll find sites that allow you to set-up polls and surveys. Sites with group services, such as Yahoo!, sometimes allow group polls as well.
Online polls are easy to fill out and are a viable option, though set-up might involve a learning curve, and not all of your players might have ‘net access. If you do use an online survey tool, be sure to add a question about who is answering the survey so you can distinguish player responses.
- Keep it short. You are better off to survey frequently with short questionnaires than you are to craft long, infrequent surveys. This approach requires less on-going planning and preparation as well.
- Make it a habit. The best case is where your players freely give you detailed feedback about your GMing. A little structure can go a long way to establishing this. For example, if you ask your players to fill out a comment book after each session, they’ll soon come to expect this part of the session and be thinking about what they’ll say ahead of time. If you slot the last 15 minutes of a session for administration and session feedback, everyone will expect this, accept it, and hopefully, open up.
- Offer a reward. Sometimes players need a bit of encouragement to finish surveys in a timely manner. You could put your foot down and demand compliance, but you might notice a growing number of empty seats at the game table. Instead, offer bribes. 🙂
Seriously though, a small reward often helps groups get into the swing of regular surveys. Experience points, story bonuses, a pool of dice bonuses or re-rolls, and similar rewards can be motivational.
- Process and react to feedback. The biggest reward players can receive for their feedback is noticeable improvement in your GMing. If your group feels that their feedback is making the game better for all, then they’ll gladly express their opinions. This requires you to study and think about survey responses, and to then plan how you’ll react. What will you specifically do differently next session in an effort to improve?
If you don’t take the time to process completed surveys, then the exercise will not bear a lot of fruit for you and your group, and the process will flounder. This is another reason to keep your surveys short: it makes processing answers a manageable, on-going task.
- Know thyself. Don’t get defensive, don’t make false assumptions, and don’t penalize players in any way for expressing their opinions. Regardless of whether a survey response is blunt, gruff, or confusing, take it all in stride and make efforts to turn all feedback into positive change. So, know how you typically react to feedback and curb any negative reactions.
- Clarify if required. Feel free to approach players (privately might be best) to clear up any aspect of their answers. It’s best not to assume, or leave things in a confused state. Ask for more detail or for help interpreting a player’s response.
- Take action. As mentioned, the whole point of a GM survey is to improve your GMing. You must absorb feedback and then try new things, tweak how you do things, and take specific action to improve.
- Focus on one change at a time. Each year I make a New Year’s resolution to improve some specific aspect of my GMing. That resolution usually provides a theme for the next twelve months. Surveys can reveal a lot of opportunity for change, so much so that it might feel overwhelming. To compensate, pick one thing about your GMing to focus on and improve at a time. Ideally, pick the thing that will result in the greatest noticeable change in the shortest length of time–a low-hanging fruit. This might take a single session or several, but use your success in this area to encourage confident change in others over time.
- Conduct follow-up surveys. So, you surveyed your group, processed the answers, came up with a plan of attack, and GM’d a couple of sessions. How do you know if your changes were successful? A good way to find out is to re-issue your survey.
Also, it’s entirely human to let one or more aspects of your GMing slip in your efforts to improve in other areas. Follow-up and repeat surveys will identify this type of trend.
Following are several examples you can use to build your GM survey. Some questions you might not feel comfortable asking. Feel free to re-word or to discard entirely.
- On a scale of 1-10, 10 being best, rate last session.
- Can you think of what might have made the session a 10?
- Which of the following statements is most applicable to the story so far:
- The story is awesome, don’t change a thing
- The story is great
- The story is good, but I don’t find it compelling
- The story is not bad, but there are a couple of inconsistencies or things that are bothering me (see my comments below)
- I don’t really understand what the story is about or what we’re supposed to be doing
- There’s a story?
- For each of the following encounters that took place last session, rate them 1-10, 10 being best…
- For each of the NPCs that appeared last session, rate them in terms of interest level or how entertaining you found them, from 1-10, 10 being the best…
- True of False:
- Your GM is prompt
- Your GM is organized
- You are enjoying the campaign a lot
- You are enjoying playing your character a lot
- You enjoy providing your GM with chocolate bars
- On a scale of 1-10, 10 being best, rate your GM on rules knowledge.
- If you gave your GM a low score on rules knowledge, do you feel this lack of knowledge impedes the game?
- Please list what rules you’d like your GM to master, and from that list, rate the top three areas you’d like him to master first.
- Does your GM make fair and consistent decisions? Please provide one or more examples of an unfair or inconsistent ruling so your GM can learn by example.
- Do you feel the GM abuses meta-game knowledge about PC plans and actions to provide unfair advantages to NPCs and foes? If so, please provide an example so your GM knows what to avoid next time.
- Did I make a rules mistake you didn’t mention? What?
- Do you feel your GM favours any particular player over the others?
- Do you feel your GM favours any particular character over the others?
- True or False:
- Your GM is good at dealing with troublesome players
- Your GM is fair
- Your GM is consistent with applying the rules
Preparation and Organization
- From 1-10, 10 being best, rate the current game room environment.
- Of the different venues we play at, which one is your favourite and why?
- On a scale of 1-10, 10 being best, rate how prepared your GM is.
- True or False:
- Your GM wastes little time looking up rules
- Your GM seems to find things, such as dice, books, and minis, quickly
- Your GM spends a lot of time reading through notes
- A lot of game time seems to be spent on mapping
- Do you think the game would benefit from a weekly “newsletter” style sheet that kept track of NPCs, story- arcs, important facts, party goals, etc., in-between gaming sessions?
- Would you like to see to-scale maps and miniatures used more often, particularly in combat situations?
- List the periods in the last session where the game seemed to drag on, you were bored, or the energy at the table seemed the lowest.
- In recent sessions, have there been times where you didn’t know what to do next? Were any of those times frustrating for you? If so, please make a list of those times and reasons why you were frustrated, such as from a lack of options, party dissension, and so on.
- Does your GM seek ways to expedite menial tasks?
- On a scale of 1-10, how lethal do combat encounters seem to you? What rating would you prefer them to be?
- On a scale of 1-10, how lethal do traps seem to you? What rating would you prefer them to be?
- On a scale of 1-10, how fun and interesting do roleplay encounters seem to you? What could be done to improve this rating?
- On a scale of 1-10, how interesting have trap and puzzle encounters been for you? What could be done to improve this rating?
- On a scale of 1-10, 10 being the best, rate how much you enjoyed the mix of combat, puzzle/trap, and roleplay encounters was last session.
- A perfect 10 would need:
- More combat
- Less combat
- More puzzles/traps
- Fewer puzzles/traps
- More roleplaying
- Less roleplaying
- A perfect 10 would need:
- Does your GM need to increase, decrease, or keep to the same length flavour text used in-game?
- Do you feel railroaded?
- Do you feel lost and without direction in the game?
- What specific scenes have you enjoyed the most in my games? Why? (Victory in great battle, great jokes, solutions to diplomatic mysteries, etc.)
- Do you think I have fudged rolls? Does such knowledge decrease your enjoyment of game?
- Were any of tonight’s settings/villains/battles/NPCs especially memorable/fun/exciting? Which?
- Name one thing the next session should have more of.
- Name one thing the next session should have less of.
- Based on last session, would you like to see less, more, or about the same of the following:
- Weaker NPCs: Less Same More
- Comparable NPCs: Less Same More
- Powerful NPCs: Less Same More
- Overall # of NPCs: Less Same More
- Political Intrigue: Less Same More
- Magical Items: Less Same More
- Combat: Less Same More
- Skill Use: Less Same More
- Focus on Character-Specific Plots: Less Same More
- Dungeon-Crawling Adventures: Less Same More
- City Adventures: Less Same More
- Wilderness Adventures: Less Same More
- World Travel: Less Same More
- Puzzles/Riddles: Less Same More
- Random Encounters: Less Same More
- Name one thing you appreciate about me as a GM.
- Give one suggestion for something I could improve as a GM.
- Is there anything else you think I should know?
- Questions/Comments/Constructive Criticism?
These question examples are a great start at crafting a comprehensive GM survey. The questionnaire is far from finished however, and your input is needed. If you have any additional questions to add to the list, or questions worded differently to match your taste, please send them on in. If there are enough submissions, I’ll craft an updated survey for all to use and benefit from.
* * *
Thanks to the following GMs who sent in their survey question ideas and thoughts: Dale Thurber, Janis Strazdins, Bernhard Arnold, Kamal Hassan, Pahl, Kate Manchester
Dragonlance: Tasslehoff’s Map Pouch
Straight from the pouch of the legendary Tasslehoff Burrfoot (the original), comes this collection of a dozen maps set in the Age of Mortals. The collection starts with a full-sized poster map of Ansalon, continues with regional and location maps, and concludes with a pair of kender-style maps rendered by the irrepressible adventurer himself….
Just read the sci-fi GM tips issue, and then I saw this:
“Celestia is a free real-time space simulation that lets you experience our universe in three dimensions. Unlike most planetarium software, Celestia doesn’t confine you to the surface of the Earth. You can travel throughout the solar system, to any of over 100,000 stars, or even beyond the galaxy. All travel in Celestia is seamless; the exponential zoom feature lets you explore space across a huge range of scales, from galaxy clusters down to spacecraft only a few meters across. A ‘point-and-goto’ interface makes it simple to navigate through the universe to the object you want to visit.”
The visuals are quite nice, and could prove a useful visual aid for sci-fi games.
I tend to have a staggering array of NPCs in games I run. Sometimes my players have a hard time remembering all the names (or spelling them!), so I help them out by creating a Dramatis Personae list.
I list each NPC by name along with a brief description, being careful not to include any secret, off-board information. NPCs are grouped in my list according to where they’re encountered, but it might work better for some GMs to group them by encounter, adventure, group affiliation, or some other criteria, depending on what type of game you run.
I keep the list updated in between sessions; my players are still responsible for keeping notes on NPCs they’ve just encountered in the current session.
This same concept works well any time you have a list of things, places, characters, etc. with which the PCs would at least be passingly familiar, but the players have a hard time keeping straight. I’ve done a similar list before with deities in a pantheon; it could work just as well with military units, starships, mecha, tribes, countries, whatever.
In solving the perennial problem of good names, I make use of my typographical errors during my computer-intensive day- job. If I type something incorrectly, I make a note in a file and save it for later. Then, I modify it as appropriate, adding prefixes/suffixes, or I use it directly.
- “withe” (should have been “with the”) became the Withe River.
- “enxt” (should have been “next”) became the city of Enxtbard.
- “tottle” (should have been “toggle”) became a gnome- barkeeper.
The sad thing is how many names I now have…not a good endorsement of my typing!
Lords of Madness: The Book of Aberrations
Wizards of the Coast
Unnatural Creatures of Unspeakable Evil.
Trembling hands have recorded horrifying stories of encounters with aboleths, beholders, mind flayers, and other aberrations. The victims of these alien creatures are quickly overwhelmed by mind-numbing terror–their only comfort is the hope for a quick death.
This supplement for the D&D game presents a comprehensive look at some of the most bizarre creatures ever to invade the world of fantasy roleplaying. Along with information about the physiology, psychology, society, and schemes of these strange beings, you’ll find spells, feats, tactics, and tools commonly employed by those who hunt them. Lords of Madness: The Book of Aberrations also provides new rules, prestige classes, monsters, sample encounters, and fully developed NPCs ready to instill fear in any hero….