I see the question on other blogs: How much play testing do you need to do? Also, I’ve gotten some feedback, that I don’t respond to feedback, and got to thinking about how challenging it can be to handle all the feedback I get. This post addresses both areas.
As far as how much testing to do, I think the answer is pretty universally agreed upon: As much as you can do. Find and create opportunities to play, and with as many different groups as you can. Play with friends, friends of friends and complete strangers. Observe people playing your game without your involvement and see how it goes.
That stated, how much testing do you NEED to do? My answer to that is, at least 20 games, no matter how long a game it is. Anything less than 20 and you can get surprises and negative play experiences that you never planned for. As stated, that doesn’t mean you should stop at 20, you still should test as much as you can, but if that number is less than 20, you definitely haven’t tested enough.
If you can hit 100 test, that would be better. I’ve played Cage Match!, or seen it played, hundreds of times, yet I still see things at times that I don’t expect. However, it’s a quick game and therefore it’s easy to conduct hundreds of play tests. I’m sure some game designers are laughing at the number 20 as way too low, but if you’ve got a 6-hour game like Eclipse, I don’t know that 100 tests are feasible. Do at least 20, try for 100, and see how far you get. I do think if you’re on a deadline and you’re trying to get something out the door, if you give it 20 good tests, you should be in decent shape. If you’re playing a game with different groups or factions or characters, you probably need to play each of them at least 5 times to get a feel for what they do. If you were to publish after only 20 plays, and after only 5 with each character, things will come up that you didn’t think of. However, that wouldn’t be the first game for which that is true. Plenty of games I’ve played had balance issues, or a rule or 2 that needed to be clarified or we had to make a decision on.
As far as how I arrived at the numbers 20 and 100, they’re not arbitrary, they’re based upon my experiences with Cage Match!
For the most part, play tests of the original version of the game — now called the “Advanced Game” since it had a lot more going on — were pretty positive. I tested the game with a father and son who were big MMA fans. Despite the availability of 173 different move combinations, they were disappointed that the game didn’t include strikes from the submissive guard position. They wanted more, even greater variety in the moves. That stated, they became big fans and supporters of the game. Others who were less into MMA, however, were not as enthusiastic. The Standing part of the game generally worked pretty well, but the Ground game really slowed things down. In particular, one way of winning — the “death by ground strike” — resulted in a very slow and long game, plus a frustrating experience of helplessness for the losing player. I saw a particularly slow and frustrating game in an early play test in the first few play tests, but I played another 15 or so times, and didn’t see it again, so I started to think it was just a one-time thing, an outlier, and thereby something I could probably live with. Then, in tests 18, 19 and 20, I saw the same type of game played 2 more times. Now instead of 1 out of 20 for a slow and frustrating experience, it was 3 out of 20. That was no longer an outlier and I could no longer live with it.
So, I took another crack at the game and simplified both the Ground and Standing games, taking a lot of dice rolling out of it, and continued to test. The responses were still generally very positive, but still not super enthusiastic. Games didn’t often result in people wanting to play again. People were having fun, but they weren’t having a great time. People were picking up the game, but not all the rules until they had played several times. That’s not necessarily a bad thing for experienced hobby gamers looking for an in-depth game, but it’s all about who the game is for, and this game was for MMA fans who were casual gamers and gamers who were casual MMA fans, not necessarily hardcore fans of both, who had the patience to learn it. The hardest part of taking feedback is that you have to know that your game is never going be for everyone. Sometimes, you have to conclude that this game just isn’t for that person, but other times, you have to think, is there something I could do to get that person to like it more, perhaps remove the objection to playing again? It’s a balancing act, and I wouldn’t pretend to know exactly how to balance it.
This is where I will bring up that I’ve been told I don’t take feedback: If you take all the feedback you get, you won’t have a game at all. You’ll have a mishmash of dozens of different ideas, pulling in different directions, with nothing cohesive. You need to listen to all feedback (and I can probably do a better job there) but you simply can’t respond to all of it, nor should you. Ultimately, it’s about getting the game you want, the experience you want. It’s also not like all the feedback is the same. Do I make it simpler for the people who want it simpler? I’ve had suggestions to just make it a card game, even a fairly simple one. I could change terms like “Hook” and “Round Kick,” to “Power Punch – Head” and “Power Kick – Body” respectively. If I did make changes to reach a larger group, would I alienate the hardcore MMA fans that this game was designed for in the first place? Or do I add more complexity to make sure that I satisfy the hardcore MMA fans, and just not worry about the gamers who don’t like MMA?
There’s a certain particular balance with Cage Match! between the casual fan and the hardcore fan. This isn’t something I would have been able to crack with only 20 plays. Instead, somewhere around test 100, something else clicked. The game was good but wasn’t getting enough “Let’s play again!” responses. The first game for new players wasn’t going quickly and easily enough for people to get excited about playing a repeat. Was this just because they weren’t MMA fans? Because hey, you know, the game isn’t for everyone.
So you have to know who it’s for. And, Cage Match! is for hardcore fans of either gaming or MMA but not for the very small crowd that is deeply into both. This game needed to be more for MMA fans who weren’t going to sit down and learn a in-depth game, especially a non-card game. Many suggested a basic version of the game. We made rules for a “First Fight” that included only the Standing position, 2 fairly straightforward fighters, and no combos. Still, Dystopia Dave himself was telling me, you have to make a basic version of the game.
I’ve blogged before about what happened next: I figured out a basic version of the game, and that became the game. Out went 3 of the 5 positions, with only Standing and Ground remaining. Out went the protective layer of muscle and the muscle rating that was tied to weight class, which made it harder to knock someone out. Out went Standing moves like Protect and Focus. Out went layers of move interactions. Out went mandatory dice rolls, with dice rolls now only occurring in particular results. Out went 10-20 extra minutes per game. Out went a long learning curve with a lot going on. As a result, the intrigue game ramped up. The game’s “curve” sped up: You throw a few punches back and forth, the game is on it’s way, and you need to start planning for the end game and figuring out how to execute it before your opponent executes his. A few big moves later and the game is over. You still might need to play one game to really figure it out, but that one game will probably take less than 10 minutes to play.