Usability testing on Halo 3

Wired Magazine just published an article about usability testing on Halo 3, the much-anticipated next installment in the Halo video game franchise, and the first Halo game for Xbox 360.  Video game usability has always fascinated me (and I try to get hands-on experience in this area as much as I can...), and this is the first time I've seen a mainstream magazine cover it in such detail.

Halo is a genre-changing first person shooter that brought gaming to a new level with its intricate story-line, cinematic feel and epic soundtrack.  And the creators got there through endless hours of testing...  Here are some excerpts from the article showing how they left no stone unturned:

The room we're monitoring is wired with video cameras that Pagulayan can swivel around to record the player's expressions or see which buttons they're pressing on the controller. Every moment of onscreen action is being digitally recorded.

Midway through the first level, his test subject stumbles into an area cluttered with boxes, where aliens — chattering little Grunts and howling, towering Brutes — quickly surround her. She's butchered in about 15 seconds. She keeps plowing back into the same battle but gets killed over and over again.

"Here's the problem," Pagulayan mutters, motioning to a computer monitor that shows us the game from the player's perspective. He points to a bunch of grenades lying on the ground. She ought to be picking those up and using them, he says, but the grenades aren't visible enough. "There's a million of them, but she just missed them. She charged right in." He shakes his head. "That's not acceptable."

After each session Pagulayan analyzes the data for patterns that he can report to Bungie. For example, he produces snapshots of where players are located in the game at various points in time — five minutes in, one hour in, eight hours in — to show how they are advancing. If they're going too fast, the game might be too easy; too slow, and it might be too hard. He can also generate a map showing where people are dying, to identify any topographical features that might be making a battle onerous. And he can produce charts that detail how players died, which might indicate that a particular alien or gun is proving unexpectedly lethal or impotent.

Pagulayan and his team have now analyzed more than 3,000 hours of Halo 3 played by some 600 everyday gamers, tracking everything from favored weapons to how and where — down to the square foot — players most frequently get killed.

The article goes into many more interesting examples of how they solved user issues with clever design.  Be sure to check it out.  And if you haven't seen the Halo 3 trailer yet, here it is for your viewing pleasure...