Print

Email story

Comment

Email editor

In search of simplicity

To test software, Microsoft brings in real people

The thinking goes, if customers can't figure out how to make the product work, what's the point in releasing it?

By DAVE GUSSOW
Published July 25, 2005

REDMOND, Wash. - "David," "Abby," "Toby" and "Nick" hang out in Building 21, Room 1070, on the sprawling Microsoft campus. And Lisa Mason and her team watch their every move.

The people are real, though the names are just Microsoft designations to identify the type of technology consumer they represent: David is the consumer who lives and breathes digital media, Abby is the beginner but decisionmaking spouse, Toby is the influential teen and Nick is the early adopter who buys the hot new thing the minute it hits the market.

Mason uses this revolving cast of Davids, Abbys, Tobys and Nicks to gauge how well - or poorly - regular people can use the Media Center PC software, one of Microsoft's major initiatives to take the computer as an entertainment center into the living room.

Room 1070 is one of Microsoft's usability labs, though it looks like a sparse living room. A couch. A table. An entertainment center with a typical collection of devices, from a stereo to a TV to a DVD player and, of course, a computer running the Media Center PC software.

Adjoining it is a room where Mason, the user research manager in charge of testing for the Media Center PC, and her team observe the tests, typically done one person at a time over about two hours.

The observation room includes recording equipment, a TV monitor that duplicates what the tester sees on the Media Center PC and a sound system so they can hear and communicate with the person on the other side of a window.

Mason has seen thousands of test participants over the 10 years she has worked at Microsoft. That experience, she says, makes her good at predicting how people will work with the company's software.

It gives her a realistic view of a process that tries to weed out problems before a product goes on sale, but that is not perfect. "We'll never go out (to market) with zero usability issues," Mason said. "I just don't think it's possible."

Being technology, the reasons are complicated. Part of it is because so many people use the software, many in ways the company doesn't anticipate, she says. Part of it is how many people a problem may affect. Part of it is that perfecting software is an ongoing process.

Microsoft has been widely criticized for years for flaws in its products, ranging from technical glitches to security holes. In response, it has stepped up efforts to make its software more user-friendly and secure.

In 1994, Microsoft employed 22 usability engineers, tested products with 2,000 consumers and had 10 labs. Last year, it had 252 usability engineers, conducted tests with 20,000 people and had 16 labs.

Most of the people who come to the labs on the software company's big campus near Seattle live within an hour's drive. They fit Microsoft's demographic profiles for consumers, the David, Abby, Toby and Nick collection.

Microsoft researchers also visit homes across the United States and other countries to see how people work with technology, though Mason's team has not made it to Florida. People can sign up to be volunteers at www.microsoft.com/usability.)

In a typical usability session, the tester is given a list of tasks to complete, such as changing the channel while watching TV, selecting a song from the digital music library or set up a photo slide show. (While it is usually one tester at a time, there are occasions when more people, representing a family, will be present.)

Testers also are asked to talk out loud as they go through the list. That lets the researchers understand the thought process.

"This is the mental model of how people think," Mason said. "We try then to design the software that fits within that mental model so we're not forcing people to change the way that they think. If we try to change the way that they think, it's going to be a very difficult product to use."

Microsoft's usability staff tries not to get involved with the tests, finding watching more valuable. Failing a task is okay, too, because that can give better understanding of where improvements are needed.

Problems get ranked by severity and scope on a three-point scale. In Microsoft-speak, "Severity 1, Scope 1" means trouble. Big trouble.

Severity 1 means the problem causes data loss on the computer, Mason says, or people fail to complete the task the software is designed to handle. Scope 1 means everyone who comes into the lab runs into the issue. Combined, Severity 1, Scope 1 is a fatal flaw.

"We will fight tooth and nail to make sure that the product doesn't get released," Mason said.

The challenge begins with balancing design and usability. A product can look good but be unusable, Mason says. Or it's usable but has an unappealing design. And then there's the built-in Microsoft factor.

"We are very aware of terminology and we are very aware that Microsoft employees are different," Mason said. "It's easy when you're building something to build it for you, and part of my job is to say, "no don't do that.' "

Some issues are unique to specific products. For example, the Media Center PC works with a remote control, not a keyboard and mouse. That makes designing an onscreen help function more difficult.

Then there's the old bugaboo about many Microsoft products, which seem to contain everything, including the kitchen sink. Some studies show that people use only 20 percent of the Office productivity suite's functions, for example.

But there's a catch, according to Thomas Gruver, group product manager at Microsoft's Center for Information Work. Everyone doesn't use the same 20 percent. So someone may use more word processing features but not as many spreadsheet functions. Someone else may be the opposite.

That translates into how the usability engineers view problems, too, Mason says. If only one or two people have difficulties, a feature will probably stay. People want more, not less.

"Even if there is a Windows that had less in it and it's focused on one task and one activity and designed specifically for you, I think people fear that they're going to outgrow it or they're going to miss something," Mason said. "So they go for the big one, the big package, that is more difficult and more complex because of the fear that they might miss out on something."

If people have problems using Microsoft products, Mason says, it's the company's fault, not user error.

"You will never hear us say, "Oh, the user is stupid or they just need to learn it,' " Mason said. "We don't believe that. We think that we can improve and we should make it better."

There are times when Mason's co-workers have a hard time believing her when she identifies something that needs a fix.

So Mason invites them to the observation room to watch as volunteers use the software.

"This whole room will be filled with team members watching the participants saying, "Oh my God, I had no idea'," Mason said.

Depending on how far along the product has been developed, designers and engineers might go back to the drawing board, rewrite code or possibly determine that the flaw isn't severe enough to delay the rollout and can be handled in a later version.

"You can't think about a usability problem in isolation because one minor change can affect the entire product," she said.

- Dave Gussow can be reached at gussow@sptimes.com or 727 771-4328.

[Last modified July 26, 2005, 15:07:02]