- Test Plans at a Glance
- Creating Test Plans
- Presenting Test Plans
- Test Plans in Context
This chapter is from the book
This chapter is from the book
This chapter is from the book
Test Plans in Context
Using a Test Plan with Other Documents
Usability testing is often treated like a self-contained activity, despite its impact on many other design activities. The test plan itself, as a foundation for testing, may seem detached from other documents, but you'll need to think about the far-reaching implications of your test, and how the test plan can anticipate those implications.
Test Plans and Other User-Needs Documents
User-needs documentation—at least in the context of this book—is any documentation that contributes to or comes from research into your target audiences. The test plan is a keystone in design methods because it is the mechanism for testing hypotheses, the means for answering the basic question, "Does this work?"
There are two other user-needs documents described in this book—personas, profiles describing the target audiences, and usability test results reports, which document what happened during the usability test. These are appropriate bookends for the test plan.
User Personas and Test Plans: User personas—the profiles of your target user groups—help you set up the test plan in two ways. From a very practical point of view, the personas tell you whom you must recruit to participate in the usability tests. The descriptions can form the foundation of your screener. Personas can also help structure the scenarios used in the test. Remember: The purpose of a usability test is to see how well a site design supports user needs. The scenarios in your test should be driven by what's documented in the personas, not in the site's functional requirements. One of the examples earlier in this chapter described a user group for a parenting web site, the Day Job Parent. This was drawn from (in our imaginary example) the personas created for the project. The script for the test would include scenarios that such a user might encounter:
Scenario 1: Quick Look-Up at Work. On your commute to work, you recall a conversation you and your spouse had last night about your daughter. You'd talked about how her grades had been slipping at school. You decide to take a moment on your lunch break to investigate possible causes.
Scenario 2: In-Depth Research at Night. After the kids have gone to bed, you and your spouse get together at the computer so you can share your research from earlier in the day. You're hoping to get further information about some of the causes you identified.
Test Results and Test Plans: The yang to the yin of the test plan, test results document the observations and conclusions from the test itself. Without test results, there's no point to the test plan, and without a test plan, you'll never get useful results. There are some direct parallels between the plan and the results: The objectives will hopefully not change, though you may have to adjust the method during the course of the test. The test results should build from the same objectives and account for any methodological changes. At the same time, when it comes to the script there may not be a direct correlation between the plan and the results report because your observations during the test may need analysis and interpretation that takes them outside the scenarios defined in the plan.
Test Plans and Strategy Documents
Strategy documents capture essential information for the design process, but do not describe the design itself. This book describes three strategy documents—the competitive analysis, the concept model, and the content model—and anyone would be hard pressed to find direct relationships between a test plan and these documents. This isn't to say that usability testing isn't strategic, just that the kinds of things in the test plan aren't driven by the contents of any of these strategy documents. Even the test objectives, which might seem the most strategic, should focus on what you want to get out of the test, not goals for the entire project.
Test Plans and Design Documents
The purpose of design documents is to capture ideas for the design of the web site. Documents like site maps or wireframes show different aspects of the web site's design from the user's perspective. Their relationship to the test results is obvious—the observations made during the usability test will strongly influence the design of the web site. The relationship between a test plan and design documents is a little more complicated because the design documents may be the things that are being tested.
Some design teams, if they don't have a working web site, will put design documents in front of users to gauge their reaction. The thinking here is that testing a representation of the final product is the next best thing to testing the product itself. If you employ a similar method, the test plan needs to be tied closely with the design documents to describe what kinds of questions you'll ask users. You might, for example, put a site map in front of someone and ask him or her to navigate to a particular section. In testing the screen design for a homepage, you might ask what features jump out at first glance. Wireframes documenting a checkout process can be an effective tool for testing the overall flow of the interface. Using paper to do usability testing is another science (or art form) entirely and may require supplementing the test script to provide further direction to the facilitator. There are a handful of books on this subject, including the widely cited Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces, by Carolyn Snyder. In lieu of a working web site or prototype to test, design documents are the next best thing for soliciting feedback from end users. Compared to a full-fledged web site, building design documentation is quick and easy, which means that site maps and wireframes can increase the amount of audience participation in the design process. On the other hand, it can make the test planning process more complex.
The Future of Usability Testing
The test plan is a stand-in for the test itself, a representation of the test without being the test. So, when we talk about why test plans are important we're really talking about whether usability testing itself is important. (The sheer audacity and recklessness of wondering whether you can do a test without any planning makes that scenario impossible to consider.)
Can we imagine a design process that doesn't include any sort of testing? Will the backlash against the usability gurus be so great that designers begin to wonder why they ever bothered with testing at all? For many a designer, a world without usability testing may seem like a world without constraints, a world that thrives on innovation and creativity without compromise. In this world, of course, the designer rules.
But design is creativity with a purpose. By its very nature, it's not arbitrary or detached from a host of criteria. A good design meets a need, and does so better than any other product that's tried to fill that need. A good design fits effortlessly into those criteria without calling too much attention to itself (unless that's what the design is supposed to do). But knowing your constraints and designing to your constraints are two different things. The design process is, out of necessity, a dance between what could be and what must be.
For the web, this means putting designs in front of the people who will be using the site and checking that the design does what it needs to do. The gold mine is innovation inside the context of use, finding a novel way of doing something that immediately clicks with users. The crowning achievement of a design is the exclamation of an enthused user: "This is awesome!" (The exact exclamation will depend on your target demographic.) Without usability testing, however, we designers would never have occasion to hear these outbursts of joy that come from using a well-designed site.