upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Unexpected Complexity in a Traditional Usability Study

Tharon W. Howard

Journal of Usability Studies, Volume 3, Issue 4, August 2008, pp. 189-205

Article Contents


Users' task environments can be improved with scenarios

The failure of both the Hacker excerpt and DK prototype on the works cited task, and users' difficulties with commas, can all be partially attributed to incomplete understandings of users' goals and task environment. It is obvious that improving an author's understanding of the users' needs is likely to result in more usable information products, hence the mantra "Know Thy User." This is well understood.

What is often less well-understood is the role that authors play in actually constructing the use or task environment for users. All too frequently, we tend to think in terms of accommodation of users' needs, and we tend to overlook the important role that authors and designers play in the construction of users' task environment. Indeed, the movement away from theories of "user-centered design" in the 1990's toward "user-experience design" is largely a recognition of this complex negotiation between accommodation of users on the one hand and creation of user-experiences on the other. Successful texts and information products create roles and provide interpretive frameworks that users can deploy in order to successfully complete tasks and achieve their goals.

The evaluating source section of the DK prototype did this fairly successfully, and it did so mainly by using "If, Then" scenarios that users could play in order to understand some of the criteria that need to be taken into consideration when deciding whether or not a source is credible and relevant (note: see Flowers, Hayes, and Swarts's 1983 article "The Scenario Principle" for more details about this approach). Rather than attempting to provide linear, step-by-step procedures for judging the value of a source, the DK prototype used mini-stories to exemplify how the complex decision of whether a source is relevant can be made. In one of these "stories," students named Pedro and Aaliyah are described. Pedro is writing a research paper for his class about social actions the U.S. ought to take to prevent pandemics. Aaliyah is researching viral marketing techniques for a marketing company. Pedro and Aaliyah are both asked to judge whether an online news story about how viruses spread electronically. The handbook guides users through the characters' decision making processes to show why the same text is not appropriate for Pedro's situational context on the one hand, and completely relevant to Aaliyah's on the other. This section creates a clear role for users to play as they use the text and it also acknowledges the importance of context on decision-making, which the other sections of the handbooks did not adequately address or address at all. And because the users understand the role they are supposed to play, they also understand the logic they're supposed to follow in order to make the decisions necessary to complete the task at hand.

This did not pertain, however, for either handbook in the MLA sections. Both texts failed because both asked users to play roles they could not adopt. Both texts assumed, erroneously, that users begin the citation process with a clear understanding of the types of works being cited and thus had a simple problem of choosing the format that matched the type of work. However, the users in this study did not understand the context and so didn't differentiate between types of works. Generally speaking, the users in this study didn't know, for example, what a corporate author was, and two of the users decided that a collection of readings was "a reference book." If you, as a user, don't know what a corporate author is, you're not going to be able to use the information on Figure 3 of the DK prototype or the tables on pages 341 and 349 in Hacker's handbook.

When users don't know what decisions they need to make (e.g., how do I decide if I have a corporate author, government author, no author, etc.), then constructing a task environment like the one created in the evaluating sources section is a positive approach. The presentation of the information in terms of "agencies" with whom the students and users could identify helps users understand where to begin the decision-making process. The authors should also ask questions as headers, such as "How do I quote or paraphrase in my text?" or "How do I format an entry for a works cited, reference list, or bibliography?" This would create the same "context of use" found in the evaluating sources section, and it allows the authors to build a decision matrix or some other tree-type structure that could be used by students and teachers to make decisions about what type of text they have so that they can make an informed judgment about the appropriate works cited pattern to use.

Previous | Next