# Creating Effective Decision Aids for Complex Tasks

Caroline Clarke Hayes and Farnaz Akhavi

Journal of Usability Studies, Volume 3, Issue 4, August 2008, pp. 152-172

## Article Contents

### Methods

We used two methods in this work: ethnographic and laboratory studies. To a lesser extent, we also drew on protocol studies. We surveyed existing models of design with special emphasis on models derived from protocol studies of designers solving actual problems, because the studies provided insights into actual behavior. A
*protocol study* is one in which subjects are asked to think aloud as they solve problems. Everything they say is recorded for later analysis (Ericsson & Simon, 1980). In contrast, *ethnographic observations* are observations of work as it is carried out in a normal setting (Bloomberg et al., 2007). The ethnographic observations differ from protocol studies in that the people observed are not asked to solve specific problems or think aloud. Laboratory studies are more highly controlled than either ethnographic or protocol studies. While the situation in laboratory studies may be somewhat artificial, these studies allow measurement and quantification of phenomena in a way that ethnographic studies cannot. Thus, each of these study types, ethnographic, protocol, and laboratory studies, can provide different views of the complex phenomena associated with product design processes. Together they provide a mix of qualitative and quantitative data that allow construction of a richer overall picture than any one method alone.

### Related Literature

The following section presents the mathematical decision making methods.

#### Mathematical decision making methods

Complex decision problems require decision-makers to choose from available alternatives characterized by multiple qualitative or quantitative criteria (Saaty, 1980). Multi-criteria decision making (MCDM) techniques (Klein, 1993) are a broad family of mathematical methods that compare alternatives in a set using multiple criteria. For example, a prospective car buyer might compare his or her car choices by criteria such as fuel efficiency, cost, and comfort. The criteria used may vary from buyer to buyer depending on what is most important to that particular person. One common MCDM method is the weighted sum method (Hayes, J. R., 1981) in which each term in the sum represents how well an alternative fulfills a given criterion, and the term's weight represents that criterion's importance to the decision maker. Variants of the weighted sum method are popular because they are relatively easy to understand and use. Note that MCDM methods do not automate the decision process, nor do we view that as a desirable goal. Instead, they provide a structured approach through which people arrive at their own decisions by allowing them to specify the criteria they view as important and their judgments of the values associated each alternative.

One can further divide MCDM (and weighted sum) methods into *deterministic* and *non-deterministic* methods. Deterministic decision making methods are those that do not explicitly incorporate a representation of uncertainty, for example, the cost of an option may be represented as a specific number or point value. While the decision maker may understand that this number is not exact, the degree to which it is not exact is not represented. In contrast, non-deterministic decision making methods are those that incorporate some explicit representation of uncertainty or unknowns. For example, the uncertainty in the cost of an option may be represented as a range of possible costs or as a function describing the likelihood of various costs.

Vagueness and ambiguity can be modeled by many techniques including those based on fuzzy set theory (Thurston & Carnahan, 1992). The merit of fuzzy techniques is that imprecision (Bellman & Zadeh, 1970) is recognized as an element of the decision model. The drawbacks of such techniques are the relatively high computational effort required for modeling the decision situation and processing the input information (Law, 1996). However, while much research has focused on the development of formal decision making methods, relatively few studies have assessed their practical utility and impact in complex tasks. In the laboratory study summarized later, deterministic and non-deterministic (fuzzy) decision making methods were compared against designers' typical, informal methods.