upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Switching Between Tools in Complex Applications

Will Schroeder

Journal of Usability Studies, Volume 3, Issue 4, August 2008, pp. 173-188

Article Contents


Background

Kuutti says, "[The focus of] mainstream HCI research is narrow, covering most adequately the area of error-free execution of predetermined sequences of actions" (1996, p. 37). Theoretical formulations that describe users' engagement with complex tasks have not been coupled with workable evaluative techniques.

Usability testing limitations

Literature on testing of tool switching, because of its specific focus (multi-tasking) and limited scope (complex software), is sparse. To one side is conventional usability work (Dumas & Parsons, 1995; Rubin, 1994), and academic studies (Chapuis & Roussel, 1999; Card and Henderson, 1987), in which the analysis of simple tasks remains within the production line paradigm. This serves because workflows are known and tool options limited or equivocal. To the other side we find ambitious projects (Kline & Saffeh, 2005; Singer, Lethbridge, Vinson, & Anquetil, 1997; Weiderman, Habermann, Borger, & Klein, 1986) proposing to manage the entire software engineering life cycle. These papers are non-starters. They recognize the need for this kind of study without proposing strategies, let alone techniques, to gather and manage the enormous amount of data and observations that User-Centered Design requires.

Studying user-driven workflows is daunting. "Real" multi-tool tasks have an untestably large number of valid workflows, branching with every opportunity for tool change. Complexity of creative tasks and idiosyncratic work practices pull together the components of integrated development environments (IDEs) into what are, in effect, unique configurations each time they are used (Sy, 2006; Redish, 2007). Reported tests (Hanson & Rosinski, 1985; Leich, Apel, Marnitz, & Saake, 2005) treat IDEs as single entities with fixed workflows by evaluating them with simple tasks despite the fact that each user applies the components differently.

Tool choice and switch timing, when they are considered at all, are taken as either an outcome of training or acquired expertise (Card, Moran, & Newell, 1983; Kuutti, 1996), or the result of the user's awareness of the task and the environment (Engestrom, 1999; Suchman, 2007). To date, neither approach has produced results that nourish effective design strategies for toolkits, which is what we are after.

Theories (Bødker, 1996; Engestrom, 1999 Green & Petre, 1996; Kuutti, 1996; Suchman, 2007) attempt to clarify the problem by more subtle description of what we observe without sufficient data and concrete examples to make them usable for the practitioner. This body of work sustains itself on elaborate schemas and thoughtfully studied examples that have undeniable explanatory and heuristic power. Unfortunately, when schemas conflict, debate collapses into whose example is more relevant.

Activity theory maintains that experienced users move unconsciously from step to step in a learned process until a problem (contradiction) intrudes, at which point the user is open to change and improvement of the process. Suchman and others suggest that step choices are "situated", that is, based on factors in the current problem state. Tension between these views remains because neither has been tested to determine the extent and actual conditions of its validity. Klein (1999), in a series of detailed interview studies of situated decisions and actions, gives evidence that the situation driven thinking described by Suchman is typically evoked by emergency or crisis situations (contradictions that cause process breakdowns). Klein's detailed study, drawing from many subjects engaged in widely different complex tasks, suggests that these two stances may be the same underlying behavior observed under different conditions.

The question of whether the "layered task execution choices" (Wild, Johnson, & Johnson, 2004) of this planning, monitoring, and decision making work can be supported, at least in part, by a user interface has also not been systematically tested. If transitions are marked and measured, their effects can be separated from the effects of the tools, and both can be studied in the context of complex tasks. The framework and vocabulary used by activity theory to describe how workflows change over time with experience provides a workable starting place. The issue of tool switching needs this depth of information to drive the design of better tools and toolkits.

Theoretical formulations of complex tasks

Activity theory calls the use of a tool an operation (Card et al., 1983; Kuutti, 1996). For the novice, an operation is a discrete series of actions, each of which must be, at first, chosen and planned. With practice, the need for planning and choice diminishes, and a chain of actions becomes one smooth operation. A tool works well if it allows productive work on an object. Two types of events take users out of the flow of productive work:

[Breakdowns] occur when work is interrupted by something; perhaps the tool behaves differently than was anticipated, thus causing the triggering of inappropriate operations or not triggering any at all. In these situations the tool as such, or part of it, becomes the object of our actions…. A focus shift is a change of focus or object of the actions or activity that is more deliberate than those caused by breakdowns…. Now the operations that she normally does become actions to her… (Bødker, 1996, p.150).

Breakdowns and focus shifts, although different in character, result from contradictions:

Activity theory uses the term contradiction to indicate a misfit within elements, between them, between different activities, or between different development phases of a single activity…. Activity theory sees contradictions as sources of development … (Kuutti, 1996, p. 34).

These are opportunities for changing tools and opportunities for learning and, potentially, opportunities for reworking and improving the chain of actions that activity theory terms the operation.

Previous | Next