upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Switching Between Tools in Complex Applications

Will Schroeder

Journal of Usability Studies, Volume 3, Issue 4, August 2008, pp. 173-188

Article Contents


Rating

We rate tool switching and tool switch patterns in two ways. First, switches that statistically dominate successful task execution (defined both as task completion and satisfaction with the task experience, as measured by SUS) are rated as preferred by users. Secondly, however, the choices made in the course of the work can be evaluated as regards timing (by observing whether task progress before the switch point was satisfactory) and in terms of choice (by observing whether users choose the most effective tool).

These ratings are concrete data descriptive of users' cognitive behavior, based on observation, consistent with other summative work in usability studies. Their quantity and quality are made possible by automated detections and pre-processing of all events in this category, and artificially extending (and compressing) the facilitator's observation time through the use of recordings and flagging of events. The technique remains qualitative; its foundation is the experienced observer's analysis of critical incidents. However, quantitative methods (plots and statistics) identify patterns and clustering, which suggest where those critical incidents lie.

Analysis

The analysis has four steps: choice of event types (or perhaps design changes) for study, parsing and processing of logs into statistics or graphics that arrange the data into useful views, pairing events and patterns with their metrics, and checking conclusions against test recordings. The final step connects the statistical abstraction with actual test observations, and also warrants that conclusions are not distorted by outliers.

We have already observed problem patterns prior to the study that stands out in the timelines and data. We expect to discover further orderings of subtasks (groupings or effective sequences) that, although they alter the na´ve flow of the task, are so evidently more productive that design should reinforce them. One example of such a technique is copying and pasting if, then, and else constructs as a block, and then updating the contents of the new block. Speed, accuracy, and neatness are achieved by "going out of order." These sometimes unexpected strategies are fruitful because of dependencies between subtasks that users discover and exploit (Card et al., 1983). Graphical presentation of workflows (Figures 3 and 4) help us detect patterns and dependencies.

Previous | Next