upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Flexible Hardware Configurations for Studying Mobile Usability

Antti Oulasvirta and Tuomo Nyyssönen

Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 93-105

Article Contents

Rationale for field testing

We believe that usability studies should be conducted in the field if the key aspects of users' environment cannot be accurately simulated in a lab. For example, a student of mobile input devices may well get along with laboratory-based evaluations for most of the time. Real-world issues like timeout (due to interruptions), one-handed use (due to reserved modalities), application-switching (due to multitasking), or slow use (due to simultaneous walking) can be staged in the lab, given that they are known in advance. On the other hand, there are applications where this strategy does not work. For example, one cannot (easily) stage a whole city for a study of mobile maps or power relationships for a study of organizational use of mobile email.

We want to take this argument a bit further and present the following three conditions in which one should carry out usability studies in the field:

There have been discussions around whether "it is worth the hassle" (Kjeldskov, Skov, Als, & Hoegh, 2004); that is, whether conducting evaluations in the field pays off in terms of increased ability to capture usability problems. The skeptics have based their arguments on experiments building on contrived operationalizations of mobility. For example, being in the field has been often operationalized by walking a pre-defined route. This, in effect, reduces environment to the role of a nuisance factor-a source of disruptive events and cognitive resource withdrawals. It is no surprise that the finding has been that field testing is less efficient in capturing usability problems. Environment is not fruitfully operationalized in such a way, rather one must think how environment may support or hinder interaction according to the three abovementioned conditions. Then, one must set up the testing situation so that such events can occur in a natural way and can be captured by the recording equipment.

Previous | Next