upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Flexible Hardware Configurations for Studying Mobile Usability

Antti Oulasvirta and Tuomo Nyyssönen

Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 93-105

Article Contents


Introduction

Science and technology go hand in hand. It is commonplace that paradigmatic changes in research are preceded by both conceptual and methodological advances. Of all methodological developments in usability studies, controlled laboratory-based usability evaluation may have had the most wide-spread impact on day-to-day operations. The two-room usability laboratory setup, deploying mirrors and multiple video cameras to record interaction on a desktop PC, was a necessary complement to the novel notions and operationalizations of usability that Jakob Nielsen put forward in his seminal 1993 book. Laboratory-based testing has become the de facto standard of usability practice worldwide.

However, nothing similar appears to have taken place on the side of mobile technology, although the research area has existed for over 10 years (the conference Mobile HCI started a decade ago as a workshop). One reason may be technical: no solution has been proposed that is technically reliable, cost-efficient, and flexible enough. In fact, the number of systems presented in the literature is in the order of a mere dozen. Figure 1 presents a sample of four of systems. These previous systems can be divided into three classes: those that allow recording from (a) both moderator-controlled and user-worn sources, (b) user-worn sources only, or (c) device-based sources only. (Sources are typically cameras and microphones but can also include logs collected on the phone.)

Figure 1. Previous systems (from left to right, top to bottom): Reichl et al.'s (2007) hat-worn system with moderator-controlled shooting, Schusteritsch et al.'s (2007) system for attaching minicameras to a mobile phone, Lyons and Starner's (2001) vest-worn multi-camera system, and Applied Science Laboratories' (2008) forehead camera

Figure 1. Previous systems (from left to right, top to bottom): Reichl et al.'s (2007) hat-worn system with moderator-controlled shooting, Schusteritsch et al.'s (2007) system for attaching minicameras to a mobile phone, Lyons and Starner's (2001) vest-worn multi-camera system, and Applied Science Laboratories' (2008) forehead camera

In this paper, we extend on our previous work on mobile usability labs (Oulasvirta, Tamminen, Roto, & Kuorelahti, 2005; Roto et al., 2004). We present rationale and design of a "swiss army knife approach," operating with the principle of supporting multiple functions and configurations with one system. The equipment should support any of the abovementioned three configurations if the study at hand so demands. We argue that aiming for (a) modularity, (b) scalability, and (c) flexibility is crucial if the equipment is to be used across many studies. We argue that another reason for the absence of conventions in mobile labs may be that one cannot simply transfer the thinking behind laboratory-based setups to mobile conditions. The system should support capturing environmental events and the user's position in and orientation toward the environment in general. This ability is central in the class of context-aware applications, but also relevant in any use situation where the user's environment affects interaction.

In this paper, we describe our system Attentional Resources in Mobile Interaction version 2 (ARMIv2) . The ARMIv2 system supports both belt-mounted and backpack-mounted configurations of recording devices, as well as totally wireless ones. Wires and wireless transfer can be chosen according to mobility conditions. The system also supports environmental cameras. It integrates all video into a single stream that can be uploaded to a PC for analysis after a trial. In addition, it has the longest operational duration reported and is light weight for the user. The single most important quality, however, is its support for different configurations.

In the latter part of the paper, we report first experiences from a real-field deployment. Of the three desirable qualities mentioned above, our approach is targeted towards flexibility in particular. Modularity and flexibility are desirable from the perspective of experimental validity. However, although our system works, the solution is perhaps too complex to be cost-efficient as it places heavy demands on researcher training and system maintenance. We nevertheless believe these problems can be overcome and that the general solution represents a promising direction for mobile usability labs.

Previous | Next