upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Flexible Hardware Configurations for Studying Mobile Usability

Antti Oulasvirta and Tuomo Nyyssönen

Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 93-105

Article Contents


To demonstrate how varied the requirements for a mobile usability lab can be, and as a further argument for the swiss army knife approach, let us analyze the following three evaluation scenarios:

For the first evaluation scenario, a setup that records only the user interface might be enough. The two latter scenarios require at least some track of what happens in the user's environment. The third scenario is the most demanding in this respect. Ideally, it requires systematic second-by-second analysis of both bodily (turning of body, deployment of gaze, use of hands; analyzed post-experimentally from video tapes) and cognitive (verbal protocols) strategies, in addition to analysis of events at the interface and in the environment. Experience with several analogous studies of mobile systems has helped us gain perspective to mobile usability labs and spell out the following general goals:

Let us explain the background for these requirements. First, we took as our starting point that the core of mobility is that it foregrounds the changing relationship between the user and the environment. It raises new constraints and resources for interaction from context, and it allows users to involve and utilize new contexts. For the recording system it means that it should be able to record any significant action or event in context. In practice, depending on the situation, this may include near bodyspace of the user, distant and proximate physical objects that a user interacts with, ad hoc environmental events, as well as deployment of gaze. The view that the core of mobile human-computer interaction is in the triad environment-user-computer demands flexibility in the placement of cameras as well as their division among the environment, the user, and the device.

Second, a threat to experimental validity is that the system itself affects the phenomenon under study, for instance due to its (a) physical qualities-e.g., the weight of the system causing fatigue, (b) ramifications to the user's processing of the interface-e.g., a camera occluding the mobile device, or (c) social consequences-e.g., a hat not being acceptable indoors. Camera types, positioning, and form are crucial qualities that affect these problems. Another factor is the moderator. The presence of a moderator may also affect events and is not always desirable. This is yet another argument for modularity and flexibility for the selection of cameras and their positioning. The cameras should enable researchers to conduct studies without a moderator, preferably without sacrificing the ability to record in an environment. One solution is the utilization of surveillance cameras.

Third, the environment is not only of interest as such, but it also introduces noise, unexpected events, and it is the cause of technical unreliabilities. For example, due to a user walking very quickly, it may happen that the moderator is unable to reliably capture events in the front bodyspace of the user. These problems call for (a) redundancy in recording and (b) online quality control. The former can be addressed if the moderator can place "just in case" cameras to augment and back up the primary ones. The latter can be supported by providing a real-time copy of the A/V stream for the moderator.

Flexibility in practice

To address these requirements, we aimed for flexibility in the following four qualities of the system:

Previous | Next