upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Usability Testing with Real Data

Alex Genov, Mark Keavney, and Todd Zazelenchuk

Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 85-92

Article Contents


Usability testing is generally well equipped to help design teams identify issues with a wide range of products and systems, frequently identifying interaction problems that were initially overlooked in even the simplest of proposed design solutions. In certain instances, however, usability testing falls short of producing results that are both reliable and valid. Of particular concern are studies of products in which users heavily interact with their own personal data, for example a user test of an email or calendaring program. In these cases, the fact that data are often fabricated for the test (and are therefore unfamiliar and possibly unrealistic for any given user) can affect users' abilities to recognize, interpret, and interact with these data in an authentic manner. Thus, usability practitioners run the risk of identifying false positives (for example, if users are completely confused by a screen because they're unfamiliar with the data in it, when in the real world they would have been able to use their own familiar data to orient themselves) or overlooking real problems (for example, if users answer questions casually about their fake data, but would be much more concerned with the meaning of items when it applies to themselves). In this paper, we examine the strategy of incorporating users' real data into usability testing to avoid these issues and increase the validity of a study.

Previous | Next