upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

The Usability of Computerized Card Sorting: A Comparison of Three Applications by Researchers and End Users

Barbara S. Chaparro, Veronica D. Hinkle, and Shannon K. Riley

Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 31-48

Article Contents

Study 2: Usability and the End User

Study 2 examined the usability of the same three card sorting applications from the end user perspective. The term end user is used to describe the participants of a card sort activity. These users typically know little, if anything, about card sorting but are representative of the target audience for the information being sorted. They are typically asked to group the information reflected on cards in a way that makes sense to them.


The following sections provide information about the participants, materials, and procedure used in this study.


Eight participants, four male and four female, ranging between 23 and 43 years of age (M= 29), volunteered for this study. Participants were recruited from undergraduate classes at a Midwestern university and its local community. Seventy-five percent of the participants reported using computers for school related activities, communication with others (not including e-mails), and using the Internet, and 87% reported using the Internet for education and gathering information for personal needs.


The programs, perceived difficulty, and satisfaction surveys used in this study were the same as those used in Study 1. In addition, three 35-item lists served as the items to be sorted in the open card sort. The lists were composed of (a) names of zoo animals, (b) names of colors, and (c) names of fruits and vegetables. Pilot tests were conducted to insure the lists were of equal difficulty to sort. The 35-item lists were representative of what may be used in a small card sort exercise. This size was chosen so that each user could complete a sort with each program in a single one-hour session.


All participants were asked to complete a background questionnaire regarding their computer and Internet habits. They were then asked to complete a series of four tasks representative of those users typically perform in an electronic open card sort. Participants completed an open card sort with each application with the tasks as follows:

  1. Sort the cards into groups.
  2. Name the groups.
  3. Move any two items from any group(s) to any other group(s).
  4. Indicate through the program that you are finished with the sorting session.

All participants completed the tasks for all three programs. The tasks were presented in sequential order while the order of the card sort programs and the prescribed card list were counterbalanced across participants. After each task, the participants were asked to provide a difficulty rating (1 = Very Easy and 5 = Very Difficult) of that task. After all tasks were completed for a program, participants were asked to complete a satisfaction survey (Brooke, 1996) and discuss what they liked and disliked about the card sort program. Measures of task success and time-on-task were collected for each task. After completing all tasks with all applications, participants were asked to rank their preference of the programs.

Previous | Next