upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

A Modified Delphi Approach to a New Card Sorting Methodology

Celeste Lyn Paul

Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 7-30

Article Contents


Using the Modified-Delphi card sort may reduce the financial and cognitive costs of a study. I have investigated if the results from this method are at least as good as results generated from Open card sorting study. If the results cannot provide the same value to the researcher, the cost benefit is lost.

This brings me to my hypotheses:

In the approach I have chosen to answer my research question, I have directly compared the Modified-Delphi card sort and Open card sort by conducting parallel studies in a laboratory environment using the same user configuration and card collection to generate result structures. I use the term information structure, rather than information architecture, because these are generated from the results of the studies without any modifications based on heuristics, logic, or experience. The information structures are a representation of the results without the assistance of an expert. Two independent studies were conducted: (a) an expert heuristic review and ranking by information design experts (b) and an Inverse card sort with the website's target user groups. The results of these studies have provided data for directly comparing the Modified-Delphi card sort versus the Open card sort.

The University of Baltimore Usability Lab conducted a usability study of a new website design for the University of Baltimore School of Law in the Fall of 2006. Results from the usability study revealed issues relating to the information architecture, and consultation with an information architect was recommended (Nomiyama, Abela, & Summers, unpublished). This presented an opportunity to conduct parallel studies using the Modified-Delphi card sort and Open card sort for direct comparison and to test the new method with a real-world problem. Using a dataset such as weather or food may not have produced realistic results. These kinds of information have strong preexisting social constructs that define how they are categorized, conventions that are learned early and are hard to break.

Previous | Next