upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Card Sorting: Current Practices and Beyond

Jed Wood and Larry Wood

Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 1-6

Article Contents


Issues Needing Additional Research

There are several other practices that could benefit from additional research. The following sections present a few practices that are on our agenda.

Sort just a few

Issue:

Most Web sites have sufficient content to produce very long lists for participants to sort. This, in turn, results in such a long list of domain items that it is questionable to expect participants to reliably perform the task.

Proposed research:

To shed some light on this issue, Tullis and Wood (2005) reported that useful results can be obtained by having participants sort a random sample of items, so long as enough participants are gathered to sufficiently cover the full set of items. We're interested in pushing the boundaries of these findings. What if a randomly selected single item is given to thousands of participants in a closed sort? How might the results differ from those of participants that have the context of a larger set of items?

Finding vs. sorting

Issue:

There have been some discussions on the Web about whether a sorting task is a sufficiently close analogue to the typical finding task (when people are browsing the Web) to warrant the value that many people put on card sorting as a basis for producing an information architecture. Two obvious differences between the tasks are (a) when browsing the Web users typically have a need (i.e., a mental description) they're trying to fill, and they hope to find something on the Web to fill that need; and (b) participants in a card sorting task have the context of an entire list of items when considering each one.

Proposed research:

We plan to conduct and compare results of two closed sort studies. One would be a typical study in which each participant is given a list of items to be sorted into a set of pre-defined categories. In the second study, participants would be given the list of categories but only one of the items for which to choose a category.

Take your time vs. hurry up

Issue:

It's common for us to observe researchers including phrases such as "take your time and carefully consider…" in the pre-sorting instructions provided to participants. However, we argue that users of the Web typically aren't particularly cautious when browsing, which calls into question whether encouraging card sorting participants to be cautious is appropriate.

Proposed research:

One recent study we conducted encouraged just the opposite of cautious card sorting. We encouraged participants to "sort the items reasonably quickly." Early results indicate that there isn't a substantial difference. Furthermore, it's worth considering that "reasonably quickly" is a better match for how most people browse and search on Web sites-clicking the first link that seems like it will yield satisfactory results. We plan to conduct additional studies relevant to this issue.

Multi-level sorting

Issue:

Given that all but the simplest of information hierarchies consist of more than two levels, it seems logical to allow participants to sort items into multiple levels. However, a difficulty arises during analysis. If "eyeball analysis" of single-level sorting is problematic with large numbers, then adding multiple levels exacerbates the problem. Though cluster analysis can be made to work with multiple levels, the amount of variation introduced can easily bring out the weakness of cluster analysis-the fact that it's essentially an "average."

Proposed research:

Because of the difficulty with data analysis mentioned previously, we have recommended a staged approach for obtaining a multi-level architecture. That is, first conduct a typical single-level sort, analyze to find a preliminary set of categories, and then use those categories as items in a second card sorting study. However, many researchers seem to believe it would be desirable to ask participants to create second level categories as well as first level categories all in one task. Even though we haven't yet found solutions to the analysis problems, we plan to continue to address this issue.

Previous | Next