upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Examining Users on News Provider Web Sites: A Review of Methodology

William Gibbs

Journal of Usability Studies, Volume 3, Issue 3, May 2008, pp. 129-148

Article Contents


The following section presents the usability testing procedure.

Usability testing procedure

Marchionini (1995) observed that there are three general types of browsing: (a) directed occurs when browsing is focused, systematic, and aimed at a target; (b) semi-directed occurs when browsing is generally purposeful but not highly systematic and the target is less definite; (c) undirected occurs when browsing is not focused and no goal has been specified. In this project, all participants performed usability tests on the six news provider Web sites. They received an information seeking task that, using Marchionini's classification, was either directed or semi-directed. For each of the six sites, those given the directed task had to locate two (top) news stories about an environmental issue and to browse to the end of the story. In addition, they had to find (a) when the story was last updated, (b) an author name, if one existed, and (c) an email link to email the story to someone. Participants given the semi-directed task viewed the same six sites and they had to (a) browse each of the sites freely, (b) locate stories of interest, and (c) browse to the end of each story.

Usability tests were conducted individually and the test order determined the task assignment. For example, participant one received the directed task, participant two received the semi-directed task, participant three received the directed task, and so on. In two instances, this ordering varied and the participants were given the semi-directed task instead of directed. The sequencing of the site presentations was counterbalanced to reduce order effects.

To capture participants' thoughts while performing tasks, a moderator asked them to think aloud. Think aloud protocols are often used in usability testing because they provide valuable insight about user reactions, thoughts, and experiences. They have potential to afford much insight into the strategies participants use as well as their behavior during task performance (Jenkins, Corritore, & Wiedenbeck, 2001). Ericsson and Simon (1996) point out that verbal behavior is one form of behavior from which observations and analyses can be made.

During all tasks, a video camera connected to the participant's computer recorded facial and verbal expressions. In addition, the Morae software program recorded all on-screen events (mouse clicks, keystrokes, screen changes) and integrated these data with the video and audio into a single digital file for later analysis. After each task, participants exited the browser and then opened the next site. The entire session lasted for approximately one hour.

In combination with a think-aloud protocol, the following measures and approaches were used to collect data:

The last category, "Information seeking behaviors," included five approaches for observing behaviors, each of which is discussed more fully in the following section.

Previous | Next