Conducting Iterative Usability Testing on a Web Site: Challenges and Benefits

Jennifer C. Romano Bergstrom, Erica L. Olmsted-Hawala, Jennifer M. Chen, and Elizabeth D. Murphy

Journal of Usability Studies, Volume 7, Issue 1, November 2011, pp. 9 - 30

Abstract

This paper demonstrates the benefits and challenges of working collaboratively with designers and developers while conducting iterative usability testing during the course of Web site design. Four rounds of usability testing were conducted using materials of increasing realism to represent the user interface of a public government site: 1) low-fidelity paper prototypes; 2) medium-fidelity, non-clickable HTML images; and 3) and 4) high-fidelity, partially-clickable Web pages. Through three rounds of usability testing, usability increased, but in the fourth round, usability declined. Iterative testing enabled evaluators to collect quantitative and qualitative data from typical users, address usability issues, and test new, revised designs throughout the design process. This study demonstrates the challenges and value of working collaboratively with designers and developers to create tasks, collect participant data, and create and test solutions to usability issues throughout the entire cycle of user-interface design.

Practitioner’s Take Away

We realize that some of these take aways apply to all usability tests but we have included recommendations that are especially important for iterative testing.

• Encourage having an advocate on the management team who is a liaison between the usability team and the design-and-development team. If this is not possible, educate the design-and-development team every step of the way about usability.
• Take the initiative to set an expectation from the beginning that there will be iterative tests and that all of the key stakeholders will be part of the process.
• Acknowledge that schedule, cost, scale, and technical constraints of a project will influence decisions to make enhancements that are beyond the control of the design and development staff.
• Request early participation in requirements gathering activities.
• Start with paper prototypes, before any hard coding has been made, so developers can get on-board with the design-test-refine process.
• Plan to include some tasks that can be repeated in all of the iterations so that the team has a measure of progress as they proceed.
• Collect quantitative measures that can be repeated over the iterations.
• At the end of each session, discuss the usability issues and possible fixes with the observers; by doing so, get them into the mindset of anticipating modifications to their design when they are still willing to make changes.
• Encourage developers, project leads, and programmers to attend usability sessions as observers. Review the major findings and observations following each session.
• Have regular, ongoing discussions about the findings and recommendations with the design-and-development team.
• Give documented results to the development team as soon as possible, within a week or two of the last session, while the issues they observed are still fresh in their minds.
• Use iterative testing to highlight where usability recommendations work as well as to uncover instances where usability recommendations create new problems.

Article Contents

• Introduction
• Project Structure and Procedure
• Iteration 1 – Conceptual Design
• Iteration 2 – Cycle 1: Search and Navigation
• Iteration 3 – Search and Navigation Plus Core Functions Available
• Iteration 4 – All Functions Plus Help Available
• Discussion
• References

Download Article

Conducting Iterative Usability Testing on a Web Site: Challenges and Benefits