Tutorials

Finding usability defects and problems early, efficiently, and cost-effectively is an ongoing challenge for usability professionals. One successful method for usability quality assurance is that of the collaborative usability inspection. This tutorial will teach participants the details of how to conduct such inspections, from writing effective inspection scenarios and choosing the participants to conducting the inspection and making use of the results.

PARTICIPANT KNOWLEDGE AND EXPERIENCE EXPECTED

This tutorial is aimed at practicing usability professionals however students and professors in the field should also find it of interest. Participants should have some basic familiarity with the concepts of usability lab testing, usability walkthroughs, heuristic evaluation methods, and basic usability heuristics and principles of good design. Familiarity with task cases, use cases, user roles, personas, and scenarios is also desirable.

GOALS FOR THE SESSION:

Participants in this tutorial will:

• Get hands-on practice in writing effective inspection scenarios
• Learn the preparation needed for a successful inspection, including who to include and who not to include on the inspection team
• Gain hands-on practice in running a collaborative usability inspection
• Learn how to manage the inspection process for maximum efficiency and effectiveness
• Find out how to create an inspection team environment that is positive, non-threatening, and friendly while being effectively critical and detailed
• Learn how to use the results of the inspection for maximum gain in terms of usability quality and overall process improvement within the organization
• Gain an understanding of the comparative benefits and risks of related methods of usability defect detection including lab testing, walkthroughs, heuristic evaluations, and collaborative prototyping.

HOW THIS TUTORIAL WILL BE CONDUCTED

The tutorial will be a combination of lecture with supporting PowerPoint presentation, small-group work based around two case studies to allow for direct practice and experience with the techniques described, and full class discussion. It is expected that at least 50% of the time will be spent in participatory activities of exercises and discussion. The hands-on work will involve the use of case study handouts, index cards, live web-site viewing, and instructor-supplied forms.

OUTLINE AND DETAILED DESCRIPTION OF TUTORIAL

Introduction and Overview

Introduces the challenges of finding usability defects early and cost-effectively within the overall development process and within the constraints of real world organizations with limited time, limited resources, and limited budgets.

Class discussion of participant experience in this area with flipchart capture of challenges encountered by participants. Brief overview of the tutorial structure and format.

Survey of Common Methods for Usability Defect Detection

Review of commonly used methods for usability defect detection Including lab-based usability testing, field-based usability testing, cognitive walkthrough techniques, heuristic evaluation techniques, and persona-based walkthroughs highlighting the strengths and weaknesses of the different methods.

Class discussion and flipchart recording of the participants' own experiences and observations about the success, strengths, and weaknesses or drawbacks to these different methods.

Introduction to Collaborative Usability Inspections (CUIs)

Covers the basics of what is a Collaborative Usability Inspection, what are its strengths, how does it fit into the larger development process, what type of design artifacts can be inspected, how does it differ from the other methods just reviewed.

Defining and Recording Usability Defects During a CUI

Looks at how usability defects are defined and recorded within the CUI context including issues of defect granularity, defect severity and defect identification. Reviews the underlying UI design principles and heuristics that guide defect definition within the CUI process. Addresses what information we need to capture about each defect and how that is information is recorded within a CUI including forms, tools, tips and techniques for effectively capturing the defect information without slowing down the inspection process or losing important information.

Inspection Team Participants

Who to include on the team from within the organization and who not to include on the team, how to choose representative end-users

Inspection Team Process Roles

Definition of the different process roles used within a CUI including lead reviewer, recorder, end-user participants, and continuity inspector.

Preparing for a Collaborative Usability Inspection

What makes for a good inspection scenario and what are the potential pitfalls( with examples)? How to inspect large applications and sites or complex systems. How to develop effective inspection scenarios from task cases, use cases, or other system analysis and requirements materials including discussion of examples.

Exercise: Writing Effective Inspection Scenarios

Includes time for group formation (class is divided into as many 6-8 person groups as warranted by overall class size), group introductions (members of each group introduce themselves to each other). Groups are given collections of task cases within the case study problem and must construct useful test scenarios from the task cases provided.

The completed scenarios will then be reviewed and discussed by the class as a whole.

Conducting a Collaborative Usability Inspection

How to enact scenarios depending on the type of design artifact being inspected (e.g. paper prototype, working prototype, wire frame), managing the organizational aspects of the inspection including making end-users feel welcome, pre-inspection team-building, process rules for CUI success, keeping the inspection process efficient, managing the inspection participants.

Exercise: Collaborative Usability Inspection of a Live Web Site

This exercise walks the whole class through a collaborative usability inspection of a live web site using an informal version of the CUI process. This exercise is designed as a warm-up and practice for the longer small group exercise which follows. Following this practice group inspection the class will discuss the experience and any questions or issues that arose.

Exercise: Collaborative Usability Inspection Using Case Study

In their small groups, participants conduct usability inspections based on the case study and using supplied paper prototypes and defect recording forms. Team members assign process roles and run each team's collaborative usability inspection including the recording of all usability defects found and post-inspection analysis of the inspection data. During the actual exercise the instructor will circulate among the teams to answer questions or coach the teams as needed. Each team will briefly present its inspection results and the whole class engages in a post-inspection debriefing concerning the process and the participants' experience with it including problems or challenges encountered.

Potential Risks and Challenges to CUI Success

Discussion of the most common problems that can lead to less productive or unsuccessful CUIs and how to avoid them.

Post-Inspection: Making Use of the Results

Covers post-inspection activities including ways of logging the final inspection results, how to prioritize the results, how to translate the inspection results for development managers and upper management, how to leverage the results for organizational learning and ongoing process improvement.

Discussion and Wrap-Up: CUIs Within the Development Cycle and As Part of the Usability Professional's Toolkit

Class discussion on how to integrate inspections into the development process, the organizational challenges of inspections, successfully leveraging the inspection results, and CUIs compared to other usability defect finding practices. What defect detection methods are best suited to what purposes? When and why would other methods be useful in addition to or instead of a collaborative usability inspection approach?

Review of the challenges, issues, and experiences collected from the discussions at the beginning of the day. Does the CUI method for usability defect detection address any of these challenges and concerns? What challenges remain? Where is more research needed or better tools? What are the organizational impediments to effective usability quality assurance and process improvement?

SPEAKER
Lucy Lockwood

President

Constantine &Lockwood, Ltd.

Lucy Lockwood, President of Constantine and Lockwood, Ltd., has spent well over a decade teaching and consulting in the U.S., Europe, and Australasia on software and web usability and on the integration of UI design and usability within mainstream software engineering practices. Co-author of the award-winning book on usage-centered design, Software for Use (Addison-Wesley, 1999), she helped develop many of the core concepts and techniques of the usage-centered design process.

Drawing on years of experience in programming and project management, she has always focused on techniques for producing significantly better software from the end-users' standpoint while addressing the needs and constraints of real world development organizations and processes. A speaker at major conferences worldwide, her work includes user interface analysis and design, expert evaluation and usability quality assurance, and design process development and integration for leading companies in the U.S., Europe, and Australia covering traditional and web-based applications, web sites, and embedded systems projects over a broad range of industries.