Tutorial Details

By Day - Selected
By Presenter
By Track
View My Agenda

Boot Camp: Preparing and Conducting Usability Tests

Tutorial for newcomers to usability testing who want to learn how to conduct usability tests

Preparing for and conducting a usability test can be intimidating, but it doesn't need to be. Learn practical strategies to successfully plan, prepare materials, and conduct your test, to get the most useful results. Tutorial includes activities to help participants become more comfortable with the entire process of usability testing

Tutorial by Thyra Rauch (IBM), Carol Smith (Midwest Research, LLC)
Usability Fundamentals (UF)
8:30am to 5:30pm on Tuesday, May 25, 2010

About the Tutorial

DETAILED DESCRIPTION OF TUTORIAL 1. Introduction to Tutorial (15 minutes) Instructors will introduce themselves and present their background and experience in usability testing. The agenda for the day will be presented. Instructors will ask the audience questions to gauge the overall experience level of participants as a guide for the rest of the day.

  1. Interactive lecture: Getting Started (20 minutes) Brief discussion about scoping the project and designing the test. The importance of creating a test plan and sharing it with the team and managers will be stressed. We will also discuss test roles for the testing team, resources and communication. Scenario creation will also be discussed.

  2. Interactive lecture: Measurements and testing protocols (35 minutes) While measurements and protocols are part of “designing the test” and not a major focus of this tutorial, it is important to mention them in the context of usability testing. A set of sample measurements will be discussed and as well as a set of methodologies or protocols such as think-aloud, active intervention, co-discovery and retrospective analysis.

  3. Activity 1 ((20 minutes) Participants will break into small groups and select an item to test. They will then do a brief discovery session to determine what the stakeholder’s needs and goals are and create high level scenarios for testing. The entire group will then come together to discuss the activity.

  4. Break (30 minutes)

  5. Interactive lecture: Planning test logistics (30 minutes) Scheduling the test and preparation that needs to be done prior to the participant’s arrival will be discussed including running a pilot study. We will cover the base needs for usability testing and also discuss what facilities are available and how a fully appointed lab differs from portable or semi-portable labs. Items that are specific to international testing and remote tests will also be pointed out.

  6. Activity 2 ((25 minutes) Building on the previous task, participants will create a screener to recruit for the test they would need to facilitate on the item they selected in Activity 1.

  7. Interactive lecture: Selecting and preparing materials (25 minutes) Writing a test script, preparing participant folders, non-disclosure agreements, consent forms, data collection forms, tasks and other materials will be discussed. The background and ethics behind consent forms will be presented in this section further reinforcing a test conductor or facilitator’s important responsibilities. A sample of all materials will be supplied in the tutorial handouts.

  8. Wrap-up and questions from the morning (10 minutes)

  9. Lunch break (90 minutes)

  10. Interactive lecture: Interacting with test participants (30 minutes) This is a key point of the tutorial. The importance and duty of making the user feel welcome and comfortable will be discussed. Bias in facilitator behavior and in questioning and interacting with users will be presented. Cultural sensitivity will also be addressed. This will lead into the exercises in the next section.

  11. Activity 3 (25 minutes) Participants will be given a set of potential interactions and asked to critique them and offer an unbiased alternative as a way of learning best practices for moderation and facilitation. Examples will include how to ask questions in an unbiased way and how to probe users for more information. After discussing these, a second set will provide even further reinforcement of these best practices.

  12. Interactive lecture: Types of materials to be tested (15 minutes) A brief discussion on the types of testing materials and how they influence moderation and goals. Examples may include: wireframes, low-fidelity prototypes, high-fidelity prototypes, and functioning products.

  13. Interactive lecture: Note taking and observations (20 minutes) A brief discussion on preparing materials to facilitate note taking will be presented. Topics may include but are not limited to hand written notes, shorthand, sets of team acronyms, pre-printed note taking sheets, and macros on a laptop. There will also be a brief section on the reporting of results and how your note-taking, observations, and data collection will be used in that reporting so you can prepare in advance. Also important is why reporting is important, for the team to take the results seriously. Types of reporting include: formal reports, executive summaries, tape summaries, action items, and daily status of issues.

  14. Break (30 minutes)

  15. Activity 4 (45 minutes) Participants will divide up into small groups and each team will be given a scenario and be asked to create a paper prototype to test. Teams will spend time designing, writing a partial script, preparing some questions, and preparing some examples of good and bad moderation techniques as reinforcement of the morning lesson on interacting with test participants.

  16. Team presentations (20 minutes) Teams will conduct usability tests on their paper prototype in front of the group and then present/discuss their scenarios and associated material choices and construction, as well as discussing examples of interacting with test participants.

  17. Interactive lecture: Types of usability tests and testing tools and “hot” topics in usability. (15 minutes) The bulk of the topics presented will be core issues around usability testing in person. We will discuss testing variations and some specific issues around field testing, lab testing, international studies, and remote testing. We will touch briefly on some of the usability testing tools available and also take any final questions.

  18. Final Q&A and wrap-up (10 minutes)