# Usability Professionals' Association

2009 International Conference

Bringing Usability to Life

Making everyday things better

Download: Schedule of Events (PDF) subject to change

If you don’t measure it you can’t manage it: Is Product A faster than Product B? Did we meet our goal of a 75% completion rate? Learn how to conduct and interpret appropriate statistical tests on usability data then communicate your results in easy to understand terms to stakeholders.

Half Day Tutorial by
**Jeff Sauro*** (Oracle)*

- Category:
- Methodology
- Track:
- Advanced Practitioners (AP)
- Time:
- 6:30pm to 9:30pm on Monday, June 08, 2009

- INTRO: The tutorial will begin with briefly reviewing the most common ways to measure usability at the task-level: completion rates, task time, errors and task-level satisfaction. (15 Min)
- Big-4 Measures are operationalized metrics of ISO 9241-pt 11.
- Understanding the role of chance in sampling data: intuition can be helpful and harmful

- Objective: Clearly understand both the limits and data available from small sample usability data through use of confidence intervals (30 Min)
- Confidence Intervals
- Interval estimation and point estimation
- Confidence Interval around a Completion Rate (Adjusted Wald)
- Exercise
- Compute confidence intervals using excel and using the online calculator using instructor or participant provided data for a task completion rate(5 Min)

- Confidence Interval around mean task time (t-statistic)
- Dealing with Non-Normal Time data
- The importance of graphing time-data
- Mean, Median or Geometric Mean?
- Exercise
- Compute confidence intervals using excel and using the online calculator using instructor or participant provided data for task times (5 Min)

- What about Sample Size?

- Confidence Intervals
- Objective: Determine if a usability test task has met or exceeded a goal (e.g. users can complete the transaction is less than 2 minutes). (30 Min)
- Identifying Specification Limits for task time and completion rates
- Comparing task completion rate against a benchmark
- e.g. 9/10 completed the task does this meet or exceed our goal of 75% completion rate for 1st time use?
- 1-proportion test using exact binomial probabilities

- Exercise
- Compare a task completion rate against a stated benchmark (e.g. 75%) using excel and using the online calculator with instructor or participant provided data (5 Min)

- Comparing Avg. Task Time against a specification limit
- e.g. The average time from 12 users was 2 minutes, does this meet or exceed our goal that users can complete the task in less than 2.5 minutes?
- 1-sample t test
- Compare the average task time against a stated specification (e.g. 2 minutes) using excel and using the online calculator with instructor or participant provided data (5 Min)

- What about Sample Size?

- Objective: Be able to compare two interfaces or versions (A/B Testing) by showing statistical significance (e.g. Product A takes 20% less time to complete a task than Product B p