Workshop Details

Download: Schedule of Events (PDF) subject to change
By Day - Selected
By Presenter
By Track
View My Agenda

Comparative Usability Task Measurement (CUE-8)

You can’t manage what you can’t measure - but can you manage measurements to generate reproducible and useful results? This workshop compares practical approaches to usability task measurement. Participants will carry out independent, simultaneous measurements of a website ahead of the workshop. At the workshop we will compare results.

Workshop by Rolf Molich (DialogDesign), Jeff Sauro (Oracle ), Jurek Kirakowski (Human Factors Research Group, Cork, Irel), Tom Tullis (Fidelity Investments)
Advanced Practitioners (AP)
8:30am to 5:30pm on Tuesday, June 09, 2009
Position Paper:
download position paper template (RTF)
download presenter's position paper ( DOC)

Note: You may wish to right-click (or control+click on Mac) the download link and choose "Save File As..." to save the file to disk.

About the Workshop

As usability specialists start working professionally with task measurements they quickly realize that measuring usability is more than buying a stopwatch and computing average task times using a spreadsheet.

Recent discussions in usability discussion forums have shown that there is little knowledge and agreement on what constitutes best practice in usability task measurement. This workshop will assemble a number of experienced professionals to discuss the state-of-the-art in usability task measurement based on a common experience in measuring a particular website. To the best of our knowledge, nothing like this has ever been done before. The seven successful Comparative Usability Evaluation (CUE) studies that have been conducted until now ( have focused on qualitative usability evaluation methods such as usability testing. The proposed workshop is the first comparative study of quantitative methods.

Before the workshop, all participants — including the organisers — will carry out independent, parallel measurements of a website using the same test tasks, which all participants will have a chance to comment on before the measurements start. The target group for the website will be the general public. Participants must be willing to spend 10-30 hours on these professional task measurements before the workshop. Each participating team must submit an anonymous measurement report to the organisers before the workshop.

Participants should be willing to measure and report task performance using common operationalized metrics of ** Effectiveness — for example, completion rate and errors ** Efficiency — for example, time on task ** Satisfaction — for example, post-task and post-test ratings. Participants who don’t normally do post-task satisfaction or post-test satisfaction, should consider using the System Usability Scale (SUS) as a post-test questionnaire.

The Agenda for the workshop is: ** Introduction ** Each team presents their approach ** Results of preliminary analysis of the measurements ** Discussion: Lessons learned. What is the state-of-the-art?` ** Conclusion - Further work

This workshop presents a unique opportunity for participants to compare their measurement and evaluation skills to those of their peers and learn from the differences.

Position Paper

Should you choose to register for this session, you will need to complete a position paper. Workshops have limited attendence. In order to select attendees who can best contribute to the discussion, the session presenters will select attendees based on your position paper.

The template contains instructions for filling in the various areas. Provide information about what background and interests you have that are relevant to the session topic.

Save the form in Word or RTF format with a filename that has your last name followed by ”_position_paper”. For example, Smith_position_paper.rtf.

Position Paper:
download position paper template ( DOC)
Note: You may wish to right-click (or control+click on Mac) the download link and choose "Save File As..." to save the file to disk.

The due date for the Position Paper is May 4, 2009.

Please send your completed document to the session leader identified via email when you registered.

The session leader will notify you of your acceptance status to the workshop and then the registration office, will complete the registration process.

If you are not accepted, you will be able to register for a tutorial. For instance, if you registered during our Early Registration period but were not accepted to the workshop, you could register for any open Tutorial paying the Early Registration price.

If you have questions about this, please contact