Featured UPA Links:
Cleaning up for the housekeeper
|Expert Review||Usability Testing|
|Addresses this question||Is the design optimized based on what we know about how people interact with computers?||
Can key users
|Recommendations derived from...||
|Complimentary benefits||Focuses on what the design brings to the user||Focuses on what users bring to the design|
Some believe that this difference between ER and UT (evaluating it yourself versus observing others) means that less experienced practitioners can do equally effective UT. All you have to do is identify where people stumbled, right?
Anybody can break things
If the goal is just to figure out where things are broken, then it doesn’t take much experience to do usability testing. On that approach, however, there isn’t really all that much value to usability testing either. Do you need someone else to tell you that the transaction is confusing? Not really.
But that’s not really the goal. The goal is to improve the user’s experience. This means that noting where people stumble is important. But, if your usability testing is worth paying for, that’s just a means, not the end.
Within UT, testers should be able to systematically identify points in the task flow or information search where users slow or stray from the intended path. With this data, the tester should also identify mismatches between how the site/application works and the user’s conceptual use model. For sites and applications, this means exploring the fit between the task model and the user’s expected task flow. On the web, we also add comparison of the navigation structure and information structure. Once you identify the mismatch, the next step is to design the fix. Interestingly, to ensure this level of result from your usability testing, it is even more critical to have experienced practitioners.
So, adequate usability testing reports describe where a site or application fails. A good usability testing report describes this in terms of mismatches between the user’s model and the site model. Really good usability testing reports provide guidance on how to minimize (or get rid of) the gaps by implementing fixes that move the site model closer to the user’s mental model. Expert reviews can’t do that. Experts don’t have access to that kind of data without observing users. See the difference?
Cleaning up for the housekeeper
When I was a kid, my friend’s mother always made her clean up her room before the housekeeper came. We were always perplexed by this. Wasn’t that the housekeeper’s job? Actually, the housekeeper was very clear on this for us: She did NOT straighten. She cleaned. She had a specific amount of time set to clean the house. If she spent that time distracted with straightening, she would not get to cleaning.
The same concept applies here. Doing ER is like straightening up before the housekeeper gets there. If you conduct ER first, ER provides feedback that allows the developers to ‘tidy up’ the interface so that the usability testing can focus on cleaning. If you don’t straighten first, both the tester and the participant are distracted and waste time. However, if the right methods are applied at the right time, ultimate outcome is a really clean house….er interface.
So, doing both ER followed by UT optimizes the return on the usability investment. ER identifies fundamental or generic challenges within the user experience. Usability Testing highlights contextually specific gaps between the user model and the site model. Executed together, UT builds on the ER, providing complimentary feedback supporting focused and actionable design recommendations. Thus, the power of combined usability review techniques significantly enhances the power of the review.
Fu, L., Salvendy, G., and Turley, L. (2002) Effectiveness of user testing and heuristic evaluation as a function of performance classification, Behaviour & Information Technology, 21(2), pp. 137- 143.