The Magazine of the Usability Professionals' Association
By Rolf Molich, Kasper Hornbaek, Steve Krug, Josephone Scott, and Jeff Johnson
DO YOU SOMETIMES FEEL LIKE YOUR USABILITY REPORTS AREN’T TAKEN SERIOUSLY? Perhaps part of the problem is that the users of your reports have trouble understanding or acting on your recommendations. Writing useful and usable recommendations is a skill that all usability professionals should master. Evidence that usability professionals sometimes miss the mark when describing usability solutions comes from an examination of the usability comments from the seventh Comparative Usability Evaluation (CUE-7).
In CUE-7, experienced usability specialists independently wrote usability recommendations for six typical usability problems taken from two websites. Problems with the quality of recommendations include:
This article outlines guidelines for developing useful and usable recommendations—that is, recommendations that are likely to be accepted, that can be implemented, and that significantly improve the system.
Rolf Molich, Kasper Hornbæk, Steve Krug, and Josephine Scott arranged an informal contest. Usability professionals were invited to submit useful and usable recommendations for six usability problems from e-commerce websites.
We gave them this scenario: “Imagine that you have been hired by IKEA. The company has asked professional usability consultancies to carry out usability evaluations of selected parts of their website. The consultancies have reported six problems [which were included in the scenario]. The consultancies have not provided recommendations on how to fix the problems. Your task is to provide usability recommendations based on the—possibly imperfect—usability problem descriptions.”
Our comparative analysis is based on recommendations for identical problems submitted by eight experienced usability professionals, including all of the authors.
Here’s a chance to test your skills in writing useful and usable recommendations. This example presents a usability problem that is analyzed further in this article. Before you read on, consider what recommendation YOU would have provided. Then compare your recommendation to those in the article.
PROBLEM: Usability tests in CUE-5 had shown that many users were confused by the initial screen of the IKEA PAX Wardrobe Planner (Figure 1). When users start from scratch, they don’t know what they should do first to build a wardrobe.
Figure 1. Users were confused by this screen.
Figure 2. Sample of submitted recommendation.
Here are three recommendations submitted by CUE-teams, followed by our master solution.
The recommendation is well illustrated (see Figure 3), but important details about the redesign are not described. It’s not entirely clear how it’s a solution to the “I don’t know what to do first,” problem. It’s not the least possible change that would solve the problem.
Figure 3. Well illustrated, but important details are not described.
The authors had extensive discussions on the best possible recommendation based on all recommendations we received. After several iterations, we arrived at a “master solution” for each of the six problems. The full problem list and the master solutions are available from Rolf Molich at molich@DialogDesign.dk.
During our discussions we found the following guidelines particularly helpful:
The rest of this article presents the lessons we learned about useful and usable recommendations from our comparative study.
We end with some advice on the process of creating great recommendations.
1. Make recommendations constructive and direct
Don’t simply criticize the current design. Some of the submitted recommendations were not true recommendations. For example, “It may not be obvious to users how to fill out the form. No instructions provided in Help.” Such indirect recommendations sound like complaints. “Include instructions to complete the form in Help” reduces the critical tone of the former recommendation.
2. Provide detail
Detail prevents misunderstandings and “creative misinterpretation.” The best recommendations included rewritten on-screen text and doctored screenshots. Avoid vague recommendations (“Make the error message clearer.”) For example, the recommendation “The way specials are promoted needs to be redesigned,” provides insufficient detail. A better recommendation is: “Specials are often overlooked. Redesign the Specials page, for example as shown in Figure x.” In short: Provide more details than you think are necessary.
A picture is worth a thousand words—even for recommendations. Illustrations—wireframes, marked up screenshots, or doctored screenshots—demonstrate what you mean. Also, examples from other systems that do what you are recommending are helpful.
This is particularly important if the recommendations may conflict with business goals, technical constraints, and the application’s current basic workflow.
5. Recommend the least possible change
Small is beautiful. Small changes increase the chance that your recommendation will be adopted, unless the system being evaluated is an early prototype. With released systems, complicated redesigns should be avoided when simpler solutions work just as well. Usability problems should not be solved by adding new features (“creeping featurism”).
6. Address only the original usability problem
Stick to the point and don’t overstep your charter. For example, some submitted recommendations suggested replacing the word “frame” in the IKEA PAX planner with “cabinet,” but we had no indication from the tests suggesting that users were confused by the word.
7. Speak the readers' language
The readers of recommendations are developers, designers, and managers, not usability specialists. Avoid usability jargon such as “user model” or “508 Accessible” in recommendations.
8. Provide a quick-and-dirty, low-cost alternative
If the ideal solution is potentially expensive or time-consuming, provide an alternative that will reduce the impact of the problem to an acceptable level. Make it clear that the alternative solution is less than perfect.
9. Solve the problem
A great recommendation solves the problem for all users, not just for a small subset. It solves the cause of the problem, not just symptoms.
10. Keep recommendations short
Readers of usability evaluation reports are busy people, so we suggest keeping recommendations short. Long recommendations are less likely to be read and acted upon. This advice, of course, may conflict with other advice—for example, “Provide detail” and “Justify.” The point is to provide as much detail as required but no more.
Creating great recommendations is not just a question of mastering guidelines for the content. It is also important to master the process by which you develop recommendations.
1. Nurture your professional skills
Writing great recommendations requires skills that differ from those required to run great usability tests. Run lots of usability tests to see what works and what doesn’t work for users. Discuss your usability recommendations with the “users” of your reports—developers, marketing, sales—to learn what works and what doesn’t work from a business and technical point of view. Study test results and recommendations from other usability professionals.
2. Understand how it works
Some participants in our contest unintentionally wrote recommendations that showed that they did not fully understand how the system that was tested worked or was intended to work. To write great recommendations, one must be fully familiar with how it works.
3. Take your time
Mediocre recommendations such as “Solve it in the help system” can be written in seconds. Writing great recommendations takes time. Usability testing often reveals problems quickly, but great recommendations are sometimes hard to produce.
4. Be humble
Before conducting this study, the authors had the naive perception that if you showed a useful and usable recommendation to other experienced usability professionals, they would immediately accept it even if they had provided a less optimal recommendation. However, our study shows that it takes a lot of discussion to convince usability professionals that their recommendation is not optimal. Often, when presenting your recommendations you will discover that someone has a better idea. Be humble; be prepared to set your ego aside and accept it.
5. Tweak but verify
Recommend the least possible change, and then recommend a quick usability test to see if you’ve solved the problem. If you have, fine. If not, try another tweak or move on to a major change.
6. Follow up
Show an interest in what happens to your great recommendations. Ask polite and timely follow-up questions if your recommendations are not adopted. Maybe you can learn something.
Agreeing on what constitutes useful and usable recommendations is more difficult than we thought. We often complain that developers don’t listen to us. Enough complaining! Let’s make our advice more usable and useful to them!UX
Rolf Molich manages DialogDesign, a Danish usability consultancy, which he founded in 1993. Rolf coordinated the Comparative Usability Evaluation studies (CUE), where sixty professional usability teams independently evaluated the same websites under realistic conditions using their favorite evaluation method. Rolf is co-inventor of the heuristic evaluation method (with Jakob Nielsen).
Kasper Hornbaek is an associate professor at the University of Copenhagen. He has published at CHI about usability, is involved in national and European research projects on the industrial use of usability evaluations, and co-edited a recent issue of International Journal of Human-Computer Interaction on this issue.
Steve Krug has been a usability consultant for thirty years. He is the author of Don't Make Me Think: A Common Sense Approach to Web Usability (New Riders, 2nd ed. 2005).
Josephine Scott was part of the TechSmith Corporation team that participated in CUE-6. She has provided usability, user research, and user-centered design services for energy companies, major booksellers, automotive manufacturers, and software producers. She is a member of the Usability Professionals’ Association's Voting and Usability Project.
Jeff Johnson Jeff Johnson is principal consultant at UI Wizards, a usability consultancy. He previously worked at Cromemco, Xerox, US West, HP, and Sun. He has taught at Stanford, Mills, and the University of Canterbury. He has authored HCI articles, chapters, and the books GUI Bloopers, Web Bloopers, and GUI Bloopers 2.0.
Usability Professionals' Association
promoting usability concepts and techniques worldwide
User Experience Magazine is by and about usability professionals, featuring significant and unique articles dealing with the broad field of usability and the user experience.
This article was originally printed in User Experience Magazine, Volume 7, Issue 4, 2008.
© Usability Professionals' Association
Contact UPA at http://www.usabilityprofessionals.org/about_upa/contact_upa.html