Tips for Usability Professionals in a Down Economy
Journal of Usability Studies, Volume 4, Issue 2, February 2009, pp. 60-69
Article Contents
Tip #9: Compare Design Alternatives
You get much more out of a usability study if you can compare design alternatives. While this may not sound like a way to be more cost-effective, I think it is in the long run. Too many design teams get locked in to one basic design solution early on, then they just fine-tune that, perhaps through iterative usability testing. This is what Bill Buxton (2007) calls "getting the design right," which he contrasts with "getting the right design." By starting down one design path too early, you may very well miss other significantly different designs that are much better. But many design teams often will resist pursuing alternatives, perhaps because of the pressure of schedules and resources. So you might have to be the "evangelist" for comparing alternatives.
Some of the kinds of comparisons you might want to evangelize include the following:
- Comparison to the old design. Rarely are we testing something totally new, for which there isn't an old version or an old way of doing it. I think any significant new design project should start with a baseline usability study of the current design. Then new versions can be compared to it.
- Comparison of significantly different designs. Following Buxton's admonition to "get the right design" usually means convincing design teams to come up with radically different design solutions to the same problem. Low-fidelity versions (e.g., sketches) of these different versions can be used in usability studies to get feedback about what works for users and what doesn't.
- Comparison to competitors. As mentioned earlier, sometimes it can be very enlightening to do usability studies of your competitors' Websites. This can also be true as part of the iterative design of a new product or Website. If you're designing a new Website for building virtual widgets, try to compare your design to one or more competitors' sites for building virtual widgets.
- Comparison of subtly different designs. After some iterations of comparing quite different design alternatives you will eventually reach the point where you start iterating on one main design—fine tuning it. Here too it's going to be more cost-effective to compare some of these alternatives simultaneously rather than sequentially. Don't just assume that Method A for entering dates in a reservation system is going to be more effective than Method B. Try them both. (And then document which one worked better, under what conditions, so that future projects will know that.)
One of the best ways to compare alternative designs is through online usability studies. We've done online studies where we simultaneously compared as many as 10 different designs in a between-subjects design. With over 1,000 participants in these studies, we got plenty of data on each design to be able to make quite accurate comparisons between them. And we've done these studies in just a few days.