upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

The Effect of Experience on System Usability Scale Ratings

Sam McLellan, Andrew Muddimer, and S. Camille Peres

Journal of Usability Studies, Volume 7, Issue 2, February 2012, pp. 56 - 67

Article Contents


Despite its age, compared to other industry survey tools measuring user satisfaction, SUS still shows itself to be a useful, practical quantitative tool for supplementing more direct observations or reviews about software use (Tullis & Stetson, 2004).

That said, SUS ratings are influenced by several factors in addition to the usability of the product being evaluated. There are factors like user experience that our own practical experience shows can dramatically affect overall SUS scores for domain products—in fact, by as much as 15-16% between our Never and Extensive groups for either product type. This is consistent with results from an assessment by Sauro, who reported that more experienced users of Web sites (a repeat user group who had been to the Web site before) tended to provide higher, more favorable, SUS scores over a first-time user group (those who’d never been to the Web site before)—on average, experience increased scores very close to our results—by 6-15% (2011a, p.96).

We should add that other factors may also affect satisfaction ratings—for example, inherent differences resulting from cultural diversity of users (Tractinsky, 1997), mistakes in understanding SUS terminology for non-English speaking users (Finstad, 2006), or upfront deployment and setup processes that may be part of product testing. Studies suggest, too, that shorter total deployment time or the existence of product upgrades or patches are not only good predictors that users will observe a failure leading to a software change (Mockus, Zhang, & Luo Li, 2005) but also turn up as clear support factors directly associated with overall product satisfaction (Shaw, DeLone, & Niederman, 2002). Others explicitly identify “ease of software installation” as a significant determinant factor in its own right for product selection (Erevelles, Srinivasan, & Ragnel, 2003; Kolo & Friedewald, 1999).


Previous | Next