The Magazine of the Usability Professionals' Association
By Timo Jokela
Usability maturity models are methods for developing user-centered design processes in companies in order to facilitate usability methodologies for creating usable products. Usability maturity models help management understand the issues surrounding organizational opportunities and to improve the usability of its products. The models also benefit usability practitioners by pinpointing areas of improvement in usability processes and practices.
A maturity model is composed of two main elements:
A usability maturity model can be regarded as an ideal model of user-centered design. The closer the company meets the ideal level of usability maturity, the higher rating it gets.
A common method of utilizing a usability maturity model is to perform an assessment of the company’s current capacity for developing usable products.
In a usability maturity assessment, an assessment team analyzes organizational areas by examining documentation and interviewing key stakeholders. The team maps the findings against a usability maturity model and determines the ratings.
Four Model Categories
The first usability maturity models were Trillium by Bell Canada (a general maturity model including usability engineering), Usability Leadership Maturity Model by IBM, HumanWare Process Assessment model by Philips, and User Centered Design Maturity by Loughborough University. All of these models were developed in the early 1990s.
In 2000, ISO 18529 was published based on work performed by European INUSE and TRUMP research projects. This model is remarkable in the sense that its format complies with the standard processes assessment model widely used in software engineering, such as CMM (Capability Maturity Model) and its revised version CMM-I. A further extended version is ISO 18152 which includes a larger set of processes.
Further developments for Usability Maturity Models include: Human-Centeredness Scale, DATech in Germany, SDOS in Japan, and KESSU in Finland. A more recent effort is the Usability/User Experience maturity model as a result of a series of workshops at the UPA and HCII conferences.
Maturity models fall into four main categories:
The models are summarized in Table 1.
Standard Process Assessment Models
As an example of standard process assessment models, ISO 18529 identifies seven processes, such as Context of Use Definition, Usability Requirements, and Usability Evaluation.
The capability levels range from Level 0 (incomplete) to Level 5 (optimizing) as defined in Table 2.
The result of an assessment is a capability profile as shown in Figure 1. Each process is rated separately: the higher the rating, the higher the capability of the process. Different processes may be at different levels of capability.
The process assessment model includes strict criteria for each level of capability. One should note, however, that an assessment is not a mechanical task. The lead assessor, in the end, makes final ratings through professional judgments. Therefore, the assessors should be experienced professionals.
What is Included in Usability Capability?
Different maturity models cover different organizational areas:
Performance of usability processes means examination of the extent to which usability activities—such as user analysis, task analysis, usability requirements determination, and usability evaluations—are carried out. Performance of usability processes is a basic area. If there is any effective usability in an organization, it should be visible in development projects. Practically all maturity models address this to some extent.
Management of usability processes in development projects means examination of issues, such as the inclusion of usability engineering activities in a project plan, follow-up of the implementation of the plan during the project, and configuration management of the documents produced. Addressing these issues decreases the potential for problems with scheduling, resources, and document control.
Usability in a quality management system is addressed in many models. The standard process assessment models (capability level 3) examine this issue. Requirements by other models include:
Systematic improvement of usability processes examples include statements such as “Management’s actions to improve the current focus on usability,” (ULMM) and “Systematic improvement of quality in use,” (UMM-HCS).
The role of usability at a strategic level is explicitly addressed in some models:
Statements on usability skills include:
The impact of usability is also addressed:
The requirement “resources available for usability work,” (ULMM) addresses usability resources.
Organizational culture is also covered by some models. Statements include:
Which Model to Apply
Standard process assessment models are well-established and well-documented. Established schemes and training exist for assessors. However, the limitation of the models is that they tend to focus too heavily on process management aspects, such as the inclusion of usability activities in a project plan, follow-up of the implementation of the plan during the project, and configuration management of the documents produced. The substance of usability should have higher consideration with less focus on processes alone.
The available documentation of ULMM, HPA, and UCDM is very limited, making these models difficult to use by anyone other than their developers. Practical implementation requires an extensive amount of interpretation which can lead to results that the creators of the models didn’t initially anticipate.
UMM-HCS, DATech-UEPA, Trillium, and KESSU are better documented, although not to the level of standard process assessment. The origin of KESSU was standard process assessment, but was developed for detailed analysis of the usability processes. UMM-HCS is for rough assessment, gaining an overall picture. Trillium is rather old, and the German language limits the use of DATech-UEPA. Language also limits the use of the Japanese models.
Most of the publications focus on describing the usability maturity models. Very little research is devoted to studying the real-life validity of the different models. Empirical research is reported on the development of ISO 18529 and KESSU. It is difficult to find any empirical research reports on the other models.
In summary, the maturity of usability maturity models is rather low. Standard process assessment is a more mature approach, but does not have the appropriate focus on usability in most cases. More empirical evidence on the real life validity and usefulness of different usability maturity models is needed, and concrete guidance of the existing UCM models is very limited.
An effective step to start improvements in usability capability is to carry out a usability maturity assessment. There are different usability maturity models that address different organizational viewpoints: usability processes, quality system, usability business strategy, awareness of usability, action improvements, usability skills and resources, and organizational position of usability.
To make to models more useful and valid, their documentation should be more detailed, and more empirical research needs to be carried out.UX
Dr. Timo Jokela is an independent usability consultant and university lecturer at Helsinki University. He pioneered usability at Nokia in 1990s and served as a professor at Oulu University until 2007. His specific interests include usability maturity models, usability processes, and verifiable usability requirements.
Usability Professionals' Association
promoting usability concepts and techniques worldwide
User Experience Magazine is by and about usability professionals, featuring significant and unique articles dealing with the broad field of usability and the user experience.
This article was originally printed in User Experience Magazine, Volume 9, Issue 1, 2010.
© Usability Professionals' Association
Contact UPA at http://www.usabilityprofessionals.org/about_upa/contact_upa.html