upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

A Strategic Approach to Metrics for User Experience Designers

Carl W. Turner

Journal of Usability Studies, Volume 6, Issue 2, February 2011, pp. 52 - 59

Article Contents


Introduction

Good design produces customer value. It seems self evident to designers that usable applications and products that connect with customers' needs and wants are central to the value of their company. Unfortunately, user experience (UX) practitioners sometimes struggle with getting decision-makers in the organization to see the connection between user experience and customer value. Quantifying one’s financial return on the investment made in UX seems an impossible task.

Within most companies there is a significant constituency that places enormous weight on metrics and meeting objective goals. Designers should know this constituency and learn how to approach them. Think of this exercise as the user-centered design of your own services to your internal business partners. If you do, there are significant opportunities for usability professionals with a background in measurement to favorably influence managers’ decisions that affect their companies’ UX practice.

Are ROI Metrics for Usability Unusable?

In 2004 Dan Rosenberg wrote a provocative article entitled “The Myths of Usability ROI.” While praising the groundbreaking book on usability return on investment (ROI) by Bias and Mayhew (1994), Rosenberg pointed out a large number of shortcomings in the literature published since 1994 on usability ROI.

According to Rosenberg, the “traditional ROI approach to defining and measuring the value of usability” doesn’t show the true value of UX activities (p. 23). In essence, typical UX metrics for ROI are unusable. As a corrective action, he proposed thinking strategically by tying one’s own UX activities to the Total Cost of Ownership (TCO) of a company’s products and services. This is certainly a strategic approach to showing ROI, but only if one’s company competes on TCO. However, the article raised a valid point: How do designers show the link between their own value and what matters to their company?

What Makes a Good Metric?

To answer that question, we must first make a digression. Most designers understand that some metrics are more valuable than others. Often, management will insist that everyone’s work “align with company goals,” usually expressed as metrics. However, not all metrics are worth aligning to. So, what makes a good metric? Price and Jaffe (2008) identify the following five qualities of a good metric.

UX practitioners must learn to distinguish between good and bad metrics, and be willing to advocate for better metrics. In fact, relying on published studies for evidence of ROI in UX activities is unconvincing to executives because they were not conducted in the context of one’s own company. Different companies value different metrics. What UX practitioners need are not more published studies conducted at other companies, they need to learn how to collect the right UX data and derive metrics that demonstrate strategic value within the context of their own companies.

To Rosenberg’s point, a good business metric keeps a company focused on the right things, and helps executives make sound decisions. We now have a definition for the strategic use of metrics. Strategic thinking means (a) understanding UX’s value to the company, (b) identifying metrics that drive company—or department-wide decisions, and (c) drawing a clear and obvious connection between one’s measurable value and a company—or department-wide metric.

Previous | Next