The Magazine of the Usability Professionals' Association
By Jurek Kirakowski
The British Lord Kelvin once said: “When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.” This quote is often reproduced (with minor variations) in introductory textbooks on statistics and engineering, as a sort of encouragement I suppose. But is it true? For some, anything that can’t be counted and expressed in numbers is just not worth talking about. This is the “quantitative view” prevalent in much of contemporary science. Why take it? Because otherwise the ground for disagreement is just too great on any but the most trivial of observations, proponents say. Numbers are sharp and words are soft.
The opposite is the “qualitative view.”
Proponents of the qualitative view claim that it
is rare that you can summarize in numbers
the things that are of real human interest. It’s
the user experience that matters, isn’t it?
Numbers can be, and often are, used as
snake oil. People who do this don’t usually
care about how the numbers were obtained
(the so-called “counting rules”). The neat way
data can be stacked in tables and graphed in
three dimensions makes it look hard and
objective. However if bits of the data are
obtained by using different counting rules, it’s
actually nonsense to combine it.
Counting is the art of assigning numbers to states or events in the world. For instance, measuring the length of your desk in inches, or timing someone logging in to your website. In science, we use such counting rules to communicate between ourselves how to assign numbers to states or events in a way that is reliable and accurate, so that anyone can replicate our work, so we all know what we’re talking about.
Here are some issues that are worth raising before we jump in with a ready-made interpretation of our numbers (such as “clearly, a menu in need of re-design.”)
What we have to grasp firmly in HCI is
that we are an engineering discipline. The
field of statistics has developed to help scientists
build, support, and demolish theories.
We use statistics in a descriptive way to get
an objective view of what is going on.
Theoretical statistical models are all well and
good, but to parody Wittgenstein, the data
that you gather “is everything that is the
case.” How did we manage to survive for all
those millennia before we invented statistics? We used common sense, counted where it made sense to count, and observed when we
knew that counting rules would fail. Let’s not forget that attitude of mind. I rather fear Lord Kelvin did.
About the Author:
Jurek Kirakowski is the director of the Human Factors Research Group at University College Cork,
Ireland (hfrg.ucc.ie). The HFRG has existed since 1984, combining research, tool development, and consultancy activities to HCI. Jurek is a viola player at heart, first learned to program in LISP, and is also known as a world authority on moving statues, about which he once wrote a bestselling book.
Usability Professionals' Association
promoting usability concepts and techniques worldwide
User Experience Magazine is by and about usability professionals, featuring significant and unique articles dealing with the broad field of usability and the user experience.
This article was originally printed in User Experience Magazine, Volume 7, Issue 3, 2008.
© Usability Professionals' Association
Contact UPA at http://www.usabilityprofessionals.org/about_upa/contact_upa.html