“Do you trust me enough to answer this question?” Trust and Data Quality

Here’s a question for you: what is your social security number?

If you’re from the U.S., you probably thought: “Why should I tell you that?” From elsewhere, you probably thought, “I don’t have one of those. Does it matter?”

Either reaction shows that simply asking a question—whether on a website, in a survey, or elsewhere—isn’t enough to guarantee an answer.

To get an answer, I’d have to persuade U.S. readers to trust me; to feel comfortable that I have a legitimate reason for asking and that I’ll look after an important piece of personal data appropriately. And I’d have to persuade readers from elsewhere to trust that the effort of working out how to deal with that question will be repaid. Either way, trust is crucial. If you don’t trust me to make proper use of the data, you won’t want to give me that data, or make the effort of obtaining it for me.

Trust and the Reservoir of Goodwill

In the second edition of Don’t Make Me Think! A Common Sense Approach to Web Usability, Steve Krug suggests that users come to each page or form with a “reservoir of goodwill (see Figure 1).” Too low a level of goodwill and users give up on their tasks, or, for the more web-savvy, replace their accurate answers with a parallel set of answers that they maintain for this purpose. (The unkind might call this lying.) Either way, the result is poor data quality.

Text in this image is at the end of the article

Figure 1. The reservoir of goodwill. Read the text in this image. (from Krug, 2009 – Reprinted with permission.)

Based on a lifetime of experiments in survey administration and analysis, Don Dillman developed his “social exchange theory” in Internet, Mail, and Mixed-mode Surveys (Dillman et al, 2009). His theory has similarities to Krug’s reservoir of goodwill (see Figure 2).

A triangle, with the three points labeled 1. Increase rewards: perceived rewards, immediate, not deferred. 2. Reduce effort: cognitive effort, emotional effort. 3. Establish trust.

Figure 2. Social exchange theory relies on trust. (Adapted from Dillman in Jarrett and Gaffney, 2008).

Dillman points out that higher response rates come from a balance between the user’s perception of the social and financial rewards on the one hand, and the cognitive and emotional effort required on the other, all in the context of trust.

Dillman’s theory is based on extensive research, including one famous experiment that looked at response rates to a paper survey sent out with three different levels of incentive:

  • No financial incentive
  • Guaranteed $10 offered after return of the completed survey
  • A $2 bill enclosed with the mailed paper survey

Although the guaranteed $10 created a small increase in response compared to no incentive, the biggest uplift came from the $2 bill. Dillman explains this in terms of trust. There is a small immediate reward, but what the $2 bill really demonstrates is that the organization issuing the survey trusts the respondent enough to hand out the reward in advance. And enough respondents feel the urge to reciprocate that trust to make an important contribution to response rates. Contrast that with the $10; even though it’s guaranteed, it requires that the respondent take the first step in demonstrating trust.

Sponsorship and Response Rates

The typical Dillman survey is a major, lengthy investigation that will inform public policy. Within that context, he recommends that you send a letter ahead of the survey from an appropriate, respected figure of authority to help create trust and lift response rates.

It’s not enough for you, as designer of the survey, to respect the authority—the users have to have the same reaction. In one experiment, (Bhandari et al in the International Journal of Epidemiology, 2003) the letter-in-advance idea didn’t work for orthopedic surgeons. An appeal from a committee of expert surgeons actually depressed response rates by 19 percent.The authors suggested that super-specialists might be perceived as arrogant by other surgeons. (The ungenerous amongst us might wonder whether some surgeons are sufficiently arrogant to feel annoyed that others are being lauded as experts above themselves.)

Your Questions in a Credible Context

Most of us aren’t doing such substantial surveys. We’re working on little, “What did you think of this website today?” bits of feedback, or on forms that are as short as we can make them and still be consistent with their purpose.

So a whole letter ahead of time? Definitely not. The effort of reading the letter would be disproportionate, increasing the perceived effort without enough corresponding increase in trust to justify it. Instead, the user’s experience of the website previous to the form has to establish your credibility.

In our book Forms that Work, (Jarrett and Gaffney, 2009) we point out that one small thing you can do to build trust is to give the user a form when a form is expected. That is, don’t offer a long list of instructions or hide the form away somewhere. Get right to it.

Without a prior notification or letter, the page of questions must stand alone as trustworthy in its own right, continuing the build-up of trust achieved by the website where it originated. If you hand your users over to a third-party service, such as a payment handler or a market research business for your satisfaction surveys, then you need to think carefully about whether to keep continuous visual branding or to opt for the branding of the third party.

How You Use the Data

If it looks like you won’t use the data against them, users are more likely to trust you with it. There’s an interesting result from an experiment where pages were branded like a cheap quiz in the typeface Comic Sans (Figure 3), or branded like an authoritative survey from a respected university (Figure 4). Teenagers were more likely to admit to illegal behaviors, such as dope smoking, when answering the cheap quiz.

Screen with three questions: Have you ever smoked marijuana (i.e. pot, weed)? Have you ever cheated while in a relationship? Have you ever driven when you were pretty sure you were over the legal blood alcohol level?

Figure 3. Social behavior questionnaire branded as a cheap quiz.

Screenshot with the same questions as Figure 3

Figure 4. Social behavior questionnaire branded as a university survey. (John et al, ”Strangers on a Plane” in the Journal of Consumer Research, 2011).

Are you thinking, “Gullible teens can be trapped by anything?” Well, think again. The researchers achieved similar results with New York Times readers. They responded to a questionnaire branded as a “Test Your Ethics” quiz on a columnist’s blog, and they revealed more unethical behaviors than on the same questionnaire sponsored by a university.

Please don’t think that I advocate hiding your data collection activities behind Comic Sans quizzes. That would verge on “black hat” methods that would be unprofessional (as well as unethical).

Questions Influence Trust

When I discuss the topic of trust in the context of surveys and forms with web-savvy audiences at conferences, they often claim to scan the questions ahead of time and then make a decision about whether they trust the organization well enough to fill in the form.

I do not know of an experimental comparison that has investigated similar issues in a controlled context, but I have often observed users in usability tests who, on a multi-page form, opt to “fill in this form with any old thing” to explore the set of questions, with the intention of repeating the process with their real data at some point later. If they encounter questions that they consider inappropriate, they will be gone, never to return.

Preserving Trust 

So far, we’ve been looking at the way that trust can affect the willingness to answer. If we turn that around, the question-and-answer process can also affect the longer term trust in the brand. Just as a bad experience with a form can empty that reservoir of goodwill, so can a good one refill it.

The longer term view leads me to a final thought—the dreaded “opt-in” questions that appear at the end of so many forms, such as those in Figure 5.

Screenshot: two check boxes labeled (1) Newsletter: keep me up to date with news, software updates and the latest information on products and services and (2) I agree to the Terms and Conditions (Required - See link below)

Figure 5. A typical pair of ”opt-in” questions at the end of a form.

Table 1 summarizes the perceived effort and perceived reward from each of those questions. Questions like these are frequent, and you may well argue that they are familiar and that users cope with them perfectly well. (Or you may have been on the receiving end of similar, “They’ll cope, and we need that data,” instructions from clients and colleagues.)

Of course, both of those arguments are correct. Users will mostly cope with this sort of thing. But will they cope gladly? Isn’t there a risk that they’ll be left with a bit of a twinge, some minor disappointment that your organization or brand has subjected them to the same old unthinking “opt-in” that they’ve suffered so many times before? Won’t that twinge slightly undermine their trust in you? Is the longer-term damage to the brand really justified by the value of the data?

Question Typical perceived effort Typical perceived reward
Email Relationship effort: many of us have learned to guard our email addresses carefully, and revealing the email address is a compromise with our standards of privacy. Figure 5 has no explanation of how the email address will be used; the link to the privacy policy is some distance away. The perceived reward is uncertain.
Newsletter Conversation effort: The sentence is a long one and takes a bit of reading. There is no promise as to how often the information will appear, so the user may be creating considerable future work. The organization offers four types of information. This could be regarded as useful and valuable, or it might be regarded as overwhelming.
Agreement to terms Appearance effort: This tick box is unchecked, whereas the previous one was checked. The user has to notice the difference and realize that this box has to be checked. Assuming that this form stands between the user and some perceived reward, there is benefit in checking this box. But will those terms and conditions be easy to understand and appropriate, or onerous?
Table 1. Effort and reward for form questions.

Summary

It’s important to think about trust when asking questions. A question isn’t just a question; it is part of a social exchange, a conversation that takes place within the context of what users expect from organizations and what they hope to achieve with that conversation.

When designing those exchanges, you should aim to:

  • Create a trustworthy context. Make it clear who you are and why you are asking the questions.
  • Be consistent. Make sure that the questions themselves are consistent with your organization’s brand, and the purpose for which they are asked.
  • Preserve trust for the next time. Avoid ending the question sequence with a series of minor irritations that can damage the perception of your organization’s brand.
Text from Figure 1

There are a few things worth noting about this reservoir:

  • It’s idiosyncratic. Some people have a large reservoir, some small. some people are more suspicious by nature, or more ornery; others are inherently more patient, trusting, or optimistic. The point is, you can’t count on a very large reserve.
  • It’s situational. If I’m in a big hurry, or have just come from a bad experience on another site, my expendable goodwill may already be low when I enter your site, even if I naturally have a large reserve.
  • You can refill it. Even if you’ve made mistakes that have diminished my goodwill, you can replenish it by doing things that make me feel like you’re looking out for my best interests.
  • Sometimes a single mistake can empty it. For instance, just opening up a registration form with tons of fields may be enough to cause some people’s reserves to plunge instantly to zero.

- From Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems, by Steve Krug, New Riders 2009. Reprinted with permission.

Comments are closed.