Go to UPA home page The U P A voice

August 2008 Contents

UPA Job Bank

UPA 2008


Teaching Election Officials Usability Testing

Whitney Quesenbery

Whitney is a past-president of UPA, and leads the Usability in Civic Life project. In civilian life, she works on user research, usability and IA projects for organizations like the UK's Open University and the US National Cancer Institute.

The election calendar is very tight, with legally mandated deadlines and other constraints, all conducted in the public view. The UPA Voting and Usability Project wanted a way to fit usability testing into that schedule, and give election officials a way to do what they all want: run excellent elections.

A group of us** got together to develop a testing kit for people who have no specific training in usability or human factors engineering. We needed a test that doesn't require any special equipment or a laboratory.

Our goals are to:

  • Make it more likely the voter's intention is carried out
  • Make it easier for voters to use the ballot
  • Identify problems in the design of the ballot that could lead to residual votes (undervotes or overvotes) or drop-off (failing to vote in races at the bottom of the ballot)
  • Make it more likely that media coverage of an election is positive

The result is "LEO Usability Testing Kit". We hope it will be part of a process of changing how the elections community thinks about the design challenge of election materials. Our goal is for the usability of the ballot, instructions and other election information to be just as (maybe even more than) important as other considerations.

Dana Chisnell ran the first training session over a year ago for officials in Washington State. Afterwards, we heard that several counties used the kit, and the results went beyond the ballot as they found other problems in voter education material -- and were able to fix them before the election!

With the release of Better Ballots, the report from the Brennan Center, and its endorsement of the importance of usability and the LEO Usability Test Kit, there has been a lot of interest in both the ballot design issues discussed in the report, and usability testing as a final check of ballot layout and instructions. We have already done presentations and training in Ohio and Iowa, and plan to present in several other states this summer.

Will it work? After the 2000 election, there were many articles saying that thousands of participants would be needed to find ballot usability problems. We disagree. We are not looking for performance numbers or "valid results" in that sense, but for a diagnostic view, in which we can consider hesitation, requests for assistance and other signs of "success after a problem" as an indicator of a larger problem that is fixable.

At the UPA conference in Baltimore this summer, Steve Krug reported that in his workshops, he runs short, informal tests on a web site volunteered by a member of the audience. and has never failed to find at least one problem. We've had some of the same experiences in training and in usability testing we have observed. Here are two examples:

  • In one training session, an election official acting as the participant missed an instruction to "Vote for no more than two" in one of the races. All of the trainee-observers noticed this and began discussing how to make races with unusual instructions easier to spot, preventing undervotes.
  • In a usability test intended to identify both design problems and need for voter education, we discovered at least three problems in the first 12 participants and were able to confirm the problems as we completed the full set of 36 participants.

The LEO Kit talks about when it's time to call for assistance, and when this quick test will not be sufficient. We hope to provide an understanding of best practices in the same way any of us might train a newly minted user experience group in a company.

This is not the only usability in the election lifecycle. There are usability and accessibility standards for voting systems, tested design templates and opportunities for public review. The LEO Testing Kit is designed to fill the last step: a final check on the finished ballot.

** The LEO Usability Testing Kit was developed in a workshop at the Michigan State University Usability and Accessibility Center with Dana Chisnell, Whitney Quesenbery, Sarah Swierenga, Mike Elledge, Josephine Scott, Fred Conrad and Laurie Kantner.


Usability Professionals' Association
promoting usability concepts and techniques worldwide
Phone + 1.630.980.4997         office@upassoc.org

Contact the Voice