The Magazine of the Usability Professionals' Association
By Robert Barnett
From time to time, organizations need to review all their forms, a task often performed by inexperienced staff with limited resources. This article is based on the author’s recent experience in assisting with such a review for Centrelink, an agency of the Australian Department of Human Services that handles social security, veterans’, and similar types of payments.
Numerous studies of both government and non-government forms have shown typical rates of error by form fillers at 80 to 100 percent. Almost all of this organization’s completed forms had one or more errors in the data collected. Some errors may have been trivial, but they were still errors and were costing millions of dollars per year to correct.
Many of the forms were difficult for people to understand. The problem became so bad that Joe Hockey, Minister for Human Services, demanded that something be done. In 2006, his letter to the head of the agency said:
“Centrelink should strengthen its focus on improving communication with customers…I will continue to take a high level of interest in improvements you make to letters and forms to make them easier to understand and use for customers… In 2004-05, the percentage of customers rating the ease of completing Centrelink’s forms as ‘good’ or ‘very good’ was 58%. I expect you to significantly increase customers’ ease of providing necessary information to Centrelink…
“Centrelink’s culture must be responsive to the needs of citizens and stakeholders, not a culture that is unduly defensive and process orientated. The organization should be willing to question itself and its performance on an ongoing basis
As part of this you should fully explore whether any criticisms and customer concerns are valid and, if so, take appropriate corrective action….
“The challenge for executive management is to recognize potential weaknesses and ensure that arrangements for monitoring, assessment, reporting, and review are sensitive to the actual operating environment. In particular, the arrangements should provide for adequate and early feedback to enable corrective action by management, and there should be clear triggers for oversight and involvement at executive level. This requires the identification of problems and acknowledgement of those problems by senior management.”
Realizing the normal government approach to improving forms takes a long time to come to fruition, the minister commissioned a parliamentary committee headed by Senator Colbeck to oversee the project and to ensure that progress was not hindered by red tape. Having a parliamentary committee conduct oversight on the project at this level ensures that decisions are made at the highest management level.
Professional forms analysts spoke to the committee at its first meeting in July 2006. Many issues familiar to forms analysts were discussed, and I was impressed with committee members’ understanding and their desire to improve the lot of the public in filling out forms.
However, the initial reaction on the part of departmental staff was essentially one of fear and disbelief. They were to be given approximately twelve months to redesign all of the department’s major public-use paper forms and make them available for the public to use. With past lead times extending over vastly longer periods, the general consensus was that it was impossible for the project to be completed on time. There are some forms that couldn’t be completed due to technical difficulties, and the size of the task means that the project is still on-going. The initial work was so successful, however, that the new Labor government is continuing the project using the same approach, and not allowing politics to interfere with public usability.
The work is based on a great deal of scientific research into the way people use forms (see sidebar). The first tasks in 2006 began with a series of workshops on the most problematic forms, during which a lot of repetition became apparent. As a result, a series of common question sets were developed to be applied across the board. For example, most forms include questions asking for details such as name (and whether respondents had used other names), address, date of birth, sex, and contact details. Once the questions were worked out, they could be applied to any form asking for the same information.
The question sets have been designed for easy pasting into any form. Form colors change automatically to match those of each form, and all that needs to be done is to change the question numbers.
As an example, the set of name questions must allow for various cultural issues in today’s Australian society. Figure 1 is an example of how the questions appear in the common question set.
After further review, some sets of questions were amended, but they have remained substantially the same. With the common question sets in place, redesign of many of the forms is faster than originally estimated.
The designs are based on the half-page column approach introduced in the UK in the early 1980s and subsequently used in many Australian forms. This layout results in a significant reduction in user error and, at the same time, saves a lot of previously wasted space. Figure 2 shows part of a page using the new half-page column design.
But just copying this layout style without understanding the reasons for the various components will invariably lead to trouble. When graphic designers latch onto appearance and copy styles without understanding the reasons for the design elements, the result is forms that often don’t work as required.
A huge problem for many forms is respondents who don’t sign in the spaces provided. The tested solution was to include a declaration and signature block together as a numbered question, as shown in Figure 3.
Most forms had a check list at the end, and people ignored it. Again, the solution was to include it as a numbered question, as shown in Figure 4.
Questions requiring attachments had additional graphics to highlight them, as shown in Figure 5. The use of the paperclip icon was found to be effective in enabling form fillers to go back and locate the question.
The old designs sometimes didn’t allow for the way people read a form. Figure 6 shows a typical example. In it, the reader typically goes from the bold question text (ending in “private trust”) to the No/Yes ballot boxes, and misses the instructional text below.
Figure 7 shows a better approach. The addition of the bold header about reading the note achieves close to 100 percent success.
Aside from using proven design and language techniques, the greatest factor in the success of the project has been the use of error analysis and usability testing. Both methods help to reduce the political problems associated with changing someone else’s designs—a common forms management problem.
As an important prerequisite for data gathering, error analysis helps to identify potential problems to deal with during redesign and usability testing.
We conducted a number of error analysis studies in which we examined many completed forms to identify where people typically went wrong in filling them out. Then we were able to quantify the real cost of the forms. Most organizations consider only printing and processing costs, but when close to 100 percent of forms have errors, the cost can run into hundreds of thousands of dollars per form and, in at least one case, over a million dollars in one year—that’s the cost just to repair the errors.
These figures are typical of most organizations. We don’t yet have data on the revised government forms, but another example will illustrate the potential savings. We redesigned the application forms for one of our major life insurance companies. Prior to redesign, 100 percent of their forms came in with one or more errors. After usability testing and redesign in the same style as Centrelink, the company reported that the error rate had dropped to five percent, and most errors were minor. Besides reduced errors, form fillers in the testing took only twenty to thirty minutes to complete a 24-page application accurately and without assistance.
The agency previously used focus groups to test forms, but our experience over the past twenty years shows that focus groups are useless for testing usability. They provide some information about people’s opinions, but next to nothing about how a person actually uses a form. Time constraints and form complexity can lead to times when usability testing isn’t feasible, but these are rare and we prefer to test all forms prior to release.
During the Centrelink project, all testing was carried out in a motel room with the observer(s) sitting across the table from the form filler. With two, or occasionally three, observers, there was no indication of hindering the form filler. The agency has recently built a new testing facility with a specialized observation room and video capability. Recent testing still allows the main observer to sit with the respondent and for both to be recorded. The form itself is also recorded, which is very important when testing form filling.
All in all, the redesigned forms are expected to save the Australian government millions of dollars per year by reducing the need to contact customers, correct errors, and deal with bad data.
A summary of Lessons from Forms Research accompanied this article
About the Author
Robert Barnett has more than forty years of experience in information design, public-use forms, electronic forms, user-centered procedure manuals, and forms management. In May 1998 he became the first person outside North America to receive the Jo Warner Award from the Business Forms Management Association. His books include Managing Business Forms, Forms for People, The Form Designer’s Quick Reference Guide and Practical Playscript. He is currently writing a new book tentatively named Error Free Forms on reducing user errors on forms.
Usability Professionals' Association
promoting usability concepts and techniques worldwide
User Experience Magazine is by and about usability professionals, featuring significant and unique articles dealing with the broad field of usability and the user experience.
This article was originally printed in User Experience Magazine, Volume 8, Issue 2, 2009.
© Usability Professionals' Association
Contact UPA at http://www.usabilityprofessionals.org/about_upa/contact_upa.html