upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Usability Evaluation of Email Applications by Blind Users

Brian Wentz and Jonathan Lazar

Journal of Usability Studies, Volume 6, Issue 2, February 2011, pp. 75 - 89

Article Contents


Desktop (Stand-Alone) Email Applications

Three desktop email applications (including Outlook 2007, Outlook Express, and Mozilla Thunderbird/Sunbird) were evaluated for their usability. Because 15 users each tested one desktop application, each of these three applications was tested by five users.

Microsoft Outlook Express

Outlook Express appeared to be one of the most usable email applications that we tested. The lack of a calendar feature may have contributed to the high rate of success with Outlook Express, and this might also make it less appropriate for many business environments (as well as the fact that it is not included with the newest operating systems). Some of the few issues that were noted with Outlook Express included the lack of a good search feature, difficulty in finding the address book, a poorly functioning spell check feature, and the lack of a prompt to save something when the Escape key is pressed on a keyboard (a feature that was said to be helpful to blind users).

Findings Specific to Blind Users

While the lack of good search, spell check, and save prompt features are things that could negatively affect both blind and sighted users, there are reasons why those features are more important for individuals who are blind. Because a blind user cannot visually scan the inbox or folders for a particular email, he or she is more reliant on search features than sighted users (particularly when managing a large number of emails). While a spell check feature is important to all users, having good spell checking functionality is even more important when you cannot quickly visually scan back through a document for spelling errors, but instead must listen to individual words and possibly letters to verify that the spelling is correct. The save prompt feature is convenient for all users, but it is often very important to blind users because they cannot see when an email or network problem caused a message not to be sent or when some other software malfunction causes the email “compose new message” window to be closed. The address book location is also more difficult for blind users because they rely on the text menus to find it rather than the visual address book button that a sighted user can easily locate.

Microsoft Office Outlook 2007

One of the problems identified in Outlook 2007 by four users was that the new Office ribbon presented some usability problems and difficulty when executing shortcuts from the keyboard. Two users could not find a way to save a contact in the address book. One user could not find a way to access the calendar, another user noted difficulty in identifying which date was currently being viewed on the calendar, and two users accidentally created blank calendar appointments (without knowledge of this) when trying to delete an appointment from the calendar.

Findings Specific to Blind Users

For a sighted user, there is an obvious visual way to access the calendar and save contacts in Outlook. This illustrates the problems and frustrations that are experienced by blind users who use this application. Creating a blank calendar appointment is something that could happen to a sighted user, but it is also a problem that would be readily visible. Listening through calendar appointments and coming across blank appointments was confusing to the blind users in this study.

Mozilla Thunderbird/Sunbird

Mozilla had the poorest completion rate and time performance of the three desktop email applications that we tested. Several users noted navigational issues with Mozilla Thunderbird as well as labeling problems for things like the “TO” and “SUBJECT” edit fields. There was no audio label read when the screen reader cursor was focused on those fields. There was a drop-down box next to the “TO” field that three out of five users thought was the “TO” edit field, and this created difficulty when trying to enter a recipient email address. Three users additionally commented on the lack of a delete confirmation, which was noted to be something that blind users rely on.

Sunbird is Mozilla’s recommended desktop calendar application, and it seemed to suffer significantly from usability problems. Three users could not tell what date was being currently displayed on the calendar. Three users also had great problems when they were trying to navigate the dates on the pop-up calendar. When creating or editing a calendar appointment, there were also problems seen when selecting the hours for a particular date (see Figure 1). Two users also discovered that tabbing through the “New Event” dialog box did not take them to the “Save and Close” button. Overall comments on both Mozilla Thunderbird and Sunbird were that these applications were both difficult and frustrating to use.

Figure 1

Figure 1. Selecting hours problem in Mozilla Sunbird: The drop-down menu for selecting hours is unusable when navigated with only a keyboard and screen reader.

Findings Specific to Blind Users

The current date being displayed on a calendar is something that is obvious to a sighted user, but for a blind user it can be frustrating to wonder what date an appointment is being created on or being viewed. The drop-down for selecting the hours for a date may be somewhat confusing to a sighted user, but it is even more confusing and unmanageable when it is navigated with a keyboard and screen reader. The fact that tabbing through a new event does not take the user to “Save and Close” is a problem that specifically affects a user who is restricted to using the keyboard and screen reader for navigation.

Previous | Next