Projects: Web Maintenance and Design

Web Project History

This page contains a history of the web project, and reports from user research and usability testing on the design of the site, information architecture and specific features.

August 2003 - Launch

There was a lot of work on the templates and even more work to move the content into the site ... and create new content.

March 2003 - Membership Forms

The membership forms (for new members, member updates and renewals) were also re-designed. Caroline Jarrett, Effortmark, did a review of the original design and Whitney Quesenbery created new designs based on this feedback. A first draft of the design was reviewed by the office for comments, to be sure it met the business requirements.

One of the reasons for the redesign was to be more usable for international members. Usability testing was done simultaneously in three countries: by Christy Mylks in Washington DC, Caroline Jarrett in London and Åsa Granlund in Linköpeng.

The results of this testing was used for another round of design updates and the prototypes were shepherded through implementation by Larry Wood, working with the developers and Allen Mayse, Membership Director.

There were also a number of ideas for future enhancements suggested during the tests. These were reported to the Membership Director and the office.

November 2002- February 2003 - Member Directory

As a simultaneous project, work began on the online Member Directory. This will be the first members-only section of the site. The Web Committee served as reviewers, and it was launched in February.

March 2002 - August 2003 - Content Analysis and Site Conversion

Once the design was complete, a team of volunteers took over, working with the site implementation vendors.

In the fall of 2002, the Web Committee was formed to manage site re-build and ongoing maintenance of the site, and a content analysis begun on the site. This included creating a full site map and defining all of the pages in each section of the site, comparing the pages indicated in the rough prototype to the actual content currently available.

Content began moving into the site - largely as a volunteer effort to spare the budget - in May/June 2003

October 2001 - January 2002 - Designs Created and Refined

Visual Design. The first mockups were created. These designs concentrated on the header area.

A sample of 100 volunteers was sent a questionnaire which asked several questions about their preference among the three designs,and specific aspects of the designs their liked or disliked. The questionnaire also included a set of semantic differential items including, for example:

The professional association whose web site looks like this is:
Unfriendly __1 __2 __3 __4 __5 __6 __7 Friendly
Amateurish __1 __2 __3 __4 __5 __6 __7 Professional

Following this, the visual designs were refined. A second set of mockups were created, and circulated for review. Finally a few of the pages were updated and the  design was completed.

Design Lessons Learned

The design team has learned several valuable lessons during the process of this project:

  • The "bcc" option should be used when sending e-mail messages to multiple recipients. Using the "bcc" option ensures that the list of e-mail addresses in a mass mailing remains hidden, unavailable to would-be spammers, and otherwise protects the anonymity of respondents.
  • Volunteers have been very friendly and accommodating. We received many more volunteers than we planned for. This was an extremely positive surprise. However, it is time consuming to manage such a lengthy list of names and associated data. In the future, we'll consider adding "managing the volunteer database" to our planning.
  • Shortly into the project, we set up unique ID to handle redesign correspondence, such as volunteer surveys. In retrospect, we should have done this from the very beginning. Because three of us were fielding volunteer correspondence, it sometimes became difficult to manage funneling these to a single place.
  • The team learned how to use a promising new tool: EZSort. Although it is in beta version, it proved well-suited to this particular project. Further, by working closely with one of the developers of the tool, we were able to not only use to tool, but customize and improve it as well.
  • When using Virtual Think Aloud, it is critical to communicate clearly the intent and scope of the study. As with in-person testing, participants may focus on aspects of the design that are not yet developed (e.g., the visual design, or the complete site content). Because the study is performed asynchronously, you usually don't have the opportunity to clarify your intent other than what is sent in the evaluation document. It is important to specify what is being tested in clear, concise directions.
  • With any request for feedback, it is important to give people an adequate amount of time to respond, but not too long so that they may forget. That "sweet spot" we found to be about a week. We usually try to send out requests for feedback late in the week, so participants can use the weekend to complete the study. We have also avoided sending requests on Mondays, as from personal experience, work is often at its most overwhelming on Mondays.
  • In several cases, we have found geocultural differences in reaction to aspects of the site. For example, we had named one link "Usability 101" to indicate it contained introductory material. The "101" code, used in the US to describe entry-level college courses, is not used universally. We thus made sure that for each study, we had respresentation from geographies other than the US.
  • The design students we worked with to generate the original mockups of the look of the pages had a background in graphic design, but had no knowledge about usability. Also, these students have been trained to design purely for aesthetic value. Finally, they are accustomed to working purely from inspiration, and not "to spec." Therefore, it was important that we communicated to them the importance of usability. It took several iterations of mockups to get them to produce something that was both attractive and usable. And, whereas thought this would be a valuable learning experience for them, it's quite possible they perceived this as a painful nuisance! It certainly raised the issue of the not-uncommon tension between usability and aesthetics. We strongly believe that both of these are key elements of the total user experience, which should go beyond just usable to engaging and delightful.

September 2001 - Virtual Think A-Loud

The first wireframes were completed in August, in preparation for usability testing.

Because this project is being undertaken remotely, the UCD Team, the client, and the volunteers are geographically distributed worldwide. One of the ways we have found to facilitate usability testing under these circumstance is to perform a "Virtual Think Aloud (see a sample)." We create a document that contains scenarios and tasks, and ask participants to complete the tasks, keeping a diary of their experiences. We then compile the results in a table (see a sample) for analysis.

July 2001 - Card Sorting

The UCD team conducts a card sorting exercise using USort to organize the site from the viewpoint of site users. Cluster analysis of the results will give the UCD Team data for designing the site architecture.

The UDC team sent 100 requests to randomly-assigned members of the volunteer pool (n = 505) to participate in a card sort exercise using EZSort. The EZSort suite contains two tools. Usort allows study participants to sort virtual cards into logical categories to reflect their conceptual model of the site information. EZCalc, its companion tool, applies cluster analysis to the card sort data and "generates tree diagrams that present clearly the page groupings suggested by the data" (Dong, Martin, & Waldo, IBM, 1999).

For this study, a set of virtual cards was created. Each card contained a name and description of a bit of content from the current UXPA site. Participants were asked to sort these cards into logical categories. Of 100 volunteers solicited for this study, 35 completed and returned their data. The team used the tree diagrams generated by EZCalc to determine the first- and second-order categories for the new site. These categories were then validated by asking a new set of participants to review them in light of the content intended to be housed within. Seven participants provided validation data. Once we had these data, we created a of the site. The wireframe was intended to represent the site architecture from a high-level; that is, page placeholders and the links among them to show the conceptual and navigation models of the site. Finally, the client - the UXPA Board and various Committee members - was asked to navigate through the wireframe and to validate the site once again. We made some changes to the wireframe based on their feedback. The wireframe continues to evolve as we test the individual sections of the site, and will eventually become a spec for the site, including not just a high-level model of the site, but eventually, all of the detailed design specifications as well, including links to the templates for the design of the sections of the site.

  • For more information about the tool, visit the IBM web site for EZSort

June 2001 - Volunteer Participants Recruited

The UCD Team sent out the following announcement:

The UCD Team is planning many user feedback activities to help drive the design of the site. These activities will include card sorts, surveys, remote usability tests, and others. To solicit volunteers, the UCD Team posted notices on several popular usability-related forums and posted flyers at the UPA conference. An astounding number of busy professionals and students in the HCI/Usability community and outside have volunteered to participate in these activities. Through their participation in the UCD process, the Team hopes to deliver a web site that is usable, useful, and delightful for all visitors.

A volunteer participant database created, with hundreds of names to be used for UCD activities during the re-design project.

June 2001 -Project Begins

The UPA awards the redesign contract to a team consisting of Carol Righi (Righi Interface Engineering) and Janice James (Simply Usable through Design).


Usability Resources UPA Store UPA Chapters UPA Projects UPA Publications Conferences and Events Membership and Directories About UPA