Be a Reviewer

The UPA 2009 Conference Committee has made it's selection for practicing usability specialists to review proposals for the 2009 conference. Thank you for your interest.

The Process

To give the review team more editorial control, we’ve opened up the review process. Submissions will still receive five reviews each but only three of five reviews will be blind. The non-blind reviewers will be asked to check the submitters’ credentials and also evaluate their presentation skills by referring to any available information, including previous presentations and recommendations from peers. This is a major change for the conference and hopefully a welcome one for both submitters and reviewers. It is our goal to balance our desires to welcome experience from people who have not previously presented at UPA with the desire to make sure that presentations are well done and solid.

UPA Conference Review Criteria Explanations

In an effort to make the review process as transparent as possible, we have decided to share the review criteria with all who are interested.

Topic Category: agree/disagree

  • the topic category matches the content of the submission

Importance: scale - 1 (low) to 5 (high)

  • rate the topic’s importance to the usability community

Practicality: scale - 1 (low) to 5 (high)

  • rate the submission for use to the usability community in application or furthering research
  • the rating for this should take into account the intended audience – new to usability or experienced practitioner

Value to Attendees: agree/disagree

  • the content will provide value to attendees which may or may not have practical application

Audience: scale - 1 (low) to 5 (high)

  • rate the appropriateness of the submission for the submitters’ intended audience

Interactive: agree/disagree

  • the submission will provide opportunity for the attendees to participate

Originality/Novelty: scale - 1 (low) to 5 (high)

  • rate the submission as to new thought on existing information or providing new information, techniques, methodology
  • the rating for this should take into account the intended audience – new to usability or experienced practitioner

Demonstration of Quality: scale - 1 (low) to 5 (high)

  • rate the submission for apparent time in preparation – does it appear to be put together in a hurry to meet the deadline or carefully crafted
  • Peer Reviewed Papers – special attention should be devoted to the methodology; flawed methodology puts collected data into question, too little description of the methodology provides little basis for examining the quality of the collected data

Challenging: scale - 1 (low) to 5 (high)

  • rate the submission for calling current dogma into question or to stimulate the thinking of attendees in new ways

Speaker’s Background: scale - 1 (low) to 5 (high)

  • does the speaker appear to have experience and/or knowledge commensurate with the topic of the submission

Overall Assessment: scale - 1 (low) to 5 (high)

  • rate the submission for overall appeal to the UPA membership
  • a high rating can still be given even if one of the specific category ratings is low – for example, if describing a new methodology, there may be less chance for interaction