Resources: UPA 2006 Idea Markets
How do you utilize peer reviews to further your designs?
Activator: Cindy Lu, HFE Consulting
The Activator's Initial Questions
It is a common practice to conducting design reviews among peers before presenting the designs to the development team, clients and/or other stakeholders. The purpose of peer reviews is usually to verify the designs and identify possible design flaws. It is not unusual that your peers have different opinions on the design outcomes than you do due to the differences in training, knowledge, experiences and design philosophy. How do you resolve the issues from peer reviews and use the comments to advance your designs?
Questions for Discussion:
- Do you conduct peer reviews for your designs? If you do, what is the process?
- What are the challenges from peer reviews?
- What strategies do you use to overcome the challenges?
12 participants were actively involved in the discussion during the 1.5-hour session. The activator discussed the questions with one or two participants at a time.
According to the participants, most companies do not have a formal peer review process except for two companies. However, most participants do have their peers review their work before the designs go to a larger audience. The designer usually calls the review meeting, records the issues and is responsible for resolving the issues.
Participants discussed various strategies of resolving issues from design reviews. Several participants indicated that it was important to differentiate the real problems from the perceived problems. Sometimes, further clarifications of designs can resolve the issues. It is common that senior designers, subject matter experts (SMEs) and/or people who have the most knowledge of users and business process help make the final design decisions when issues arise. Several participants also indicated that they made notes for future usability testing even the issues were resolved.
There are two types of reviews:
- UI designs – screen flows, layout, navigations, interaction models, etc.
- UI specifications
The reviews usually focus on verifying the designs in terms of:
- Support of user interactions and workflows
- Compliance with the company’s UI standards and/or common UI design principles
1. Formal Review Process
In one company, usability researchers and UX designers work in pairs. The researcher conducts user interviews and the designer observes the session to obtain user knowledge. After the user interview, the researcher works on the report while the designer starts the designs. The design solution then goes through three levels of reviews.
- Reviews between the usability researcher and designer
- UX group reviews
- Development team reviews
A company that has achieved CMM Level 5 has two levels of design reviews
- Peer reviews
- Supervisor reviews
Peer review process:
The designer is responsible to organize the meeting, recording issues and resolving the issues. The review is conducted with four other senior designers.
- The designer sends out the review information (Use Case, Design Specifications and UI designs) on Friday.
- Peers conduct reviews independently.
- A review meeting is conducted on Tuesday morning 9-10AM.
- The meeting focuses on recording issues and reasons.
After the meeting, the designer works with his/her supervisor to resolve the issues. Supervisors of the designers are experienced UI designers.
2. Informal Reviews
Some participants indicated that they usually just find another peer who is available to review their work. Some participants also call for a UX group review meeting when needed.
Challenges from Peer Reviews
- Peers do not think the design can support users effectively/efficiently. For example, the screen is too crowded.
- Peers think some behaviors deviate from standards/common practice.
- Comments are from managers or senior members.
Strategies to Resolve the Issues
In companies where formal review process is in place, the issues and reasons are documented. In companies where informal reviews are adopted, the designers usually go to users or people who have most business and user knowledge to help resolve the issues. Participants discussed the following strategies:
- Find the real cause of the problem. Check the both design and issue assumptions. Differentiate the real vs. perceived problems.
- Sometimes, further clarifying the designs can resolve the issue
- Sometimes, the issue assumptions are not valid.
- Sometimes, the UI standards are misinterpreted.
- Verify the designs and issues with users (when the designer has access to users) or conduct a usability testing.
- Review the issues with subject matter experts or people who have the most business knowledge.
- Bring a neutral person who has both business knowledge and design experience to the team.
- Work with the supervisor or senior designer who has more business knowledge and design experience.
- Vote – based on majority votes.
- Use field data to support the design.
- Because of time/cost constraints, there is no time for further discussion. The designer just picks up one solution or compromises the solution.
Other Comments from Participants
- Negotiation skills and the ability to work with the team are important in design reviews and the process of resolving the issues.
- Do not bring your boss to the review. Some members may be intimated by the presence of the manager.
- Even the issues are resolved, make notes and observe users in the usability testing
Participants who work in larger UX groups agree that a formal peer review process is important to ensure design quality. When issues arise, finding the real cause of the problem is the first step before attempting to resolve the issues. Involving people who have most business and user knowledge can help make decisions. Ultimately, conducting usability testing can verify the issues and solutions. Designers must have good communication, negotiation and team work skills.