The Magazine of the Usability Professionals' Association
By John Webb and Tomer Sharon
Does this sound familiar? You plan and execute the perfect research study. You gain insights that have huge impact for the product. The research findings touch not only workflow and design but suggest major business implications addressing questions your product manager has been asking for months—and some that she hasn’t even thought to ask. You craft a detailed, voluminous report that will most likely end up in the Smithsonian as an exceptional example of 21st century user experience research. However, there’s one critical problem…nobody reads it. Or, if someone does read part of it, they forget the main findings and their application to the product as soon as another email plops into their inbox. Why does this happen? What might you have done differently?
Our team recently concluded a field study examining a new platform component for managing online display advertising. Our goals were to define the strategy around the product, to learn more about the users, and to test preliminary design concepts.
Data collection took place in four countries and involved contextual observation, interviews, and an artifact walk-through. Participants were also asked to keep an incident diary to share their experiences with the new component. The study team included two UX researchers, an interaction designer, a product manager, and the lead engineer.
The field study produced a large number of insightful findings that we needed to communicate to our stakeholders. However, because we had not previously conducted fieldwork for these particular stakeholders, there was uncertainty surrounding the value and substance of the research. Additionally, we were unsure how the stakeholders processed information or utilized research findings. We also wished to promote and demonstrate the value of field studies to other organizations such as support, sales, product management, and engineering.
Fearful that we might end up writing a report that would get passed over, we decided to try the new idea of holding an expo where stakeholders could “experience” the research instead of reading about it. We imagined a large meeting room with a self-guided exhibition of posters, artifacts, and videos inviting stakeholders to learn about our results. We planned to conduct this expo for a full day, during which time the study team was available to discuss research findings and recommendations with the extended team.
Preparing the Expo
During the preparation phase, we brainstormed the contents of the expo and developed a “mind map” of findings. We recruited a designer who created visual representations of our findings and helped us design several posters for the expo. Posters included:
Themes with product implications (see Figure 1)
Representative artifacts gathered from participants were selected to showcase. Insightful entries from the incident diaries were also included. In addition, edited video clips from the study sessions were set up in viewing stations around the expo room.
A slide show was produced including the following topics:
Promoting the Expo
Marketing and promotion were crucial to gaining visibility and attendance. Several days prior to the expo, we sent email invites to the direct and indirect stakeholders. We asked key engineering and product management directors to invite their teams. We hung invitation posters in elevators, mini-kitchens, and outside the expo room (see Figure 2). The day of the event, we sent two email reminders to all of our stakeholders: one in the morning and another in the afternoon.
We used a room with a glass wall and looped the videos and presentation to attract the attention of people who casually walked by. As word of the expo spread throughout the office, more people showed up to see what was happening.
Holding the Expo
We created a multimedia experience, and set up the room like a gallery exhibit, including video viewing stations (to watch select user clips), posters illustrating key findings and product implications, printed blog posts (incident journal entries), collected artifacts that people could pick up and discuss, and a slide show that ran in a continuous loop in the room (see Figure 3). During the expo, the researchers, product manager, and lead engineer answered questions about findings, encouraged discussions about the meaning of the findings, and shared our field study experience.
After the expo, we provided copies of the posters to the product manager, engineering director, and product management director. The following week, we gave presentations to those stakeholders who were unable to attend the expo.
Using all the content we had prepared for the expo, we created an internal website to be launched on the morning of the expo. The site served as a repository of artifacts, diary entries, videos, and notes from the study. This interactive “report” pretty much wrote itself thanks to all the expo preparation. The website was easily discoverable through the intranet search, provided an engaging presentation format, and directly linked the report to the project site.
The results from the expo exceeded our expectations. Approximately fifty people attended the expo, and over 100 visited the expo website. It’s highly doubtful that this many people would have taken the time to read a standard research report. Product managers, engineers, sales representatives, support staff, and UX researchers and designers visited the room throughout the day, watching video clips, discussing the artifacts, and intensely debating the study findings and their implications (see Figure 4). The research really came to life!
The chief benefits of holding an expo included creating a high level of engagement, rendering study results more memorable, raising the profile and impact of UX research, and increasing acceptance for field studies.
Reflecting on this effort, expo attendees still utilize findings and recommendations from this study, even a year after it was conducted, and our team members ask for more studies with similar deliverables.
The expo helped us to better appreciate the power of face-to-face interaction with our stakeholders. Facing so many tangible findings in an expo
setting made our stakeholders engage with the study results and recommendations.
We found that presenting findings via an expo “democratized” the experience because attendees were more willing to ask questions and engage with the material. This is less likely to happen during traditional report presentations, which are often dominated by lead product managers and one or two vocal participants. As a result, many more ideas were generated from a wider group of people.
Additional lessons we learned include:
The recipe for an expo provided in this article may, like any recipe, be tweaked and adapted by the chef to match the nature of the research being reported and the stakeholders involved. The important point is that rich reporting is an improvement beyond the traditional written report leading to a more meaningful engagement among a wider variety of stakeholders.UX
John Webb is a user experience researcher at Google where he conducts lab-based usability studies, field studies, and contextual interviews. His projects focus on both the sell and buy side of the online advertising space. Prior to joining Google, John was a senior user experience architect for DoubleClick where he worked in an Agile development environment.
Tomer Sharon is a user experience researcher at Google in New York City. He is a graduate of Bentley University and founder and past president of UPA Israel. In addition, Tomer is a former UX Magazine editorial board member. Prior to joining Google in 2008, Tomer worked for nine years as a researcher at Check Point Software Technologies in Israel.
Usability Professionals' Association
promoting usability concepts and techniques worldwide
User Experience Magazine is by and about usability professionals, featuring significant and unique articles dealing with the broad field of usability and the user experience.
This article was originally printed in User Experience Magazine, Volume 9, Issue 3, 2010.
© Usability Professionals' Association
Contact UPA at http://www.usabilityprofessionals.org/about_upa/contact_upa.html