Resources: UPA 2006 Idea Markets
What are the positive and negative impacts on the usability process (and the usability of products) that result from working with off-shore development & international teams?
Activator: James Michael Herold, IBM
There is no doubt that off-shoring has been a topic of much discussion in technology in recent years. Even more specifically, the relationship between usability and off-shoring has been discussed in great detail as well. For example, see the March/April issue of Interactions.
This session wasn't conducted to discuss the rights and wrongs of this business model. Instead, the motivation was to share ideas and personal stories on how this model may positively or negatively impact the usability process, and ultimately, the usability of products.
In an effort to start with a diverse set of discussion points, the following questions were posted:
Thought Starter Questions
Which usability techniques do you find to be the most/least effective when working with international teams and users?
How are you involving international users in the design process?
What are the positive and negative impacts on usability that result from “Globalization” of your products?
What are the ways working in international teams have impacted how you document or specify the usability aspects of your products?
Based on your experience, what are the fiscal and scheduling impacts that result from working with an international team?
Having listed these questions as thought starters, most participants did not use them in this way. Instead what happened was people stopped and shared their stories about working in this changing environment. I don’t know if this was intentional given the theme of the conference: “Usability through Storytelling,” but it did prove to be a fruitful way to work through this topic.
Listed below are the major themes
that were shared between participants via their stories. Please keep in
mind that these themes do not provide a complete description of how usability
work is done in today’s environment. The themes are based only on
the experiences of the individuals who shared their stories in this Idea
Two models of usability work with international teams:
Before getting in to the other themes it is important to describe the two models of usability as they relate to working with off-shore development and international teams. Each of the participants said there organizations were based on one of the following models.
- Local usability, remote development. In this model, the usability work is done locally or someplace closer to the corporate headquarters and the actual development work is done by a remote vendor or team. The local usability work includes all stages of the usability process: initial design concept, requirements validation, task analyses, roles and goals modeling, prototyping, evaluations, etc.
- Remote usability and development. In a slight variation from the above model, here the usability and the development are done by a team remote from the design team. The interesting theme seemed to be that in this model the usability work that was done remotely didn't include nearly as many stages of the usability process. Instead, the remote usability team was more concentrated on evaluation and usability testing.
The “down the hall” effect:
This was a very popular theme. What participants were trying to describe was the ease of access between designers/evaluators and implementers. When design, evaluation, and implementation are done by a collocated team, it is very easy for someone to walk “down the hall” to another team members office and have a quick conversation about something that may not be right in a design. As the different roles of the team are spread across time zones and continents, it's not as easy to have a quick conversation on design issues. The “down the hall” effect led to the next two themes.
Culture, Politics, and Hierarchies:
Several participants mentioned that not only does the spatial separation create difficulties in terms of collaborative productivity, so do culture, politics, and organizational hierarchies. In some cultures it is not as accepted to question the design of a product. On the other hand, this process of constantly questioning and asking “why” is a normal and encouraged during usability and development activities. The negative impact here is that some usability professionals would rely on the implementation team to catch things in the design that were designed poorly, or overlooked during the design process. Of course there are official test processes, but inquisitive developers add another level of checking.
Participants also talked about how even if there is back and forth dialog between the design team and a remote development team, it often gets complicated by reporting structures and chains of command. In other words, communications on the design can get sent up one management chain, across to the same level person at the different organization, and back down to the team member who actually needs the feedback.
The impact of time differences:
Participants in the session mentioned time zone differences as having both positive and negative impacts on the design and development process. Some participants said because of the difference in time zones, many team members had to work long and/or odd hours in order to communicate in real time via conference calls or instant messaging. Other participants mentioned that having team members in different time zones can actually increase the length of the work day. One participant gave the example of using a remotely located team to develop software. At the end of their work day the remote team would deliver their code to the local team. The local team would then test the code delivered and so on. The participant said he could not pull this off with everyone in the same location.
Quality, Process, and Technology:
One of the more interesting issues to be brought up suggested that perhaps a remote development model was a test of an organization's existing development process. If an organization is struggling with the off-shore, or remote development model in terms of schedule and quality problems, it might suggest their process was broken to begin with.
Participants also commented that the usability process and development in general, has become more iterative based on this model, therefore requiring more time in schedules because it takes more time to get things done correctly.
Lots of TLC: Time, Language, and Commitment
Almost every participant said that when projects were started with teams working in different locations, time was always a factor. Every activity seems to take more time. However, like any new challenge, you have to be committed to making it work. None of the participants mentioned any projects failing, but they all also said that it took a great deal of effort to make it work. They specifically talked about having to specify their designs at a much lower level. Where before working with remote teams they could take some leeway in the amount of information they provided describing their designs, it changed so that each tiny detail had to be specified.
As you may have noticed reading through the themes of the idea market session, not that many participants addressed the usability process specifically. Most people indicated that the usability process itself is relatively unchanged. This is the good news. The bad news is that there does seem to be a negative impact on the implementation phase that follows the usability work. It should also be stated that this impact most likely varies between companies and organizations based on their experience with this model.
The vast majority of the comments can be summarized in the graph below. On the horizontal axis is a somewhat ambiguous timeline that runs from the start of a project to its completion. On the vertical axis is an equally ambiguous representation of productivity. As the team progresses on a project, and works through the issues discussed above, productivity increases and the development process improves. The word ambiguous is used because in reality the plot probably doesn't look like this. Actual teams undoubtedly experience peaks and valleys throughout the usability and development lifecycle, where sometimes things are going well, and sometimes they aren't.During the Idea Market discussions and exchange of stories it became apparent that this same graph could be used to analyze the progress of the usability profession and technology development more generally. We would probably all like to believe that productivity is at a higher level than it really is. A good estimate is that we are about 2/3rds our way up the line. We have made progress, but there is still a good deal of room for improvement.