Workshops

This one day workshop will collect information on the use of graphics and sound in complex, data-intensive, and mission-critical designs. The participants will share information about the neuropsychology of visualization and auralization; when multiple media are and are not useful; and the challenges of adding multimedia to applications. The result will be guidelines, a resource list, and a bibliography.

WORKSHOP DESCRIPTION

Some software interfaces cannot be designed using a general style guide. For example, some systems combine data feeds of thousands of pieces of information with manual input; in these situations, the standard screen-design rules are too simplistic. Some systems are mission- or life-critical packages in which the users face many hours of nothing, then an overwhelming amount of activity. Most guidelines don’t address the intersection of boredom and crisis. Some are highly technical analyses in which a misplaced plus sign will send researchers down a million-dollar dead end. Most standards don’t address checking for common sense as well as correctness.

Designers are sometimes able to crack these hard cases with visualizations and auralizations and can use multiple media to solve certain simpler problems.

However, the software tools for creating visualizations and auralizations are not as well known or as well understood as those for creating buttons, frames, calls to databases, and so on. For this reason, the goals of this workshop are two-fold:

1. To develop a set of guidelines that help practitioners choose the appropriate type of visualization or sound for a particular type of problem—for example, a diagram laid over a geographic map for tracking congestion or an oral indication of success or failure in a data synchronization process.
2. To collect as much information as possible about public, open-source, academic, and commercial software toolkits, applets, and servlets in the area of visualization and sound.

This collection will act as a resource list for practitioners who want to add visualizations and sounds to their software interfaces.

Background: Why Visualizations and Sounds Are Useful

Multi-sensory tools are useful for these reasons:

1. People remember and understand information better when it is in context, and the richer the context, the easier it is to understand.
2. People are especially capable when the new information can be organized into, or recognized as part of, patterns and schemas.
3. Multi-sensory systems support fast, accurate reflexive action when needed.
4. Systems that use multiple senses and modes help people take advantage of their own native talents and intelligences.
   

To Understand Better and More Quickly, Add More Context

People do not remember isolated pieces of information very well. When information is divorced from context, the often-cited “7 plus or minus 2” rule is probably valid. In other words, if you ask people to repeat sets of unrelated digits or words back to you, they can do so fairly well if there are no more than five to nine digits or short words in the set. Beyond nine, accuracy falls off quickly.

However, as the experimental psychologist George Miller says, “Everyday experience teaches us that we can identify accurately any one of several hundred faces, any one of several thousand words, any one of several thousand objects, etc.” A possible explanation for our failure with more than seven digits or words, he says, is that “[o]bjects, faces, words, and the like differ from one another in many ways, whereas the simple stimuli we have considered thus far differ from one another in only one respect” [1].
By enriching the context, in other words, people can remember much more.

Visualizations and Music Encourage Pattern Recognition

Research psychologists as well as trainers have shown that you can get around working memory’s limitations if you use patterns, or in other words, show how new information fits into an existing structure.

Experienced or expert users like visualizations because graphics often show patterns more readily. For example, it is much easier to see an outlying data point on a graph than it is on a table, or to understand the structure of a molecule from a 3D picture than from a description. The problem is turning novices into experts, which occurs only with experience, performance-support training, and/or simulations.

Sounds used in an organized way (as music and as auralizations) can help people recognize and isolate errors and mismatches immediately. In many cases, rhythm may be even a stronger cue than melody—try humming tunelessly to the rhythm of “Three Blind Mice” (or “Silent Night”) vs. humming the correct tune but with a different rhythm. Your audience will instantly recognize a pattern through multitudes of “bad” notes, but rarely through an altered rhythm. Similarly to visualizations, with sounds there will be a great difference between novices who can interpret information from “standard” rhythms/musical patterns and those to whom intricate rhythms and multiphonics are the norm.

Multi-sensory Tools Speed up Decisions

Visual or multi-sensory tools are useful in another way as well. They bypass the conscious mind and allow people to react instantaneously when necessary and to solve “insight problems”—tasks that are perceptual, complex, and nonverbal.
Companies in high-risk businesses—for example, warplane designers and nuclear power plant construction firms—create simulators for training the people who will be using the actual machinery. These simulators are probably not just training users kinesthetically—in other words, giving them body memory—but are also training the users’ minds to make intelligent, intuitive, and pre-conscious decisions faster than their conscious selves could if they had to think about it.

In short, visualizations, simulations, and other multi-sensory programs should not be seen as frivolous, marketer-driven add-ons to the “more serious” text-based software analysis tools. Rather, they tap into some very basic and important types of awareness and intelligence, and if well designed, can support decision-making at a profound level.

Multiple Modes Take Advantage of Multiple Intelligences

In 1983, Harvard University professor of education Howard Gardner codified what used to be called “talents” into seven intelligences: Linguistic, musical, logical-mathematical, spatial, bodily-kinesthetic, inter-personal, and intra-personal. (He later added naturalist and spiritual.) We each have some of these, he said, to greater or lesser degrees, and we will learn better if our teachers address each of these intelligences in turn, rather than restricting themselves to the linguistic or logical-mathematical.
As professionals with many degrees under our belts, we are comfortable with linguistic and logical-mathematical modes.

However, we are not our users in this situation either. In unscientific experiments with college students and usability engineers, we have found that they (and we) have multiple intelligences in many areas, especially musical. We have also found that the strengths of the test subjects’ significant others were rarely linguistic or logical-mathematical. Yet applications and even web sites are very dependent on reading, writing, and making logical inferences based only on textual material.

In other words, we suspect a real mismatch between user abilities and preferences and what we provide in our applications and training systems. In mission-critical situations or when difficult analyses are called for, we need to take advantage of every sense and every intelligence possible, not just the ones with which we are most familiar.

References

Miller, George. “The magical number seven, plus or minus two: Some limits on our capacity for processing information.” Psychological Review, vol. 63 (1956), 81-97. Also available at http://www.well.com/user/smalin/miller.html (accessed 14 November 2002).

Background Reading

Bertin,Jacques. 1983. Semiology of graphics. Madison, WI: University of Wisconsin Press.

Gardner, Howard. 1983. Frames of mind: The theory of multiple intelligences. New York: Basic Books.

Harris, Robert L. 1999. Information graphics: A comprehensive illustrated reference. New York: Oxford University Press.

Jourdain, Robert. 1997. Music, the brain, and ecstasy, How music captures our imagination. New York: William Morrow & Company.
Klein, Gary. 1999. Sources of power: How people make decisions. Cambridge, MA: MIT Press.

Myers, David G. 2002. Intuition: Its powers and perils. New Haven, CT: Yale University Press.

Tobin, Jacqueline L., Raymond G. Dobard. 1999. Hidden in plain view: A secret story of quilts and the Underground Railroad. New York: Anchor Books.

Vickers, Paul: CAITLIN: Auditory external representations of programs.” http://computing.unn.ac.uk/staff/cgpv1/caitlin/ (accessed 6 November 2003).

PARTICIPANT SELECTION CRITERIA

We would like participants to have an interest or background in any of these areas: The use of sound (not just in software) to provide information; the use of visualizations to provide information; the use of sound or graphics to manipulate or analyze information; music, drawing, or painting; the design of complex systems used for decision-making or analysis; the design of large, complex systems in general; the psychology of visuals and music; cultural and internationalization issues.

Click here for a sample position paper

Applying to Participate in This Workshop

A workshop is a closed session. Admission to a workshop requires an approved position paper from you addressing the issues suggested by the coordinator(s). Please send your position paper (which should be roughly 1 to 3 pages) to Susan Fowler, susan@fast-consulting.com. Position papers received by March 24 will be accepted or rejected by March 31, in time for you to register before the early registration deadline on April 2. Position papers received by May 5 will be accepted or rejected by May 12, in time for the May 14 registration discount. Papers received after May 12 will be evaluated at the facilitator's discretion. If you want to register early for UPA and have not completed your position paper by these deadlines, you may register for the rest of the conference and add the workshop fee later.

The content of the paper should be:

• Why the participant is interested in the workshop.
• The participant’s background in music, visualization, cognitive psychology, or another area that we should consider during the workshop.
• If possible, information about any systems they’ve helped design.
• A list of books, articles, websites, and software programs that the participant has found useful in this area

POST-CONFERENCE DISSEMINATION OF RESULTS

We will publish preliminary results on the FAST Consulting website immediately and develop a final version as an article for UPA Voice. We might also set up a web log (blog) as a way to continually collect and disseminate information.

ADDITIONAL CONFERENCE ACTIVITIES

The workshop will be followed with an Advanced Topic Seminar on Thursday at 1:30. All workshop participants are invited to describe their findings to the audience of the seminar.

POST-CONFERENCE ACTIVITIES

If the web log idea is of interest to participants, we will continue to collect and disseminate information after the conference.

FACILITATORS
Alice Preston
Senior Usability Engineer
Telcordia Technologies, Inc.

Alice Preston is a Senior Usability Engineer at Telcordia Technologies, Inc. In that capacity, she has conducted participatory design sessions and usability tests, worked with cross-discipline and cross-cultural teams, and has designed applications on several platforms, including Web applications. Prior to working in Usability, she had a long career in Technical Communications and Marketing Communications, working for a number of companies in the software and turnkey systems areas, writing and editing various kinds of documentation, including help systems. Alice has served as team lead and manager for groups from 3 to 10 people, but most enjoys working directly with customers and colleagues to prevent or detect and solve usability problems.

Alice has long served as an assistant editor and copyeditor for "Usability Interface," the newsletter of the STC Usability SIG. She holds a BS from William Jewell College in Liberty MO, with majors in math and music.

Susan Fowler
Consultant
FAST Consulting

Susan Fowler is a principal of FAST Consulting and co-author, with Victor Stanwick, of three software design books, The GUI Style Guide, the GUI Design Handbook, and the Web Application Design Handbook (Morgan Kaufmann Publishing, 2004).
During the 18 years of FAST Consulting's existence, Susan has done technical writing, training, and application design for major Wall Street, pharmaceutical, reinsurance, and telecommunications firms. Recently, she led a multicultural team designing diagram and geographic mapping software for the telecommunications software firm Telcordia Technologies in Piscataway, NJ.
She currently teaches technical communication to engineering students at Fairleigh Dickinson University in Teaneck, NJ, and runs training seminars on software interface design.