upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Examining the Order Effect of Website Navigation Menus With Eye Tracking

Alex J DeWitt

Journal of Usability Studies, Volume 6, Issue 1, November 2010, pp. 39 - 47

Article Contents


A meta-analysis was made of several website usability projects over the past year at OTOinsights. A total of 15 unique navigation menus were included in the study, across 13 unique websites, with eye-tracking data from a total of 147 unique participants. Each menu was the primary navigation menu on the website, and data was taken from first exposure to this menu on the homepage (at the beginning of the usability study). The menu items were all textual and different numbers of words made up the menu item, meaning that some menu items occupied slightly more or less screen space than others. Examples of two such menus are shown in Figure 1.

Figure 1

Figure 1. Examples of the web menus that were analysed, a 9 item horizontal menu and a 10 item vertical menu

Each project followed the same methodology of a task-based approach with retrospective stimulated think aloud (RSTA) protocols. Participants were recruited from the public, meeting a demographic specific to the test websites (e.g., for a travel site, have booked a holiday in the past 6 months or intend to in the next 6 months). The participants were then invited to complete a series of simple tasks representing real world journeys they would complete on the websites (e.g., on a travel site, to make a booking for a holiday). Eye tracking was used in the first of these tasks to record the user’s fixation and saccade patterns across the web page upon their first exposure to it. The tasks were presented to the user on a sheet of paper for them to read aloud, the moderator asked them to clarify the purpose of the task, then the participant was told to begin as soon as the website had loaded. During the task both the moderator and participant were silent. Fixations represent moments when the eye stops moving and the brain directs attention to the area within the foveal field, i.e., the area the user is looking directly at. Saccades are moments of very rapid eye movement between fixations, where visual resolution is lost. Using the RSTA method involves allowing participants to complete eye tracked portions of usability studies without any interjections or requirements to think aloud, as these would influence their eye movements. After the given task was completed, the eye-track video was replayed to the user and they were asked to explain retrospectively what they were thinking, what they were looking for, and what their reactions to the stimulus at the time they completed the task were. The equipment used was a Tobii T60 infra red monitor that requires very little calibration, is non-intrusive, and accurate to 0.5 degrees.

During analysis, each web page stimulus was marked with Areas of Interest (AOI) that precisely enclosed each navigational menu area. The AOI enclosed the textual label used for the menu item, where tabs or buttons were used in the menu, the entire tab, or button. The Tobii software was used to calculate the Time To First Fixation (TTFF), a measure of how many seconds elapsed between task start and fixation inside an AOI. TTFF was chosen as a metric because it specifically addressed the aim of this study to establish if the placement of a link in a menu affected the average time it took users to discover it. There were several other measures yielded by the eye-tracking software, including number of fixations and duration of each fixation on a menu element, but these were considered out of scope as they occurred after the menu item was located, and this study was considering only the process of locating menu items, i.e., data before the menu items were located. TTFF values refer to the first time the user fixated on the menu item; they had never looked at it previously in the entire test session.

The TTFF data was amalgamated and marked up with the link name. We noted whether the menu was horizontal or vertical and whether the link was relevant to the task at hand. We calculated the distance of an item from the centre of the menu. This distance was calculated using the following formula:

Distance = (N/2) + 0.5 – P

Where N is the total number of items in the menu, and P is the position of the item in question. For example, an item that appears second in a menu of eight items the formula is the following:

D = (8/2) + 0.5 – 2 = 2.5

This formula was constructed specially for this test and was felt to be the simplest way to show numerically how far from the centre of a menu each item was. The number of menu items was halved to find the middle point, then 0.5 was added because the middle point was a whole menu item (e.g., the middle point of a 10 item menu would be in between items 5 and 6, or 5.5). Whether the item was at the beginning or the end of the menu was considered irrelevant for the first part of analysis, only its distance from the centre was considered. However, possible effects of position were considered in the results section.

Graphs were plotted for the distance from centre of the menu versus the TTFF for each website, and estimated for possible significant patterns. A graph of the mean TTFF for each unique distance was plotted and subjected to a Spearman’s rank correlation test to check for significance across the entire data set.

Previous | Next