upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Usability Evaluation of a Tag-Based Interface

Rajinesh Ravendran, Ian MacColl, and Michael Docherty

Journal of Usability Studies, Volume 7, Issue 4, August 2012, pp. 143 - 160

Article Contents


Discussion

The results from this study support our hypothesis that a tag-based interface can improve usability, particularly user satisfaction, of online and mobile banking. The paired t-test analysis showed that the SUS scores and summative experience ratings were significantly higher with the tag-based interface. Conversely, the difference in average efficiency, measured through task completion times, was not significant in either context. There was no difference in effectiveness, measured by task completion rates, between the interfaces. We did not expect to see a difference in efficiency primarily due to participants’ lack of experience with the tag-based interface. Likewise, the familiarity and generic nature of the evaluation tasks may have made them very easy to complete, and as a result all tasks were successfully completed.

Based on the SUS scores and summative experience ratings, participants were more satisfied with the tag-based interface than the conventional interface. This outcome is more apparent in the mobile context compared to the online context. That finding is possibly due to the ability to carry out transactions via simple tag selections, which in effect reduces the effort required of users on mobile devices. The favorable rating of user-friendliness for tags compared to the conventional design may result from participants finding it easier and simpler to interact via their own tags.

Interestingly, participants with no prior experience in mobile banking were more satisfied with the tag-based interface compared to mobile-experienced participants (see Figure 10 and 11). Alternatively, with the conventional interface, a difference of about 11% was observed as an effect of past mobile banking experiences, which is consistent with the findings reported by Sauro (2011). The high levels of satisfaction observed among inexperienced mobile banking participants suggest that there is a potential for the tag-based interface to positively affect the adoption and acceptance of mobile banking, which is important given that the global customer base of mobile banking is expected to reach close to one billion users by 20154.

As part of the post-evaluation debriefing, we asked participants about minimal information on screen (tag-based) versus detailed information on screen (conventional). Participants preferred seeing minimal information on screen by default and being presented with detailed information when they request it. This may be strongly tied to the security and privacy concerns of banking users especially those related to mobile banking (Wessels & Drennan, 2010). In addition, from our observations and debriefing with participants, the notion of providing tag suggestions as a way to encourage the user to tag and re-use tags already present in the system was well received especially in the mobile context where many found it cumbersome to tag. It was obvious that they preferred to select suggested tags that were appropriate rather than typing their own. However when they did enter their own tags, they expected them to be shown on the top of the list of suggested tags.

The actual user performance is not significantly improved in both contexts. However, participants perceived the tag-based interface as one that can improve their performance and appear to be more satisfied and inclined to use the tag-based interface than the conventional interface. One possible explanation for the poor user performance on the tag-based interface is the unfamiliarity and lack of experience with the new tag-based interaction style. Also, in the mobile context, this issue may have been further exacerbated by a smaller display. Nevertheless, longer task completion times were anticipated for this study partly due to the way the study was carried out (see the Limitations section).

Challenges and Recommendations

In this section we outline the challenges we identified with the tag-based interface, and we provide recommendations for meeting those challenges. These issues are not based on the quantitative data we described above; they are based on qualitative observations made during task performance and on comments from participants during the post-task interviews. There are four issues: minimal information on the tag-based screen, idiosyncratic and ambiguous tags, tagging behavior due to use of the conventional interface, and navigation through a large number of tags.

The first issue is the limitation around the level of information displayed on tag-based screens. The conventional interface provides all relevant information by default to a banking user for decision making. Alternatively, the tag-based interface only displays tags on screen, placing an increased level of responsibility on users to recall or recognize a tag with regards to a resource such as bank account or biller. Therefore, it is important to enable users to retrieve relevant information associated with a tag effortlessly and unobtrusively when required. This ensures users are confident and comfortable with the interface as locus of control (Shneiderman & Plaisant, 2004) remains with them.

This issue can be remedied through dynamic tooltip popups (i.e., jquery tooltip5). Dynamic tooltip popups allow information to be displayed on demand in an unobtrusive fashion. The browser events “hover” and “tap” can be used to trigger the tooltip in online and mobile contexts, respectively. This places a reduced level of effort to view detailed information associated with a tag. From our work, we found that the majority of users prefer information on demand, especially in the mobile context. In addition, visual cues such as font and background color can be used to deliver subtle messages to users. For example, the color green can indicate a healthy account balance and red for an unhealthy account balance. Likewise, font colors can help indicate transaction status: green for success, orange for pending, and red for failure.

The second issue is related to assignment of tags that lack meaning, particularly those that are idiosyncratic and ambiguous. The use of idiosyncratic tags, which at first may appear to users as easy and very personal, can impede the usability of the interface especially in the presence of a large number of tags. Users generally struggle to remember or contextualize tags assigned to transactions that are not meaningful enough like “##21” assigned to a bill payment of 21 dollars, for example. Moreover, idiosyncratic tags need to be excluded from cross-user tag suggestions that otherwise could distort the quality of suggestions and users’ perceived usefulness. Conversely, ambiguous tags can stem from overly generic or synonymous keywords used to describe or categorize a resource. For example, a mobile bill payment tagged as “bill pay” or “payment” does not offer much detail about the transaction nor context for future recall.

This issue can be alleviated by suggesting tags, and educating and training users on tagging best practices. Firstly, through the use of tag suggestions, users can be persuaded to choose an appropriate tag instead of entering their own. Also, to increase the chances of users selecting a suggested tag, the tag suggestion popup can be displayed in a focus of form input box rather than on a keystroke. In other words, the tag suggestion list is shown even before the user starts typing, and tags are filtered as the user types. Secondly, a more sustainable and long-term approach may be education and training on tagging best practices.

The third issue is concerned with the tagging behavior of users, largely influenced by the way in which the conventional interface functions. Instead of inserting discrete and meaningful keywords as tags, users provide a one-line description that is generally lengthy and specific. For example, a rent payment in the month of April was described as “rental payment apr 12” or “rent up to 21 apr.” These descriptions, despite being useful references, do not permit reuse or simple categorization of transactions.

This issue can be addressed by inserting more than one tag. In the example above, tags such as “rental payment” and “april” can help to categorize and simplify the process of tracking and reconciliation. Also, in the following instance, the primary tag “rental payment” can be retained and the secondary tag “april” can be changed to “may,” for example. As a result, tags can be re-used for similar transactions with the option of adding more contexts through addition or replacement of tags.

The fourth and final issue is the challenge of navigating through large numbers of tags (>100). This is likely to become a problem in the long-term, exacerbated by random and unorganized assignment of tags. Also, this issue is a potential challenge for users who use online banking more extensively than others and have a multitude of financial needs. The issue itself is likely to have a more significant impact in the mobile context than online given the display constraints due to smaller screen sizes. Nevertheless, regardless of the banking context, users are likely to find it cumbersome to navigate through their tags for each transaction they conduct. Therefore, a simple and convenient way to discriminate and select tags is essential and paramount.

This issue can be addressed through the addition of several design features. First, a search feature that allows users to quickly lookup tags based on tag name and related transactional details such as date or amount. That feature can be more intuitive by filtering tags as users type. Furthermore, tags can be sorted based on usage in both online and mobile contexts. As a result, the most commonly used tags are shown first by default. Second, the ability to edit past tags can be incorporated to allow users to reorganize and manage their tags. This is important and part of the learning process to better understand the utility of tags and ways to tag effectively.

Limitations

Participants’ lack of experience and familiarity with tags/tagging specifically in the banking context affected the outcome of this study. We believe this played an important role in the actual user performance. In an effort to evaluate the usability of the tag-based interface in a real-world setting, we did not explain what each customized interaction entailed in detail, instead leaving participants to “discover.” We only offered explanation in the event participants experienced difficulties in completing a particular task. We believe this was a good way to assess the overall usability of the tag-based interface, especially to tease out design issues and make the interaction as intuitive as possible.

The results of this study were also influenced by the evaluation tasks. Although the tasks were meant to be generic, participants who had conducted similar transactions in real life to the ones given during the study might have been able to better relate to and be personally engaged with the tasks as opposed to others who did not. This may have influenced the tags assigned and also the perceived usefulness of the customizations offered.

Conclusion

In this study, we report on the comparative usability evaluation between a tag-based interface and a conventional interface in both online and mobile banking contexts. The results suggest that user satisfaction can be improved by adopting a tag-based interface. This is particularly true for inexperienced mobile banking users where a more significant improvement was observed. As a result, we conclude that the tag-based interface may positively affect mobile banking adoption and acceptance mainly from a usability point of view. Also, this study highlights the key issues and challenges identified relating to the use of tags for interaction and offers potential solutions to tackle them.

Future Work

In this study, we evaluated the first-time use of tags in a banking context. A longitudinal study would examine the long-term effects of the tag-based interface. Such a study, according to Vaughan and Courage (2007), can help understand the long-term usability issues, which are not usually found during users’ first encounter with a new technology.

Additionally, a study of the affective aspects of interaction (i.e., aesthetics) could help us to further understand other dimensions that affect usability. The visual representation of tags may well have influenced user’s perceived usability, which may explain the increased levels of user satisfaction observed in both online and mobile contexts.


4Global Industry Analysts, http://www.prweb.com/releases/2010/02/prweb3553494.htm

5Jquery Tooltip, http://jquerytools.org/demos/tooltip/index.html

 

Previous | Next