A Meta-Analytical Review of Empirical Mobile Usability Studies

Constantinos K. Coursaris and Dan J. Kim

Journal of Usability Studies, Volume 6, Issue 3, May 2011, pp. 117 - 171

Article Contents

---

References

• Agarwal, R., & Venkatesh, V. (2002). Assessing a firm's web presence: A heuristic evaluation procedure for the measurement of usability. Information Systems Research, 13(2), 168-186.
• Andon, C. (2004). Usability analysis of wireless tablet computing in an academic emergency department.  Master of Biomedical Informatics, Oregon Health & Science University, Portland, Oregon. Retrieved from http://www.ohsu.edu/library/newbooklists/newbooks200406.shtml  
• Barnard, L., Yi, J. S., Jacko, J. A., & Sears, A. (2005). An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices. International Journal of Human-Computer Studies, 62(4), 487-520.
• Barnard, L., Yi, J. S., Jacko, J. A., & Sears, A. (2007). Capturing the effects of context on human performance in mobile computing systems. Personal & Ubiquitous Computing, 11(2), 81-96.
• Bevan, N., & Macleod, M. (1994). Usability measurement in context. Behavior and Information Technology, 13, 132-145.
• Bødker, M., Gimpel, G., & Hedman, J. (2009). Smart phones and their substitutes: Task-medium fit and business models. Paper presented at the Eighth International Conference on Mobile Business, Dalian, Liaoning, China.
• Bohnenberger, T., Jameson, A., Kruger, A., & Butz, A. (2002). Location-aware shopping assistance: Evaluation of a decision-theoretic approach. Paper presented at the Mobile HCI 2002, Pisa, Italy.
• Brereton, E. (2005). Don't neglect usability in the total cost of ownership. Communications of the ACM, 47(7), 10-11.
• Brewster, S. (2002). Overcoming the lack of screen space on mobile computers. Personal and Ubiquitous Computing, 6, 188-205.
• Brewster, S., & Murray, R. (2000). Presenting dynamic information on mobile computers. Personal and Ubiquitous Computing, 4, 209-212.
• Bruijn, O. D., Spence, R., & Chong, M. Y. (2002). RSVP browser: Web browsing on small screen devices. Personal and Ubiquitous Computing, 6(4), 245-252.
• Burigat, S., Chittaro, L., & Gabrielli, S. (2008). Navigation techniques for small-screen devices: An evaluation on maps and web pages. International Journal of Human-Computer Studies, 66(2), 78-97.
• Butts, L., & Cockburn, A. (2002). An evaluation of mobile phone text input methods. ACM International Conference Proceeding Series, 20, 55 - 59  
• Buyukkoten, O., Garcia-Molina, H., & Paepcke, A. (2001). Seeing the whole in parts: Text summarization for web browsing on handheld devices. Paper presented at the Intl. World Wide Web Conf.
• Chan, S. S., Fang, X., & Brzezinski, J. (2002). Usability for mobile commerce across multiple form factors. Journal of Electronic Commerce Research, 3(3), 187-199.
• Cheverst, K., Davies, N., Mitchell, K., Friday, A., & Efstratiou, C. (2000). Developing a context-aware electronic tourist guide: Some issues and experiences. In Proceedings of CHI2000, The Hauge, Netherlands.
• Chin, A., & Salomaa, J. P. (2009). A user study of mobile web services and applications from the 2008 Beijing Olympics. Paper presented at the 20th ACM conference on Hypertext and hypermedia, Torino, Italy.
• Chittaro, L., & Dal Cin, P. (2002). Evaluating interface design choices on WAP phones: Navigation and selection. Personal and Ubiquitous Computing, 6(4), 237-244.
• Chittaro, L., & Dal Cin, P. (2001). Evaluating interface design choices on WAP phones: Single-choice list Selection and navigation among cards. Paper presented at the Mobile HCI 2001, Lille, France.
• Chong, A. Y., Darmawan, N., Ooi, K. B., & Binshan, L. (2010). Adoption of 3G services among Malaysian consumers: An empirical analysis. International Journal of Mobile Communications, 8(2), 129-149.
• Clarkson, E., Clawson, J., Lyons, K., & Starner, T. (2005). An empirical study of typing rates on mini-qwerty keyboards. Paper presented at the Conference on Human Factors in Computing Systems, Portland, Oregon, USA.
• Costa, C. J., Silva, J. P., & Aparicio, M. (2007). Evaluating web usability using small display devices. Paper presented at the the 25th annual ACM international conference on Design of communication, El Paso, TX.
• Coursaris, C., Swierenga, S., & Watrall, E. A. (2008). An empirical investigation of color temperature and gender effects on web aesthetics. Journal of Usability Studies, 3(3), 103-117.
• Coursaris, C. K., & Kim, D. J. (2006). A qualitative review of empirical mobile usability studies. Paper presented at the 2006 Americas Conference on Information Systems (AMCIS), Acapulco, Mexico.
• Cyr, D., Head, M., & Ivanov, A. (2006). Design aesthetics leading to m-loyalty in mobile commerce. Information & Management, 43(8), 950-963.
• Dahlberg, T., & Öörni, A. (2007). Understanding changes in consumer payment habits - Do mobile payments and electronic invoices attract consumers? Paper presented at the HICSS 2007, 40th Annual Hawaii International Conference on  System Sciences, Waikoloa, HI.
• Danesh, A., Inkpen, K., Lau, F., Shu, K., & Booth, K. (2001). Geney: Designing a collaborative activity for the palm handheld computer. Paper presented at the CHI2001, Seattle, WA, USA.
• Duda, S., Schiel, M., & Hess, J. M. (2002). Mobile usability. Usability—Nutzerfreundliches web-design (pp. 173-199). Berlin: Springer-Verlag.
• Duh, H. B.-L., Tan, G. C. B., & Chen, V. H.-h. (2006). Mobile usability: Usability evaluation for mobile device: a comparison of laboratory and field tests. Paper presented at the the 8th conference on Human-computer interaction with mobile devices and services, Stockholm, Sweden.
• Ebner, M., Stickel, C., Scerbakov, N., & Holzinger, A. (2009). A study on the compatibility of ubiquitous learning (u-Learning) systems at university level. Universal Access in Human-Computer Interaction. Applications and Services (pp. 34-43). San Diego, CA, USA.
• Ervasti, M., & Helaakoski, H. (2010). Case study of application-based mobile service acceptance and development in Finland. International Journal of Information Technology and Management 9(3), 243-259.
• Fang, X., Chan, S., Brzezinski J., & Xu, S. (2003). A study of task characteristics and user intention to use handheld devices for mobile commerce. Paper presented at the the 2nd HCI in MIS Research Workshop.
• Fitchett, S., & Cockburn, A. (2009). Evaluating reading and analysis tasks on mobile devices: A case study of tilt and flick scrolling. Paper presented at the The 21st Annual Conference of the Australian Computer-Human Interaction Melbourne, Australia.
• Fithian, R., Iachello, G., Moghazy, J., Pousman, Z., & Stasko, J. (2003). The design and evaluation of a mobile location-aware handheld event planner. Paper presented at the the 5th International Symposium on Human-Computer Interaction with Mobile Devices and Services, Mobile HCI 2003, Udine, Italy.
• Glass, G., McGaw, B., & Smith, M. (1981). Meta-analysis in social research. Sage Publications.
• Goldstein, M., Alsio, G., & Werdenhoff, J. (2002). The media equation does not always apply: People are not polite towards small computers. Personal and Ubiquitous Computing, 6, 87-96.
• Gupta, D. D., & Sharma, A. (2009). Customer loyalty and approach of service providers: An empirical study of mobile airtime service industry in India. Services Marketing Quarterly, 30(4), 342 - 364.
• Han, S. H., Yun, M. H., Kwahk, J., & Hong, S. W. (2001). Usability of consumer electronic products. International Journal of Industrial Ergonomics, 28(3-4), 143-151.
• Hassanein, K., & Head, M. (2003). The impact of product type on website adoption constructs. Paper presented at the the Sixth International Conference on Electronic Commerce Research (ICECR6), Dallas, Texas.
• Heyer, C., Brereton, M., & Viller, S. (2008). Cross-channel mobile social software: An empirical study. Paper presented at the Conference on Human Factors in Computing Systems, Florence, Italy.
• Hinckley, K., Pierce, J., Sinclair, M., & Horvitz, E. (2000). Sensing techniques for mobile interaction. Paper presented at the UIST2000, San Diego, CA, USA.
• Hornbaek, K., & Law, E. L.-C. (2007). Meta-analysis of correlations among usability measures. Paper presented at the CHI 2007 San Jose, California, USA.
• Hsu, C. L., Lu, H. P., & Hsu, H. H. (2007). Adoption of the mobile Internet: An empirical study of multimedia message service (MMS). Omega, 35(3), 715-726.<
• Huang, S.-C., Chou, I.-F., & Bias, R. G. (2006). Empirical evaluation of a popular cellular phone's menu system: Theory meets practice. Journal of Usability Studies 1(2), 91-108.
• Hummel, K. A., Hess, A., & Grill, T. (2008). Environmental context sensing for usability evaluation in mobile HCI by means of  small wireless sensor networks. Paper presented at the Tthe 6th International Conference on Advances in Mobile Computing and Multimedia, Linz, Austria.
• ISO. (2004). Ergonomic requirements for office work with visual display terminals. Par 11: Guidance on usability. London: International Standards Organization.
• James, C. L., Reischel, K. M. (2001). Text input for mobile devices: Comparing model prediction to actual performance. CHI2001, Seattle, WA, USA.
• Jeng, J. (2005). What is usability in the context of the digital library and how can it be measured? Information Technology and Libraries, 24(2), 47-56.
• Jones, M., Buchanan, G., & Thimbleby, H. (2002). Sorting out searching on small screen devices. Paper presented at the Mobile HCI 2002, Pisa, Italy.
• Jones, S., Jones, M., Marsden, G., Patel, D., & Cockburn, A. (2005). An evaluation of integrated zooming and scrolling on small screens. International Journal of Human-Computer Studies, 63(3), 271-303.
• Juola, J., & Voegele, D. (2004). First time usability testing for Bluetooth-enabled devices. The University of Kansas.
• Kaasinen, E. (2003). User needs for location-aware mobile services. Personal and Ubiquitous Computing, 7(1), 70-79.
• Kaikkonen, A. (2005). Usability problems in today’s mobile Internet portals. Paper presented at the 2nd International Conference on Mobile Technology, Applications and Systems.
• Kaikkonen, A., Kallio, T., Kekäläinen, A., Kankainen, A., & Cankar, A. (2005). Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability Studies, 1(1), 4-16.
• Kallinen, K. (2004). The effects of background music on using a pocket computer in a cafeteria: Immersion, emotional responses, and social richness of medium. Paper presented at the Conference on Human Factors in Computing Systems Vienna, Austria
• Kargin, B., Basoglu, N., & Daim, T. (2009). Factors affecting the adoption of mobile services. International Journal of Services Sciences 2(1), 29-52.
• Keeker, K. (1997). Improving web-site usability and appeal: Guidelines compiled by MSN usability research. Retrieved from http://msdn.microsoft.com/en-us/library/cc889361(v=office.11).aspx 
• Khalifa, M., & Cheng, S. (2002). Adoption of mobile commerce: Role of exposure. Proceedings of the 35th Hawaii International Conference on System Sciences
• Kim, H., Kim, J., & Lee, Y. (2005). An empirical study of use of contexts in the mobile Internet, Focusing on the usability of information architecture. Information Systems Frontier, 7(2), 175-186.
• Kim, H., Kim, J., Lee, Y., Chae, M., & Choi, Y. (2002). An empirical study of the use contexts and usability problems in mobile Internet. Paper presented at the The 35th Hawaii International Conference on System Sciences
• Kim, H. W., Chan, H. C., & Gupta, S. (2007). Value-based adoption of mobile Internet: An empirical investigation. Decision Support Systems, 43(1), 111-126.
• Kim, S., Lee, I., Lee, K., Jung, S., Park, J., Kim, Y. B., et al. (2010). Mobile web 2.0 with multi-display buttons. Communications of the ACM, 53(1), 136-141.
• King, S. O., & Mbogho, A. (2009). Evaluating the usability and suitability of mobile tagging media in educational settings in a developing country. Paper presented at the IADIS International Conference Mobile Learning 2009, Barcelona, Spain.
• Kjeldskov, J., & Graham, C. (2003). A review of mobile HCI research methods. Paper presented at the The 5th International Mobile HCI 2003 conference, Udine, Italy.
• Kjeldskov, J., Skov, M. B., & Stage, J. (2010). A longitudinal study of usability in health care—Does time heal? International Journal of Medical Informatics, 79(6), 135-143.
• Kleijnen, M., Ruyter, K., & Wetzels, M. (2007). An assessment of value creation in mobile service delivery and the moderating role of time consciousness. Journal of Retailing, 83(1), 33-46.
• Kofod-Petersen, A., Gransæther, P. A., & Krogstie, J. (2010). An empirical investigation of attitude towards location-aware social network service. International Journal of Mobile Communications, 8(1), 53-70.
• Koivumäki, T., Ristola, A., & Kesti, M. (2006). Predicting consumer acceptance in mobile services: empirical evidence from an experimental end user environment. International Journal of Mobile Communications, 4(4), 418-435.
• Koltringer, T., & Grechenig, T. (2004). Comparing the immediate usability of Graffiti 2 and Virtual Keyboard. Paper presented at the Conference on Human Factors in Computing Systems, Vienna, Austria.
• Kowatsch, T., Maass, W., & Fleisch, E. (2009). The use of free and paid digital product reviews on mobile devices in in-store purchase situations. Paper presented at the The 4th Mediterranean Conference on Information Systems, Athens, Greece.
• Kurniawan, S. (In Press). Older people and mobile phones: A multi-method investigation. International Journal of Human-Computer Studies.
• Kwahk, J., & Han, S. H. (2002). A methodology for evaluating the usability of audiovisual consumer electronic products. Applied Ergonomics, 33, 419-431.
• Langan-Fox, J., Platania-Phung, C., & Waycott, J. (2006). Effects of advance organizers, mental models and abilities on task and recall performance using a mobile phone network. Applied Cognitive Psychology, 20(9), 1143-1165.
• Lee, C. C., Cheng, H. K., & Cheng, H. H. (2007). An Empirical study of mobile commerce in insurance industry: Task-technology fit and individual differences. Decision Support Systems, 43(1), 95-110.
• Lee, Y. E., & Benbasat, I. (2003). A framework for the study of customer interface design for mobile commerce. International Journal of Electronic Commerce, 46(12), 48-52.
• Lehikoinen, J., & Salminen, I. (2002). An empirical and theoretical evaluation of BinScroll: A rapid selection technique for alphanumeric lists. Personal and Ubiquitous Computing, 6, 141-150.
• Li, W., & McQueen, R. J. (2008). Barriers to mobile commerce adoption: an analysis framework for a country-level perspective. International Journal of Mobile Communications 6(2), 231-257.
• Li, Y. M., & Yeh, Y. S. (2010). Increasing trust in mobile commerce through design aesthetics. Computers in Human Behavior, 26(4), 673-684.
• Liang, T. P., Huang, C. W., & Yeh, Y. H. (2007). Adoption of mobile technology in business: A fit-viability model. International Material Data System, 107 (8), 1154-1169.
• Licoppe, C., & Heurtin, J. P. (2001). Managing one’s availability to telephone communication through mobile phones: A French case study of the development of dynamics of mobile phone use. Personal and Ubiquitous Computing, 5, 99-108.
• Lin, M., Goldman, R., Price, K. J., Sears, A., & Jacko, J. (2007). How do people tap when walking? An empirical investigation of nomadic data entry. International Journal of Human-Computer Studies, 65(9), 759-769.
• Lindroth, T., Nilsson, S., & Rasmussen, P. (2001). Mobile usability—rigour meets relevance when usability goes mobile. Paper presented at the IRIS24, Ulvik, Norway.
• Ling, C., Hwang, W., & Salvendy, G. (2006). Diversified users' satisfaction with advanced mobile phone features. Universal Access in the Information Society, 5(2), 239-249.
• Ling, R. (2001). We release them little by little: Maturation and gender identity as seen in the use of mobile telephony. Personal and Ubiquitous Computing, 5, 123-136.
• Lipsey, M., & Wilson, D. (2000). Practical meta-analysis. Thousand Oaks, CA: Sage Publications.
• MacKenzie, I. S., Kober, H., Smith, D., Jones, T., & Skepner, E. (2001). LetterWise: Prefix-based disambiguation for mobile text input. Paper presented at the UIST 2001.
• Maguire, M. (2001). Methods to support human-centered design. International Journal of Human-Computer Interaction, 55, 587-634.
• Mallat, N. (2007). Exploring consumer adoption of mobile payments—A qualitative study. Journal of Strategic Information Systems, 16(4), 413-432.
• Mao, E., Srite, M., Thatcher, J. B., & Yaprak, O. (2005). A research model for mobile phone service behaviors: Empirical validation in the U.S. and Turkey. Journal of Global Information Technology Management, 8(4), 7.
• Massey, A. P., Khatri, V., & Ramesh, V. (2005). From the web to the wireless web: Technology readiness and usability. Paper presented at the the 38th Annual Hawaii International Conference on System Sciences (HICSS'05).
• Massimi, M., & Baecker, R. M. (2008). An empirical study of seniors’ perceptions of mobile phones as memory aids. In A. Mihailidis, J. Boger, H. Kautz & L. Normie (Eds.), Technology and aging—Selected Papers from the 2007 International Conference on Technology and Aging: Vol. 21 (pp. 59-66).
• Merisavo, M., Vesanen, J., Arponen, A., Kajalo, S., & Raulas, M. (2006). The effectiveness of targeted mobile advertising in selling mobile services: An empirical study. International Journal of Mobile Communications, 4(2), 119-127.
• Nagata, S. F. (2003). Multitasking and interruptions during mobile web tasks. Paper presented at the Human Factors and Ergonomics Society 47th Annual Meeting.
• Nah, F. F., Siau, K., & Sheng, H. (2005). The value of mobile applications: A utility company study. Communications of the ACM, 48(2), 85-90.
• Nielsen, C. M., Overgaard, M., Pedersen, M. B., Stage, J., & Stenild, S. (2006). It's worth the hassle!: The added value of evaluating the usability of mobile systems in the field. Paper presented at the 4th Nordic conference on Human-computer interaction.
• Nielsen, J. (1993). Usability engineering.New York: AP Professional.
• Nielsen, J., & Levy, J. (1994). Measuring usability: Preference vs. performance. Communications of the ACM, 37(4), 66-75.
• Olmsted, E. L. (2004). Usability study on the use of handheld devices to collect census data. Paper presented at the Professional Communication Conference.
• Pagani, M. (2004). Determinants of adoption of third generation mobile multimedia services. Journal of Interactive Marketing, 18(3), 46.
• Palen, L., & Salzman, M. (2002). Beyond the handset: Designing for wireless communications usability. ACM Transactions on Human Computer Interaction, 9(2), 125-151.
• Palen, L., Salzman, M., & Youngs, E. (2001). Discovery and integration of mobile communications in everyday life. Personal and Ubiquitous Computing, 5, 109-122.
• Poupyrev, I., Maruyama, S., & Rekimoto, J. (2002). Ambient touch: Designing tactile interfaces for handheld devices. Paper presented at the UIST2002, Paris, France.
• Pousttchi, K., & Thurnher, B. (2006). Understanding effects and determinants of mobile support tools: A usability-centered field study on IT service technicians. Paper presented at the ICMB '06, International Conference on Mobile Business.
• Qiu, M. K., Zhang, K., & Huang, M. (2004). An empirical study of web interface design on small display devices. Paper presented at the IEEE/WIC/ACM International Conference on Web Intelligence (WI' 04).
• Rodden, K., Milic-Frayling, N., Sommerer, R., & Blackwell, A. (2003). Effective web searching on mobile devices. Paper presented at the Proceedings of the 17th Annual Conference on Human-Computer Interaction, Bath, United Kingdom.
• Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage Publications.
• Ross, D. A., & Blasch, B. B. (2002). Development of a wearable Computer Orientation System. Personal and Ubiquitous Computing, 6, 49-63.
• Roto, V., Popescu, A., Koivisto, A., & Vartiainen, E. (2006). Minimap: A web page visualization method for mobile phones. Paper presented at the ACM CHI 2006, Montreal, QC, Canada.
• Ryan, C., & Gonsalves, A. (2005). The effect of context and application type on mobile usability: An empirical study. Paper presented at the Twenty-eighth Australasian conference on Computer Science.
• Sarker, S., & Wells, J. (2003). Understanding mobile handheld device use and adoption. Communications of the ACM, 46(12), 35-40.
• Seth, A., Momaya, K., & Gupta, H. M. (2008). Managing the customer perceived service quality for cellular mobile telephony: An empirical investigation. Vikalpa: The Journal for Decision Makers, 33(1), 19-34.
• Shackel, B. (1991). Usability-context, framework, definition, design and evaluation. In B. Shackel & S. Richardson (Eds.), Human Factors for Informatics Usability (pp. 21-38). Cambridge: Cambridge University Press.
• Shackel, B. (2009, December, In Memoriam: Professor Brian Shackel 1927-2007), Usability – Context, framework, definition, design and evaluation. Interacting with Computers, 21 (5-6), 339-346.
• Shami, N. S., Leshed, G., & Klein, D. (2005). Context of use evaluation of peripheral displays INTERACT 2005, LNCS#3585, 579-587.
• Sodnik, J., Dicke, C., Tomazic, S., & Billinghurst, M. (2008). A user study of auditory versus visual interfaces for use while driving. International Journal of Human-Computer Studies, 66(5), 318-332.
• Strom, G. (2001). Mobile devices as props in daily role playing. Paper presented at the Mobile HCI 2001, Lille, France.
• Suzuki, S., Nakao, Y., Asahi, T., Bellotti, V., Yee, N., & Fukuzumi, S. (2009). Empirical comparison of task completion time between mobile phone models with matched interaction sequences. Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction (pp. 114-122). San Diego, CA: Springer Berlin/Heidelberg.
• Svanæsa, D., Alsosa, O. A., & Dahla, Y. (2010). Usability testing of mobile ICT for clinical settings: Methodological and practical challenges. International Journal of Medical Informatics, 79(4), 24-34.
• Tarasewich, P. (2003). Designing mobile commerce applications. Communications of the ACM, 46(12), 57-60.
• Thimbleby, H., Cairns, P., & Jones, M. (2001). Usability analysis with Markov models. ACM Transactions on Computer-Human Interaction, 8(2), 69-73.
• Thomas, P., & Macredie, R. (2002). Introduction to the new usability. ACM Transactions on Computer-Human Interaction, 9(2), 69-73.
• UMTS-Forum. (2005). Magic mobile future 2010-2020 Report No 37. London, UK: UMTS Forum 2005.
• Venkatesh, V., & Ramesh, V. (2006). Web and wireless site usability: Understanding differences and modeling use. MIS Quarterly, 30(1), 181-206.
• Venkatesh, V., Ramesh, V., & Massey, A. P. (2003). Understanding usability in mobile commerce. Communications of the ACM, 46(1246), 53-56.
• Wang, W., Zhong, S., Zhang, Z., Lv, S., & Wang, L. (2009). Empirical research and design of M-learning system for college English. Learning by playing. Game-based education system design and development (pp. 524–535). Berlin/Heidelberg: Springer.
• Waterson, S., Landay, J. A., & Matthews, T. (2002). In the lab and out in the wild: Remote web usability testing for mobile devices. Paper presented at the Conference on Human Factors in Computing Systems Minneapolis, Minnesota, USA
• Wigdor, D., & Balakrishnan, R. (2003). TiltText: Using tilt for text input to mobile phones. Paper presented at the the 16th Annual ACM UIST Symposium on User Interface Software and Technology.
• Wu, J.-H., & Wang, S.-C. (2005). What drives mobile commerce? An empirical evaluation of the revised technology acceptance model. Information and Management 42 (5), 719-729.
• Xu, D. J., Liao, S. S., & Li, Q. (2008). Combining empirical experimentation and modeling techniques: A design research approach for personalized mobile advertising applications Decision Support Systems, 44(3), 710-724.
• Yuan, Y., & Zheng, W. (2005). Stationary work support to mobile work support: A theoretical framework. Paper presented at the International Conference on Mobile Business (ICMB 2005) Sidney, Australia.

 

Previous | Next