How We Type: Eye and Finger Movement Strategies in Mobile Typing

In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI 2020.
What do you know about typing on a mobile soft keyboard?
  • Rivalry for attention between screen areas:
  • Even though mobile devices are small, one cannot monitor the keyboard and the text display at the same time. --> A strategy must be selected that determines which to give attention and when.
  • Speed-accuracy tradeoff:
  • A strategy strikes a compromise between the cost of NOT correcting errors early and the time lost in glancing at the text display, when the fingers cannot be guided. More errors --> more attention to text display --> worse WPM.
Abstract

Relatively little is known about eye and finger movement in typing with mobile devices. Most prior studies of mobile typing rely on log data, while data on finger and eye movements in typing come from studies with physical keyboards. The paper presents new findings from a transcription task with mobile touchscreen devices. Movement strategies were found to emerge in response to sharing of visual attention: attention is needed for guiding finger movements and detecting typing errors. In contrast to typing on physical keyboards, visual attention is kept mostly on the virtual keyboard, and glances at the text display are detrimental to performance. When typing with two fingers, users, while making more errors, manage to detect and correct them more quickly, which explains part of the known superiority of two-thumb typing over one-finger typing. We release the extensive dataset on everyday typing on smartphones.

Materials
Press Releases
  1. in English
  2. in Finnish

Publication
paper

PDF, 2,9 MB
Jiang, X., Li, Y., Jokinen, J. P. P., Hirvola, V., Oulasvirta, A., & Ren, X. 2020. How We Type: Eye and Finger Movement Strategies in Mobile Typing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20).

	    
@inproceedings{Jiang2020,
author = {Jiang, Xinhui and Li, Yang, and Jokinen, Jussi P P and Hirvola, Viet Ba and Oulasvirta, Antti and Ren, Xiangshi},
booktitle = {Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20)},
title = {{How We Type: Eye and Finger Movement Strategies in Mobile Typing.}},
year = {2020}
}
				
			
Frequently Asked Questions
Q: What kind of setup did you have for this? real phone? head-worn eye tracker?
A: Yes, we used a head-worn eye tracker (SMI W2), marker-based optical motion tracking (OptiTrack Prime13), and key logging. --> Synchronized and registered in the same coordinate system.

Q: ...?
A: ...

Q: ...?
A: ...
Contact

For questions and further information, please contact:

Xinhui Jiang

Email:
jiangxinhui0077@gmail.com

This work has received funding from the joint JST–AoF project "User Interface Design for the Ageing Population" (AoF grant 291556) as an activity of FY2014 Strategic International Collaborative Research Program (SICORP), and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 637991).