UEyes: Understanding Visual Saliency across User Interface Types

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI 2023), Hamburg, Germany.

Download Video(MP4, 57.7 MB)

Abstract

While user interfaces (UIs) display elements such as images and text in a grid-based layout, UI types differ significantly in the number of elements and how they are displayed. For example, webpage designs rely heavily on images and text, whereas desktop UIs tend to feature numerous small images. To examine how such differences affect the way users look at UIs, we collected and analyzed a large eye-tracking-based dataset, UEyes (62 participants and 1,980 UI screenshots), covering four major UI types: webpage, desktop UI, mobile UI, and poster. We analyze its differences in biases related to such factors as color, location, and gaze direction. We also compare state-of-the-art predictive models and propose improvements for better capturing typical tendencies across UI types. Both the dataset and the models are publicly available.

Resources and Downloads



Correction:

    We regenerated Table 2 in the paper after fixing some calibration error in the dataset. Sorry for the inconvenience.

    correction

Citation

BibTeX, 1 KB

@inproceedings{jiang2023ueyes,
	author = {Jiang, Yue and Leiva, Luis A. and Rezazadegan Tavakoli, Hamed and R. B. Houssel, Paul and Kylm\"{a}l\"{a}, Julia and Oulasvirta, Antti},
	title = {UEyes: Understanding Visual Saliency across User Interface Types},
	year = {2023},
	isbn = {9781450394215},
	publisher = {Association for Computing Machinery},
	address = {New York, NY, USA},
	url = {https://doi.org/10.1145/3544548.3581096},
	doi = {10.1145/3544548.3581096},
	booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},
	articleno = {285},
	numpages = {21},
	keywords = {Computer Vision, Eye Tracking, Deep Learning, Interaction Design, Human Perception and Cognition},
	location = {Hamburg, Germany},
	series = {CHI '23}
}

Contact

For questions and clarifications, please get in touch with:
Yue Jiang yuenj.jiang@gmail.com