Understanding Visual Saliency in Mobile User Interfaces

In Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2020.
User Interfaces group, Aalto University
  • Luis A. Leiva Aalto University, Finland
  • Yunfei Xue Aalto University, Finland
  • Avya Bansal Indian Institute Of Technology, Goa, India
  • Hamed R. Tavakoli Nokia Technologies, Finland
  • Tuğçe Köroğlu Yildiz Technical University, Turkey
  • Jingzhou Du Huawei Technologies, China
  • Niraj R. Dayama Aalto University, Finland
  • Antti Oulasvirta Aalto University, Finland
Teaser image A UI for a mobile device can difer in visual features' distribution from the equivalent UI on a laptop computer. Shown here are the screen resolution, aspect ratio, and color distributions of a sample webpage. Red circles represent areas of foveal vision in viewing the UIs from a 60 cm distance.
Highlights
  • Presents the first empirical investigation of statistical efects of visual saliency in mobile UIs.
  • Reports validation of several well-known computational models, including state-of-the-art deep learning models.
  • Releases a high-quality dataset characterizing the attraction of visual attention in mobile UIs.
Abstract

For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs difer from these in several respects. We present indings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size afected saliency less. Classic, parameter-free saliency models showed a weak it with the data, and data-driven models improved signiicantly when trained speciically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the irst annotated dataset for investigating visual saliency in mobile UIs.

Materials

  • Visual saliency examples (click to enlarge)

    Example Example Example Example Example Example Example

  • Mobile UI Saliency Dataset: ZIP, 45 MB (65 MB uncompressed)

Publication
Paper

PDF, 2.4 MB
Luis A. Leiva, Yunfei Xue, Avya Bansal, Hamed R. Tavakoli, Tuğçe Köroğlu, Jingzhou Du, Niraj R. Dayama, and Antti Oulasvirta. 2020. Understanding Visual Saliency in Mobile User Interfaces. In Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI'20) 

                            
@inproceedings{leiva20_mobsal,
  author = {Luis A. Leiva and Yunfei Xue and Avya Bansal and Hamed R. Tavakoli and Tuğçe Köroğlu and Jingzhou Du and Niraj R. Dayama and Antti Oulasvirta},
  title = {Understanding Visual Saliency in Mobile User Interfaces},
  booktitle = {Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services},
  series = {MobileHCI'20},
  year = {2020},
  doi = {10.1145/3379503.3403557},
}
Contact

For questions and further information, please contact:

Luis A. Leiva

Email:
luis.leiva (at) aalto.fi

Acknowledgements: We thank Marko Repo for his help with UI element annotation and the anonymous referees for their feedback. We also thank the computational resources provided by the Aalto Science-IT project. This work has been supported by the Academy of Finland (grant no. 318559), the Huawei UCD Center, and the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 637991).