Adapting User Interfaces with Model-based
Reinforcement Learning

In Proceedings of the ACM CHI Conference on Human Factors in Computing Systems, CHI 2021.
How can Artifial Intelligence be used to adapt User Interfaces?

Planning is a powerful tool for making series of decisions that can lead to success. Recent work in Artificial Intelligence (AI) has shown that remarkable capabilities can be achieved using Reinforcement Learning. AlphaGo is one such example, where an AI-powered agent mastered the incredibly challenging game of Go, and could consistently outperform expert human players.

We have now developed similar methods through with user interfaces (UI) can adapt themselves automatically to improve usability. The key to making good adaptations is to plan every change to the UI by fully considering its impact on usability — both benefits and costs to the user.

We formulate this problem of adapting an interface as a stochastic sequential decision-making problem where an adaptive system should:

  • decide what (or if) to adapt given some user observations from previous interactions
  • pick sequences of adaptations to maximise value over long term by consider both costs and benefits
  • act conservatively under uncertainty as the world is noisy and not fully known

Our work develops reinforcement learning methods for solving such problems:

  • Predictive HCI models are applied to simulate consequences of different adaptations ('rewards')
  • Monte-Carlo Tree Search (MCTS) is used to efficiently find promising adaptations through long-term planning
  • Deep neural networks provide reliable value estimations, enabling scaling to larger problem sizes and real-time computations

We showcase our approach for applications in adaptive menus that can reorganise themselves by swapping, moving, and grouping items to reduce average selection time for a user.

Abstract

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user – for example, due to surprise or relearning effort – or "trap" the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.

Watch the five-minute presentation video to learn more:

Materials

Our implementation (in Python) of the adaptive menus application is available on Github along with examples and instructions.

https://github.com/aalto-ui/chi21adaptive

Publication
paper

PDF, 5.3 MB
Kashyap Todi, Gilles Bailly, Luis A. Leiva, Antti Oulasvirta. 2021. Adapting User Interfaces with Model-based Reinforcement Learning. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). 


						
@inproceedings{todi21adaptive,
author = {Todi, Kashyap and Bailly, Gilles, and Leiva, Luis A., and and Oulasvirta, Antti},
title = {{Adapting User Interfaces with Model-based Reinforcement Learning}},
year = {2021},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3411764.3445497},
doi = {10.1145/3411764.3445497},
booktitle = {Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems},
keywords = {Adaptive User Interfaces, Reinforcement Learning, Predictive Models, Monte Carlo Tree Search},
series = {CHI '21}}
In The Media
On Twitter:

Read the entire thread

Contact

For questions and further information, please contact:
Kashyap Todi
kashyap.todi@gmail.com

Acknowledgements: This work has been funded by the Finnish Center for Artificial Intelligence (FCAI), Academy of Finland projects "Human Automata" and "BAD", Agence Nationale de la Recherche (grant number ANR-16-CE33-0023), and HumaneAI Net (H2020 ICT 48 Network of Centers of Excellence).