Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques

In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI 2022.
National Yang Ming Chiao Tung University
Aalto User Interfaces Lab
Aalto User Interfaces Lab
What are the benefits and limitations of an optimization-driven desgin process?
  • Designers often need to explore vast design spaces when determining the parameter settings of an interaction technique.
  • Multi-objective Bayesian optimization (MOBO) is a machine learning technique which can help to guide the design process:
    • Bayesian optimization is a state-of-the-art optimization algorithm for black-box functions.
    • Bayesian optimization is able to quickly identify the design instance which maximizes the objective value(s).
    • Instead of searching for one optimal design instance, MOBO searches for the Pareto-optimal designs.
  • This paper contributes an experiment comparing designer-led versus optimizer-driven approaches for tackling a 3D touch interaction design challenge.
  • Our key findings include:
    • The optimizer-driven design process can identify better designs while reducing the designer's mental demand.
    • MOBO promotes wider exploration.
    • However, the designers may lose their sense of agency during the design process.
Abstract

Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. In HCI, design optimization problems are often exceedingly complex, involving multiple objectives and expensive empirical evaluations. Model-based computational design algorithms assist designers by generating design examples during design, however they assume a model of the interaction domain. Black box methods for assistance, on the other hand, can work with any design problem. However, virtually all empirical studies of this human-in-the-loop approach have been carried out by either researchers or end-users. The question stands out if such methods can help designers in realistic tasks. In this paper, we study Bayesian optimization as an algorithmic method to guide the design optimization process. It operates by proposing to a designer which design candidate to try next, given previous observations. We report observations from a comparative study with 40 novice designers who were tasked to optimize a complex 3D touch interaction technique. The optimizer helped designers explore larger proportions of the design space and arrive at a better solution, however they reported lower agency and expressiveness. Designers guided by an optimizer reported lower mental effort but also felt less creative and less in charge of the progress. We conclude that human-in-the-loop optimization can support novice designers in cases where agency is not critical.

Watch the eight-minute presentation video to learn more:

Materials

Our implementation (in Python) of the optimizer and the 3D touch interaction is available on Github along with examples and instructions.

Publication
paper

PDF, 4.2 MB
Liwei Chan, Yi-Chi Liao, George B. Mo, John Dudley, Chun-Lien Cheng, Per Ola Kristensson, Antti Oulasvita. 2022. Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). 

	    
@inproceedings{investigating_chan,
    address = {New York, NY, USA},
    title = {Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques},
    url = {https://doi.org/10.1145/3491102.3501850},
    doi = {10.1145/3491102.3501850},
    booktitle = {Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems},
    author = {Chan, Liwei and Liao, Yi-Chi and Mo, George B. and Dudley, John J. and Cheng, Chun-Lien and Kristensson, Per Ola and Oulasvirta, Antti},
    year = {2022},
}
Contact

For questions and further information, please contact:

Liwei Chan

Email:
liweichan (at) cs.nycu.edu.tw

Acknowledgements: The research was supported by the Ministry of Science and Technology of Taiwan (MOST109-2628-E-009-010-MY3), the Finnish Center for Artifcial Intelligence (FCAI), Academy of Finland (grants 'OptiHAFE' and 'BAD'), and the Engineering and Physical Sciences Research Council (EPSRC EP/S027432/1).