Neuromechanics of a Button Press

In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2018), ACM Press.
The humble button press is more complex than meets the eye

Pressing a button appears effortless and one easily dismisses how challenging it is. This research project created detailed simulations of button-pressing with the goal of producing human-like presses. The project was triggered by admiration of our remarkable capability to adapt button-pressing We push a button on a remote controller differently than a piano key. The press of a skilled user is surprisingly elegant when looked at terms of timing, reliability, and energy use. People successfully press buttons without ever knowing the inner workings of a button. It is essentially a black box to our motor system. On the other hand, we also fail to activate buttons, and some buttons are known to be worse than others. To improve buttons, we need to understand the motor control processes and the effects of design parameters like button dimensions, materials, force curves, activation point, and feedback.

Overview of theory
Neuromechanics views button-pressing as optimal control limited by physical, neural, and sensory bounds. It proceeds from the biological fact that the central nervous system has no direct access to the physical body (here, finger) it is moving. It is enveloped by neural interfaces. Our account of the motor controller rests on the convergence of two principles:

Perceptual control: With no direct access to the button, the motor system relies on its limited sensations to adapt the motor commands. We formulate a perceptual control objective for button-pressing: the goal of the motor system is to control its own sensations arising as a consequence of the button press. In particular, we model timing estimates that a user makes based on noisy sensory signals.

Probabilistic internal model: To control a button, the brain must learn to predict the consequences of the actions it takes. We build on previous work on predictive processing and the Bayesian brain hypothesis to posit a probabilistic internal model. It tries to learn to predict the perceptual effects that its open-loop commands have over episodes of attempts with a button.

NEUROMECHANIC is a computational implementation. It predicts optimal button-pressing performance under constraints. In particular, it predicts an upper limit to button-pressing performance as bounded by neural, physical, and physiological factors in the model. The model can be used as a workbench for exploring theoretical ideas or comparing button designs.

NEUROMECHANIC: A computational implementation of the model

Improvements to button designs

One implication of the theory is that activating the button at the moment when the sensation is strongest will help users better rhythm their keypresses. To test this hypothesis, we created a new method for changing the way buttons are activated. The technique is called Impact Activation. Instead of activating the button at first contact, it activates it when the button cap or finger hits the floor with maximum impact. The technique was 94% better in rapid tapping than the regular activation method for a push-button (Cherry MX switch) and 37% than a regular touchscreen button using a capacitive touch sensor.

Article and Citation
Best Paper Honorable Mention

PDF (3.2 MB)
Oulasvirta, A., Kim, S., & Byungjoo, L. 2018.
Neuromechanics of a Button Press
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ’18), ACM Press, pp. 4099-4112.
Best Paper Honorable Mention

Abstract: To press a button, a finger must push down and pull up with the right force and timing. How the motor system succeeds in button-pressing, in spite of neural noise and lacking direct access to the mechanism of the button, is poorly understood. This paper investigates a unifying account based on neuromechanics. Mechanics is used to model muscles controlling the finger that contacts the button. Neurocognitive principles are used to model how the motor system learns appropriate muscle activations over repeated strokes though relying on degraded sensory feedback. Neuromechanical simulations yield a rich set of predictions for kinematics, dynamics, and user performance and may aid in understanding and improving input devices. We present a computational implementation and evaluate predictions for common button types.

     title = {Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18)},
     author = {Oulasvirta, Antti and Kim, Sunjun and Lee, Byungjoo},
     title = {Neuromechanics of a Button Press},
     pages = {4099--4112},
     year = {2018},
     doi = {10.1145/3173574.3174082}
     organization = {ACM Press}}

NEUROMECHANIC is a computational implementation released as a modeling workbench used in MATLAB. Button designs and user tasks can be explored by manipulating parameters as presented above in the Simulations section of paper. After optimization, the model outputs such values as means and statistical indices for task-level performance (button activation and perceptual error), cognition (p-centers), and dynamics and kinematics (FD, DV, pulp contraction, and muscle forces). A visual simulation of the dynamics is available also. A database of 20 FD curves of commercial button types is available too (see Data).

Media: Photography and videos of button-pressing

For more information, please contact:

Antti Oulasvirta

antti.oulasvirta (at)

+358 (09) 47001

Acknowledgements: This work was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement 637991) and by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2017R1C1B2002101). We thank Jong-In Lee for help with the video.