The Secrets of “Moving” Toward Success: Reinforcement Learning in Human Sensorimotor Control
Loading...
Authors
Zhu, Tianyao
Date
2025-07-23
Type
thesis
Language
eng
Keyword
Reinforcement learning , Sensorimotor control , Motor learning , Reaching movement
Alternative Title
Abstract
Motor reinforcement learning is the process of optimizing movements through feedback from reward outcomes in complex environments. This thesis investigates key mechanisms of reinforcement learning in human sensorimotor control using behavioral experiments, computational modeling, and neuroimaging. First, we examined the interaction between error-based and reinforcement learning in a center-out reaching task including both bias and noise. Participants moved a cursor, controlled by the hand, to one of three optional targets. The cursor position was altered with a visuomotor rotation that has a constant component (bias) as well as random component (noise) that depended on target direction. We found that adaptation to bias often interfered with identifying low-noise target directions, revealing a trade-off between local adaptation and global exploration. Second, we investigated how humans use reinforcement learning strategies to represent reachable space and guide hand movements. In a haptic maze task, participants moved a robotic handle to a cued target while receiving haptic feedback about the invisible maze. Their behavior shifted from Model-Based to Model-Free control, contingent on uncertainty and repetition, flexibly balancing planning and caching. Third, we explored the role of working memory in temporal credit assignment using a multi-step reaching task in which the participant moved a cursor across a set of bars, receiving a score based on how close the crossing locations were to hidden targets on the bars. Across conditions, we manipulated memory demands by varying the number of bars, the reward feedback scheme, and the visibility of past crossing locations. Performance declined with greater memory demands, and a Bayesian Actor-Critic model revealed distinct effects of observation and process uncertainty on learning dynamics, showing systematic influence of memory constraints. Finally, using fMRI and manifold learning, we identified a low-dimensional neural space that tracked learning-related changes in whole-brain organization. Periods of accelerated learning were characterized by significant manifold contractions across multiple brain regions, including limbic and hippocampal cortex as well as the cerebellum, reflecting enhanced network integration. Altogether, this thesis demonstrates that human sensorimotor reinforcement learning arises from an interplay between adaptive control strategies, cognitive resource limitations, and large-scale neural dynamics.
Description
Citation
Publisher
License
Queen's University's Thesis/Dissertation Non-Exclusive License for Deposit to QSpace and Library and Archives Canada
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.