Decision in space

Anggraini, Dian (2018): Decision in space: reinforcement learning models predict individual differences in and neural correlates of strategy adoption in spatial navigation. Dissertation, LMU München: Graduate School of Systemic Neurosciences (GSN)

Abstract

Human navigation is generally believed to rely on two types of strategy adoption, route- based and map-based strategies. Both types of navigation require making spatial decisions along the traversed way. Nevertheless, formal computational and neural links between navigational strategies and mechanisms of value based decision making have so far been underexplored in humans. Here, we employed functional magnetic resonance imaging (fMRI) while subjects located different target objects in a virtual environment. We then modelled their paths using reinforcement learning (RL) algorithms, which successfully explain decision behaviour and its neural correlates. Our results show that subjects used a mixture of route and map-based navigation, and their paths could be well explained by the model-free and model-based RL algorithms. Furthermore, the value signals of model-free choices during route-based navigation modulated the BOLD signals in the ventro-medial prefrontal cortex (vmPFC). On the contrary, the BOLD signals in parahippocampal and medial temporal lobe (MTL) regions pertained to model- based value signals during map-based navigation. Our findings suggest that the brain might share computational mechanisms and neural substrates for navigation and value- based decisions, such that model-free choice guides route-based navigation and model- based choice directs map-based navigation. These findings open new avenues for computational modelling of wayfinding by directing attention to value-based decision, differing from common direction and distances approaches. The ability to find one’s way in a complex environment is crucial to everyday functioning. This navigational ability relies on the integrity of several cognitive functions and different strategies, route and map-based navigation, that individuals may adopt while navigating in the environment. As the integrity of these cognitive functions often decline with age, navigational abilities show marked changes in both normal aging and dementia. Combining a wayfinding task in a virtual reality (VR) environment and modeling technique based on reinforcement learning (RL) algorithms, we investigated the effects of cognitive aging on the selection and adoption of navigation strategies in human. The older participants performed the wayfinding task while undergoing functional Magnetic Resonance Imaging (fMRI), and the younger participants performed the same task outside the MRI machine. Compared with younger participants, older participants traversed a longer distance. They also exhibited a higher tendency to repeat previously established routes to locate the target objects. Despite these differences, the traversed paths in both groups could be well explained by the model-free and model-based RL algorithms. Furthermore, neuroimaging results from the older participants show that BOLD signal in the ventromedial prefrontal cortex (vmPFC) pertained to model-free value signals. This result provide evidence on the utility of the RL algorithms to explain how the aging brain computationally prefer to rely more on the route-based navigation.

• Dokumententyp: Dissertationen (Dissertation, LMU München)
• Keywords: fMRI, spatial navigation, value-based decision making, reinforcement learning
• Themengebiete: 500 Naturwissenschaften und Mathematik ; 500 Naturwissenschaften und Mathematik > 570 Biowissenschaften, Biologie
• Fakultäten: Graduate School of Systemic Neurosciences (GSN)
• Sprache der Hochschulschrift: Englisch
• Datum der mündlichen Prüfung: 20. September 2018
• 1. Bericht­erstatter:in: Wunderlich, Klaus