Reinforced Holography

Summary

There is considerable heterogeneity in the activity patterns among neurons, especially in higher cortical areas. To better understand the function of this functional diversity it is therefore necessary to dissect neural activity patterns during behavior on the level of individual neurons. However, experimentally addressing the behavioral contributions of functionally distinct, but spatially intermingled neurons, has been challenging due to the difficulty of perturbing the activity of individual neurons in a precise spatiotemporal manner in vivo. In this joint project with the Diester Lab, we propose to overcome this challenge by using an inverse reinforcement learning approach to generate predictions based on the analysis of neural activity patterns of 2-photon imaging data. In this approach, we first extract a functional representation of the factors driving observed animal behavior, and link it to the measured neural activity in a second step. We then use this representation to make precise predictions about expected behavioural changes based on simulated trials with perturbed neuronal activity. 

Building on a recent technology which we developed, we will train a computational inverse reinforcement learning algorithm to predict the behaviour of subjects based on their neuronal activity (Kalweit et al., 2021). This technology is based on Inverse Q-learning (Kalweit et al., 2020) and will allow us to identify particularly relevant neurons for specific task aspects (e.g. correct choice, impatience measured as responses before the cue, late responses etc.). Based on these predictions we will then be able to target the identified cells via our holographic optogenetic stimulation unit. We will develop novel and improved ways of fast data handling, extraction and analysis to create a nearly closed loop situation in which we will be able to read neural activity via 2P imaging, extract activity patterns and create predictions via Inverse Q learning, and generate spatial masks for the holographic stimulation. With this approach we will be able to causally test the role of individual neurons and address the question of the critical threshold of perturbed neurons.