Linking Neural and Behavioral Data: Discriminative and Generative Models

Abstract

A central goal in neuroscience is to understand how neural activity gives rise to behavior. Encoding and decoding models formalize this relationship by estimating conditional distributions of neural activity given behavior or other covariates, and vice versa. Advances in neural and behavioral recording techniques now enable recordings of thousands of neurons during continuous behaviors beyond rigid trial structures. These developments pose a challenge to classical encoding and decoding methods, which often struggle with high dimensionality, variability, and non-linear relationships. Thus, the overarching aim of this thesis was to develop statistical approaches linking high-dimensional neural activity with continuous behavior.

The first contribution addresses whether the midbrain, in particular superior colliculus (SC), differentiates visual motion resulting from self-movement or object-movement. We tested this in virtual reality, where mice experienced identical visual loom for these two contexts. Because neural activity drives, but is also shaped by behavior, we had to account for behavioral differences when decoding context from neural activity. Using a multivariate discriminative decoding framework, we found that SC activity, particularly in intermediate layers, differs between contexts, even after controlling for behavior.

The second contribution directly accounts for this neural-behavioral bidirectionality, hidden in distinct encoding and decoding models. We developed a probabilistic latent variable model based on masked variational autoencoders (VAEs) to jointly model conditional distributions of neural activity and behavior. This framework allowed us to model and sample from the distribution over continuous behaviors given neural activity and to generate neural activity conditioned on unseen behavior. Masked VAEs provided calibrated uncertainty estimates, indicating higher uncertainty when predictions were likely wrong—an advance increasingly important for highly variable data and only achievable with probabilistic approaches.

In the third contribution, we extended these approaches to diffusion-based probabilistic models that enhance sampling fidelity and conditioning flexibility. To preserve low-dimensional neural representations and account for the discrete nature of neural spikes, we introduced Latent Diffusion for Neural Spiking Data (LDNS). LDNS enabled the extraction of behaviorally meaningful neural latents, and training a diffusion model directly on these latents generated realistic spiking data for various tasks. Flexible conditioning on scalars or entire time-series renders LDNS a highly powerful encoding model enabling scalable hypothesis generation.

Collectively, these works develop complementary statistical approaches for continuous neural-behavioral datasets. By integrating classical encoding and decoding with probabilistic deep generative models, this work scales classical analyses to large-scale datasets and highlights the importance of modeling variability and uncertainty.