Multimodal data fusion on the cortex: neuroscientific basis and challenges facing joint inference

A few months ago, my colleagues suggested that I attempt to give a neuroscience overview to the rest of our group. As computer scientists and image processing specialists working in the medical imaging arena, we know surprisingly little about neuro-anatomy or the origins of the signals we are are  processing.

As neuroscience methods developers we need to continue to work closely with clinical colleagues. All too easily, we can get swept up the the mathematical intricacies of a method and lose sight of the fact that this is a clinical problem much to be contributed to by those with accumulated knowledge on the complex aspects of neuro-anatomy

Despite still being woefully ignorant myself, I had been lucky to pick up a little relevant knowledge during my close collaboration with clinical colleagues on the Human Connectome project. This presentation is my attempt to begin to address some of the main issues, that in my opinion, influence our ability to compare multiple imaging modalities during connectomics research. In the header figure is some of the modalilties we can work with, clockwise from top left: cortical folding, resting state functional MRI, cortical myelin mapping and retinotopy.

One very important question still to be addressed is why cortical folding patterns do not seem to follow patterns of regional functional specialism. Is this because folding and function genuinely do not match? Or, more likely in my opinion, since we know that the development of cortical folds is linked to the formation of neuronal connections, is it that our methods are not yet up to the job of learning the complex relationships between genetics and its links to functional and structural topological variability in the cortex?

For Presentation, Click here: multi-modalfusion

Below – an overview of my key research. Long term, I wish to be able to co-align multiple different brain imaging modalities on a common cortical surface, but the complexities introduced by measurement and modelling errors noise, and lack of complete knowledge of brain anatomy makes this a difficult problem.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s