Selected Projects | Mahyar's world 🌏

Temporal Predictive Coding

Sun, 15 Dec 2024 00:00:00 +0000

Authors: Beren Millidge, Mufeng Tang, Mahyar Osanlouy, Nicol S. Harper, Rafal Bogacz
Publication: PLOS Computational Biology (April 2024)
Code: GitHub Repository
Read the paper: PLOS Computational Biology

Overview

How does the brain process dynamic sensory inputs; like tracking a moving object or predicting the next note in a melody? In this work, we propose temporal predictive coding (tPC), a biologically plausible neural framework that extends classical predictive coding to handle time-varying stimuli. By unifying Bayesian inference, neural dynamics, and synaptic plasticity, tPC explains how cortical circuits might learn to predict and adapt to continuously changing environments.

Key Contributions

A unified model for temporal prediction: tPC bridges predictive coding (for static inputs) and Kalman filtering (for dynamic systems), enabling online inference and learning without complex matrix operations.
Biologically plausible implementation: The model uses only local computations and Hebbian plasticity, aligning with known cortical microcircuit architecture.
Motion-sensitive receptive fields: When trained on natural movies, tPC neurons develop Gabor-like, direction-selective receptive fields—matching properties of early visual cortex neurons.
Nonlinear generalization: The framework extends to nonlinear systems (e.g., pendulum dynamics), outperforming linear models in complex tasks.

Methodology

Core Principles

Generative model: Hidden states evolve via a Markov process, with observations generated from hidden states.

$$ x_k = A f(x_{k-1}) + B u_k + \omega_x $$

$$ y_k = C f(x_k) + \omega_y $$

Free energy minimization: Neurons minimize prediction errors at two levels:
1. Sensory errors: Mismatch between predicted and actual observations.
2. Temporal errors: Mismatch between predicted and inferred hidden states.
Neural implementation:
- Explicit prediction error neurons or dendritic computations.
- Synaptic updates follow local Hebbian rules (e.g., ΔA ∝ prediction error × presynaptic activity).

Comparison to Kalman Filtering

tPC approximates Kalman filtering but avoids tracking posterior uncertainty.
Achieves similar accuracy in linear tracking tasks with simpler computations.

$$ \hat{x}_k^- = A \hat{x}_{k-1} + B u_k $$

$$ \hat{x}_k = \hat{x}_k^- + K(y_k - C\hat{x}_k^-) $$

$$ K = \Sigma_x C^T \left[C \Sigma_x C^T + \Sigma_y \right]^{-1} $$

Results

1. Robust Tracking of Dynamic States

tPC accurately infers position, velocity, and acceleration from noisy observations, even with scrambled sensory inputs.
Matches Kalman filter performance with 5 inference steps (MSE difference < 1%).

2. Learning Synaptic Weights

Hebbian plasticity enables online learning of dynamics (A, B) and observation (C) matrices.
Learns to predict observations even with random initial weights.

3. Biological Receptive Fields

Trained on natural movies, tPC develops:
- Spatiotemporal receptive fields with motion direction selectivity.
- Gabor-like filters resembling V1 simple cells. Representations developed by the model when trained with patches from movies of dynamic natural scenes. A: First 10 frames of 2 example training movies used in our experiments. Patches extracted from movies obtained at websites pexels.com, pixabay.com and commons.wikimedia.org (for wikimedia attributions see https://github.com/C16Mftang/temporal-predictive-coding). B: The projective fields C developed Gabor-like filters after training. C: Space-time receptive fields developed by hidden neurons of the tPC model. https://doi.org/10.1371/journal.pcbi.1011183.g006

4. Nonlinear Dynamics

Outperforms linear models in predicting pendulum motion (20% lower MSE).
Captures nonlinearities where linear approximations fail (e.g., large-angle oscillations). Simulations of the pendulum. A: A free-body diagram of a simple pendulum that has a mass m attached to a string with length $L$. Also shown are the forces applied to the mass. The restoring force $-mg sin \theta$ is a net force toward the equilibrium position. B: A phase portrait of the pendulum simulation showing the result of our linear versus nonlinear models prediction for the ground-truth data. The vector field (i.e. set of small arrows) was created by computing the derivatives of $\frac{d\theta_1}{dt}$ and $\frac{d\theta_2}{dt}$ at $t = 0$ a grid of 30 points over the range of $-\pi$ to $\pi$ and -4 to +4 for $\theta_1$ and $\theta_2$ respectively. C: The barplot shows the difference between the mean prediction errors of the linear model versus the nonlinear model from 100 simulations with varying noise profiles. The mean errors are significantly different $(p \ll 0.001)$ https://doi.org/10.1371/journal.pcbi.1011183.g007

Implications

Theoretical: Provides a unified account of static and dynamic sensory processing under predictive coding.
Experimental: Predicts neural responses to time-varying stimuli and receptive field properties.
Translational: Framework for neuromorphic hardware and efficient machine learning models.

3D Reconstruction of Human Olfactory Projection

Fri, 10 Nov 2023 00:00:00 +0000

Authors: Victoria F Low, Chinchien Lin, Shan Su, Mahyar Osanlouy, Mona Khan, Soroush Safaei, Gonzalo Maso Talou, Maurice A Curtis, Peter Mombaerts
Publication: Communications Biology (November 2024)
Code: Zenodo Repository
Interactive 3D Model: Neuroglancer Platform

Overview

This study presents a computational pipeline to reconstruct the 3D microanatomy of the human olfactory system, from the nasal cavity to the olfactory bulb—using fluorescence histology, deep learning, and high-performance computing (HPC). The workflow addresses challenges in processing terabyte-scale datasets and provides new insights into olfactory sensory neuron (OSN) distribution and axon trajectories.

Key Computational Contributions

First end-to-end pipeline for large-scale 3D reconstruction of human olfactory tissues (~7.45 cm³ specimen, 1234 sections).
CNN-based segmentation achieving Dice scores >0.85 for critical structures (OSNs, glomeruli, vasculature).
HPC-optimized registration reducing banana-effect artifacts via multi-resolution deformable alignment.
Public 3D dataset enabling interactive exploration of olfactory projection via Neuroglancer.

Methodology

Computational Pipeline

From fluorescence histology to 3D visualization – click to expand

Fluorescence Histology
- Quadruple staining: Hoechst (nuclei), UEA1 (OSNs), OMP (mature OSNs), VGLUT2 (axon terminals).
- Whole-slide scanning: 1.097 µm/pixel resolution, ~2.9 TB raw data.
CNN Segmentation
- Architecture: Modified 2D U-Net with 4 input channels (Hoechst + 3 markers).
- Training: Bootstrap approach with iterative ground truth expansion (20-45 sections/structure).
- Key layers:
```
# Simplified U-Net structure
encoder = [Conv2D(8→16→32), MaxPooling2D]
bottleneck = Conv2D(64) + Upsampling2D
decoder = [Conv2D(32→16→8), Concatenate(skip connections)]
```
HPC Registration
- Intra-block: Parallel registration of 247 blocks (5 sections each) using SimpleElastix.
- Inter-block: Banana-effect correction via affine + B-spline transformations between blocks.
- Metrics: Mutual information for intensity alignment, DSC for structural consistency.

Results

Segmentation Performance

Structure	Dice Score	Binary Cross-Entropy
Vasculature	0.808	0.0148
OSNs	0.760	0.0173
Glomeruli	0.779	0.0017

OSN segmentation achieved single-cell resolution in sparse regions but grouped cells in dense zones.

Registration Efficiency

1082 CPU hours on 96 Intel Xeon Gold 6136 cores
16% error reduction vs. sequential registration
Tolerance: ±80 µm axial drift corrected

Key Findings

OSN Count: ~2.7 million OSNs calculated via morphometric extrapolation (90% CI: 2.4–2.9M).
Fila Olfactoria: 34 foramina identified in cribriform plate (17/side).
Non-uniform Distribution: Olfactory epithelium showed serrated borders and posterior-anterior density gradient.

Implications

Technical Advancements

Scalable ML: Method enables processing of whole-brain datasets (~100x mouse brain volume).
Clinical Potential: Pipeline adaptable for Parkinson’s/Alzheimer’s studies via α-synuclein/tau staining.
Open Science: First public 3D olfactory dataset with ~5.8 GB/channel resolution.

Biological Insights

Challenges mouse-to-human extrapolation: ~10x fewer OSNs/glomerulus vs. mice.
Provides baseline for studying SARS-CoV-2 olfactory dysfunction.

Computational Tools Used

Segmentation: TensorFlow U-Net, Fiji for ground truth
Registration: SimpleElastix, ITK
Visualization: ParaView, Neuroglancer
HPC: New Zealand eScience Infrastructure (NeSI)

Interactive 3D exploration

Spatiotemporal patterns in the embryonic heart

Mon, 25 Sep 2023 00:00:00 +0000

Authors: Nazanin Ebrahimi, Mahyar Osanlouy, Chris Bradley, Fabiana Kubke, Dane Gerneke, Peter Hunter
Publication: iScience. (July 2022)
Dataset: Mendeley Data

Overview

This study presents an innovative hybrid experimental-computational pipeline to investigate the spatiotemporal growth patterns underlying cardiac C-looping in embryonic chick hearts. C-looping is a critical phase in heart development where the straight heart tube transforms into a C-shaped structure, setting the stage for proper chamber formation. Abnormalities during this process are linked to congenital heart defects. The work combines multi-scale imaging, deep learning-based cell segmentation, and biomechanical modeling to bridge cellular dynamics with tissue-level deformations, offering new insights into the mechanisms driving heart morphogenesis.

Key Contributions

First 3D multi-scale dataset of C-looping hearts, integrating cell-to-organism level data from confocal microscopy and micro-CT.
Fully automated 3D myocardial cell segmentation using a custom convolutional neural network (CNN), achieving a Dice coefficient of 0.91 ± 0.1.
Finite Element (FE) biomechanical models capturing spatiotemporal heart geometry changes across four developmental timepoints.
Variance-driven analysis revealing how inter-cellular space (ICS) and cellular heterogeneity contribute to tissue growth.
Open-source pipeline for integrating cellular features with tissue-level kinematics, enabling future studies on cardiac morphogenesis.

Methodology

Experimental Workflow

Multi-modal imaging:
- Confocal microscopy: Whole-mount staining (WGA, NCAM-1, DAPI) provided 3D cell-resolution images of myocardial cells.
- Micro-CT scanning: Sub-micron resolution imaging contextualized heart geometry within the entire embryo.
- Optical clearing: Enabled high-resolution imaging while preserving 3D architecture.
Temporal staging: Embryos were ordered developmentally using anatomical landmarks and dorsal detachment metrics.

Computational Workflow

3D Deep Learning for Cell Segmentation:
- A 3D U-Net architecture was trained to segment individual myocardial cells from confocal stacks.
- Achieved 91% Dice similarity compared to manual segmentation, reducing processing time from 8 hours to seconds per 100 cells.
- Addressed memory constraints via a traceable slicing-merging algorithm for large-scale image processing.
Finite Element Modeling:
- Anatomically accurate FE meshes were fitted to heart geometries using OpenCMISS.
- RMS fitting error < 3 μm enabled precise representation of tissue deformation.
Spatiotemporal Analysis:
- Cellular features (volume, anisotropy, orientation) were mapped onto FE meshes.
- General Linear Model (GLM) linked tissue growth to cellular dynamics (cell number, ICS volume, anisotropy).

Results

Differential Growth Patterns:
- Ventral regions showed ~2x higher tissue growth than dorsal regions, driven by increased cell proliferation and ICS expansion.
- Outer curvature cells exhibited circumferential alignment, while inner curvature cells remained isotropic.
Role of Inter-Cellular Space (ICS):
- ICS accounted for 30–45% of tissue volume changes, highlighting its role in accommodating growth.
- Ventral ICS volume increased by 62% during bending phases.
Variance as a Developmental Signal:
- Ring-shaped variance patterns in cell volume/anisotropy emerged around the outer curvature, suggesting mechanical feedback loops.
- High-variance regions correlated with zones of rapid tissue remodeling.
Tissue-Cell Dynamics:
- GLM analysis explained 70% of tissue growth variance, with cell number (β = 0.48, p < 0.001) and ICS (β = 0.32, p < 0.01) as key predictors.
- Cell orientation aligned with tissue deformation vectors during rotation (r > 0.8) but not bending phases.

Implications

Mechanistic Insights: Challenges the “differential growth hypothesis” by showing ICS and cellular heterogeneity are equally critical to looping.
Toolkit for Developmental Biology: The pipeline is extensible to other organs/species, enabling studies on how cellular noise shapes tissue patterning.
Clinical Relevance: Provides a framework to explore how genetic mutations disrupt growth coordination, informing congenital heart defect research.
AI/ML Impact: Demonstrates the power of deep learning in automating large-scale 3D biological image analysis.

SFEAL

Fri, 22 Sep 2023 00:00:00 +0000

Authors: Mahyar Osanlouy, Alys R. Clark, Haribalan Kumar, Clair King, Margaret L. Wilsher, David G. Milne, Ken Whyte, Eric A. Hoffman & Merryn H. Tawhai Publication: Nature Scientific Reports. (September 2020)
Code: Mendeley Data

Overview

This project presents a quantitative investigation into age‐related variations in lung and fissure shape among healthy adults (aged 20–90 years). By integrating advanced finite element mesh fitting techniques with statistical shape modeling and principal component analysis (PCA), the study establishes a robust framework for capturing and quantifying subtle geometric changes in lung morphology. Such a mathematical framework provides a normative baseline against which pathological alterations may be compared.

Key Contributions

Integration of Finite Element Modeling and PCA:
• Developed a high-order finite element (FE) mesh fitted to segmented lung surfaces, incorporating curvature and smoothness constraints.
• Applied PCA to reduce dimensionality, isolating principal modes of shape variation.
Mathematical Modeling of Shape Deformations:
• Formulated an energy minimization problem that couples data fidelity with smoothness constraints.
• Expressed lung shape variations as weighted perturbations from a mean model in a high-dimensional space.
Clinical Insights:
• Revealed strong associations between lung geometry, age, and BMI.
• Enabled detection of subtle fissure shifts and volumetric changes, likely reflecting underlying tissue elasticity.

Methodology

Finite Element Mesh Construction

Each lung is represented by a finite element mesh with a fixed topology (225 nodes in this study). To ensure an accurate and smooth representation, the mesh is fitted to imaging data by minimizing an energy functional that balances data fidelity against surface smoothness.

Energy Model Formulation

The energy function is defined as:

$$ T(u) = \sum_{i=1}^{N} \gamma_i \left| z(\xi_i) - z_d \right|^2 + \int_{\Omega} g(u(\xi)), d\xi, $$

where:

$u$ is the vector of shape (or nodal displacement) parameters.
$z(\xi_i)$ represents the geometric position of a node in local parametric coordinates $\xi_i$.
$z_d$ is the target data point from the segmented lung surface.
$\gamma_i$ is a weight factor controlling data matching for each node.
$g(u(\xi))$ is the smoothness (regularization) term that penalizes rapid or non-physical deformations.
$\Omega$ denotes the domain over which the mesh is defined.

This formulation ensures that the FE mesh adapts to the lung’s surface while preserving a biologically plausible smooth geometry.

Pseudo-code for Mesh Fitting

for each subject:
// Step 1: Initialize template mesh
Initialize template_mesh with 225 nodes
// Step 2: Define anatomical landmarks
Define landmarks:
- Apices
- Diaphragm
- Lung base edges
- Anterior segments
// Step 3: Optimize node positions using energy minimization
Optimize node positions by minimizing the energy function T(u):
T(u) = Σγ ||z(ξ₁, ξ₂) - z_d||² + ∫ g(u(ξ)) dξ
// where:
// z = nodal coordinates (x, y, z)
// g = smoothness constraint (curvature regularization)
// γ = data matching weight factor

Mean fitting error: 5.2 ± 2.3 mm

Data Alignment and Statistical Shape Modeling

Prior to PCA, lung shapes are aligned using General Procrustes Analysis (GPA) to eradicate variations due to translation, rotation, and, in the case of the size-exclusive model, scaling.

Alignment Equation

The alignment is expressed as:

$$ \bar{S} = \alpha R S + T, $$

where:

$S$ is the original shape vector,
$R$ is the rotation matrix,
$T$ is the translation vector,
$\alpha$ is the scaling factor (applied only in size-inclusive models).

Principal Component Analysis (PCA)

PCA is employed to reduce the high-dimensional shape vector into principal modes that capture the primary sources of variance.

PCA Decomposition via SVD

The shape matrix $\mathbf{S}$ is decomposed using singular value decomposition (SVD):

$$ \mathbf{S} = \mathbf{U} {\Sigma} \mathbf{V}^{T}, $$

where:

$\mathbf{U}$ contains the eigenvectors of $\mathbf{S} \mathbf{S}^{T}$,
${\Sigma}$ is the diagonal matrix of singular values $\sigma_i$ (with $\lambda_i = \sigma_i^2$ being the eigenvalues),
$\mathbf{V}^{T}$ holds the principal directions in the data space.

Each lung shape is reconstructed as:

$$ M_{l}(w) = \bar{S}_0 + w, u_{l}, $$

with:

$\bar{S}_0$ being the mean shape,
$u_l$ the $l$th principal component,
$w$ the corresponding weight factor.

The variance explained by each mode is:

$$ \text{Variance Explained} = \frac{\sigma_l^2}{\sum_{i} \sigma_i^2} \times 100%. $$

Results

Correlation with Age:
Specific principal shape modes display strong correlations with age (e.g., $R = -0.75, ; p < 0.001$),
indicating that lung geometries become more “pyramidal” as age increases.
Sex and BMI Associations:
Differences in shape related to lung size manifest in the size-inclusive model, highlighting sex-related volumetric differences. In contrast, the size-exclusive model emphasizes purely geometric deformations, with moderate correlations observed for BMI.

Implications

Clinical Applications:
The detailed mathematical framework enables lung shape to be employed as a biomarker for aging and disease. Deviations from the normative PCA model may assist in early detection or staging of conditions like COPD and IPF.
Technical Advancements:
The fusion of finite element modeling with PCA offers a versatile and robust method for analyzing complex organ shapes. This approach can be adapted to other anatomical structures requiring similar quantitative assessments.
Future Directions:
Enhancing the energy model and incorporating dynamic imaging (e.g., covering breathing cycles) could further refine shape analysis. Additionally, exploring non-linear dimensionality reduction techniques may uncover further subtleties in lung morphology.

The SPARC DRC

Wed, 20 Sep 2023 00:00:00 +0000

Authors: Mahyar Osanlouy, Anita Bandrowski, Bernard de Bono, David Brooks, Antonino M. Cassarà, Richard Christie, Nazanin Ebrahimi, Tom Gillespie, Jeffrey S. Grethe, Leonardo A. Guercio, Maci Heal, Mabelle Lin, Niels Kuster, Maryann E. Martone, Esra Neufeld, David P. Nickerson, Elias G. Soltani, Susan Tappan, Joost B. Wagenaar, Katie Zhuang, Peter J. Hunter
Publication: Front. Physiol. (June 2021)
Website: SPARC Science
Interactive 3D Models & Data: MAP Web App

Overview

The NIH Common Fund’s Stimulating Peripheral Activity to Relieve Conditions (SPARC) program is an ambitious initiative aimed at revolutionizing our understanding of the autonomic nervous system (ANS) and its role in organ function. As the lead developer for the Mapping Core (MAP-Core) of the SPARC Data and Resource Center (DRC), I collaborated with an interdisciplinary team to create a comprehensive framework for curating, visualizing, and analyzing ANS data across different species.

The SPARC DRC serves as the technological foundation for the SPARC program, providing researchers worldwide with access to standardized experimental data, computational models, and visualization tools through a unified web portal (https://sparc.science). Our work addresses a critical gap in biomedical research: the need for standardized, reproducible, and interoperable data and computational resources to advance our understanding of neural control of organ function and develop effective neuromodulation therapies.

Key Contributions

Our team developed several groundbreaking innovations that form the technological foundation of the SPARC program:

1. FAIR-Compliant Data Management Platform

We implemented comprehensive data standards and workflows to ensure all SPARC data adheres to FAIR principles (Findable, Accessible, Interoperable, and Reusable). This includes standardized metadata schemas, dataset organization structures, and persistent identifiers for long-term data discoverability.

2. Knowledge Management System

We created a sophisticated knowledge graph integrating anatomical relationships, neural connectivity, and semantic annotations from multiple species. This system enables powerful cross-species comparisons and semantic searches that were previously impossible with traditional database approaches.

3. Multi-Scale Anatomical Mapping Framework

One of our most significant innovations is the development of 3D material coordinate systems (“scaffolds”) for organs and bodies. These scaffolds provide a revolutionary solution to a complex problem: how to map and compare data from organs that undergo substantial deformation (e.g., beating hearts, inflating lungs) across different experimental conditions and species.

4. Computational Modeling Platform

We developed o²S²PARC (“open, online simulations for SPARC”), an online computational environment that enables researchers to analyze data, develop models, and simulate neuromodulation scenarios using cloud computing resources.

Methodology

Developing Standardized Data Workflows

We established comprehensive curation and data standards to ensure consistency across all SPARC datasets. Key aspects include:

Implementation of the SPARC Dataset Structure (SDS) for consistent file organization
Development of the Minimal Information Standard (MIS) for semantic metadata
Integration with community ontologies and specialized annotation tools
Creation of SODA (Software for Organizing Data Automatically) to assist researchers in preparing data submissions

Creating Neural Connectivity Maps

To represent the complex connectivity of the ANS across different species, we:

Utilized the ApiNATOMY toolkit to create topological and semantic models of neural pathways
Generated interactive flatmap diagrams that visualize neural connectivity in 2D for multiple species
Linked these maps to the SPARC Knowledge Graph for data integration

3D Scaffold Framework for Anatomical Mapping

A cornerstone of our innovation is the development of the 3D scaffold system. This methodology addresses the fundamental challenge of mapping data from organs that undergo significant deformation. Our approach includes:

1. Material Coordinate System Development

We developed Scaffold-Maker, specialized CAD software that creates 3D material coordinate systems for body organs. These scaffolds use finite element meshes with smooth interpolation to represent complex anatomical geometries.

2. Cross-Species Anatomical Representation

We designed the scaffolds to accommodate topological differences between species, allowing for meaningful cross-species comparisons despite anatomical variations. For example, we created heart scaffolds that accommodate different numbers of pulmonary veins in humans (4), pigs (2), and rats (3).

3. Deformation-Invariant Mapping

Our material coordinate system maintains consistent references to tissue locations regardless of organ deformation. This allows data to be mapped and compared across different physiological states (e.g., cardiac contraction phases).

4. Integration with Whole-Body Models

We developed methods to embed organ scaffolds within whole-body coordinate systems, facilitating multi-scale data integration from cellular to organism levels.

Results

Our work has produced several tangible outcomes that demonstrate the power of the SPARC DRC infrastructure:

Comprehensive ANS Data Resource

The SPARC Portal now serves as the world’s most comprehensive resource for standardized ANS data, with:

Curated datasets from multiple species and organ systems
Rich semantic annotations linking experimental data to anatomical locations
Standardized protocols and metadata ensuring reproducibility

Species-Specific Anatomical Scaffolds

We have successfully created detailed 3D scaffolds for key organs across multiple species:

Heart scaffolds for human, pig, and rat with species-specific topologies
Colon scaffolds capturing species variations in haustra and taeniae coli structures
Whole-body scaffolds with embedded organ systems
Bladder, stomach, and lung scaffolds for integrating diverse experimental data

Neural Mapping Demonstrations

We have demonstrated successful registration of neural data to our scaffold systems:

Mapping of intrinsic cardiac neurons (ICNs) from rat hearts to a standardized cardiac scaffold
Integration of data from multiple specimens into “integrative” scaffolds
Preservation of spatial relationships across deformed organ states

Computational Modeling Environment

The o²S²PARC platform now provides:

Online access to sophisticated computational models
Tools for simulating neuromodulation effects
Reproducible workflows for data analysis
Collaborative environments for model development

Implications

The SPARC DRC infrastructure we’ve developed has significant implications for both basic science and clinical applications:

Advancing Basic Science

Our work enables new approaches to understanding the ANS by:

Facilitating cross-species comparisons of neural circuitry
Providing standardized reference systems for integrating multi-scale data
Creating a foundation for quantitative, data-driven neural mapping
Enabling reproducible computational analyses

Clinical Applications

The infrastructure directly supports the development of bioelectronic medicine by:

Providing detailed anatomical maps for targeting neuromodulation
Enabling simulation of device effects on neural activity
Supporting the optimization of stimulation parameters
Facilitating translation between animal models and human applications

Beyond SPARC

The methodologies and tools we’ve developed have applications beyond the ANS:

The scaffold concept can be applied to any deformable biological system
Our FAIR data management approaches set standards for other large-scale initiatives
The o²S²PARC computational platform can support diverse modeling applications

Conclusion

The SPARC DRC represents a paradigm shift in how we approach the study of the autonomic nervous system. By creating standardized frameworks for data organization, anatomical mapping, and computational modeling, we’ve laid the groundwork for a new era of quantitative, integrative neuroscience research that can directly inform the development of bioelectronic therapies. Our 3D scaffold methodology, in particular, solves the fundamental challenge of mapping biological data in deformable systems, enabling unprecedented integration of experimental data across scales, species, and physiological states.

AI-Driven Radiomics for Precision Prostate Cancer Therapy

Sun, 20 Aug 2023 00:00:00 +0000

AI-Driven Radiomics for Precision Prostate Cancer Therapy

Authors: Tsz Him Chan, Annette Haworth, Alan Wang, Mahyar Osanlouy et al.
Publication: EJNMMI Research (2023)
Code: Available on Request

Overview

This pioneering study developed an AI-powered radiomics pipeline to optimize biologically targeted radiation therapy (BiRT) for prostate cancer. By integrating PSMA PET/CT with multiparametric MRI (mpMRI), we created voxel-level predictions of tumor location and grade through advanced image registration and machine learning techniques.

Key Innovations

Multi-modal fusion: First study combining PSMA PET radiomics with DCE MRI perfusion parameters
Voxel-wise prediction: Achieved 0.89 AUC for tumor detection using 3D radiomic features
Grade differentiation: Developed two-stage RFC model separating high/low-grade disease (Accuracy: 0.67-0.99)
HPC-optimized registration: Reduced spatial uncertainty to 3.3mm using deformable histology alignment

Computational Pipeline

Data Integration: Co-registered PSMA PET/CT (5 scanners), mpMRI (2x Siemens 3T), and whole-mount histology
Key Steps:

align_pet_ct() → rigid_registration(mpMRI) →
deformable_registration(ex_vivo_MRI) →
histology_annotation_propagation()

Challenges Solved:
Bladder filling artifacts in PET
Partial volume effects in 3.27mm PET slices
Non-linear prostate deformation post-resection

2. Radiomic Feature Engineering

Modality	Features Extracted	Key Parameters
PSMA PET	3D LoG, LBP, GLCM textures	SUVmax, metabolic tumor volume
DCE MRI	Ktrans, iAUGC60, TTP perfusion maps	Pharmacokinetic modeling
ADC Maps	NGTDM coarseness, percentile values	b=1200 s/mm² diffusion restriction

Feature Selection:

ANOVA filtering → Gini impurity ranking
Final feature set: 50 most discriminative parameters

Machine Learning Architecture

Tumor Detection Model

Two-stage RFC Framework:

Location Detection: 842 sensitivity/804 specificity
Grade Classification: Low vs High Grade (ISUP ≥3)

Performance Comparison:

Model	AUC	Sensitivity	Specificity
PET Alone	0.865	0.781	0.799
mpMRI Alone	0.882	0.802	0.801
Combined	0.890	0.842	0.804

Receiver operating characteristics for different models

Key Radiomic Predictors

PET: 3D LoG(σ=3mm) minimum
ADC: NGTDM Coarseness
DCE MRI: Ktrans 90th percentile

Clinical Implications

Personalized Radiotherapy: Enables voxel-level dose painting based on metabolic/perfusion features
Early Recurrence Prediction: High-grade lesions showed 2.8× higher PSMA uptake (p<0.01)
Technical Impact:
Solved partial volume effects in PET-guided planning
Demonstrated perfusion > diffusion for grade prediction
Open-source registration framework [3D Slicer Plugin]

Future Directions

Multi-institutional validation across PET/MRI scanners
DL Enhancement: Replace handcrafted features with 3D CNNs
Real-time Adaptation: Integrate with MR-Linac systems

This project received funding from Prostate Cancer Foundation of Australia and Health Research Council of New Zealand.

COVID-19 Modelling Effort

Tue, 21 Jul 2020 00:00:00 +0000

Authors: Mahyar Osanlouy, Gonzalo D. Maso Talou, David Brooks, Soroush Safaei, Alan Garny, and Peter J. Hunter

Introduction

Amid COVID-19 pandemic, a group of computational scientists and engineers in the ABI decide to set up models of COVID- 19 transmission and infection in New Zealand. The taskforce developed technologies to model/analyze epidemic outbreaks, and a pipeline from data to model to visualization that offers a sandbox environment to test policies and track the infection process with an unprecedented level of details.

Modelling infection in buildings, cities and countries: A multiscale integrative model

Modelling the spread of COVID-19 is of paramount importance to strategize effective policies to reduce the impact of the disease in our health and economic system. By means of this model, we can identify social interactions, places and activities that favour the spread of the virus in order to propose appropriate countermeasures. The model is also a sandbox environment where we can test policies and observe the consequences in the complex environment of a city, to understand the pros and cons of each action.

The model is composed of three scales: i) place/building scale; ii) transport/city scale; iii) cluster/country scale. Each scale is implemented with a different type of model (see Figure 1), and data between these models is communicated by a homogenization process. In this process, data of the smaller scale computes the parameters of the larger scale such that it represents the same phenomenology, e.g., if the number of infections grew by 10 times during 10 hours in a transport scale, then rate of infection for the country scale can be computed as one infection per hour assuming a base 10 exponential growth model.

Finally, the model evolution and statistical analysis of the simulation is visualized on top of a dynamic web interface that allows the users to easily navigate through the results in time and space using cartographic and 3D views powered by Unity. In the current development, simulations can be executed real-time, online (visualizing results as the simulation is taking place) or offline. The offline modality allows the user to navigate forward and backwards on time, turning the model into an ideal tool for contact tracing and understanding the causality of a particular scenario.

Person-to-person transmission: Place model

The place model is implemented by means of agent-based models (ABMs) where the actions and status of each person can be modelled, predicted and, if needed, controlled. These models offer high-resolution inferences of infection transmissions at the level of individuals giving us a clear understanding of the impact of different infection mechanisms (sneeze, direct or via surface transmission, social distancing efficiency, among others).

ABMs capture the interactions of individuals in time and space using real data and detailed transmission pathways, providing insight into settings where transmission is occurring and how clusters emerge. Various hypotheses can be tested by modelling and evaluating social behaviour within different scenarios giving insights about which elements are key to draw effective policies. In modelling disease transmission, these scenarios may include different pandemic intervention strategies deployed by government and/or policymakers such as containment, mitigation, and suppression. Every individual (agent) also has specific attributes which may contribute to the behaviour of the model and, as a result, to the result of the disease transmission. Therefore, ABMs simulate the dynamics of a system where individual-level characteristics, environmental factors, and the interaction of the two are of importance.

As an example, we have developed an ABM to simulate the transmission of COVID- 19 across shoppers in a supermarket (see Figure 2). Here, shoppers are the agents of the model which have particular attributes including basic demographics (as accurate as possible), health states and characteristics (e.g. susceptible, immunity to the disease, etc.), probability of practising social distancing and basic hygiene, etc. These are what we call the specifics of the local transmission dynamics. Additionally, a natural disease history model for COVID-19, i.e., the infectivity profile from the onset of infection, over an incubation period, to the peak of infectivity, to recovery, for a single symptomatic or asymptomatic individual becomes extremely important in longitudinal simulations. The supermarket is the spatial environment where the simulation is confined. It is precisely this environment where the rules and policies are defined and the mutual interaction with the shoppers occurs. The environment and its rules have direct and indirect implications on the shoppers and, ultimately, on the probability of the disease transmission. The same is also true for the effects of the shoppers and their behaviour on the environment.

Circulation and infection through the city arteries: Transport model

Cities are a complex aggregate of place models, where people navigate between them at specific times of day using specific means of transportation, among which infection is also occurring. The proposed transportation model communicates numerous place models and computes infections while agents commute between places.

This model deals with three complexities: (i) How to transport an agent from place A to B for a given type of transportation; (ii) Which models of infection are appropriate for each type of transportation mechanism (bus, private service,personal transport, walking, cycling, running, aeroplane and ferry); (iii) How an agent decides the next action to perform after visiting a place.

To properly address (i), we need to consider the available path when the agent is using a specific transport type. The route for commuting to work will be quite different for bus, cycling and walking, as some spaces can be traversed for one mode but not for the remaining ones. Then, specific transportation graphs are derived for each type of transportation. Once the route is known, the agent will have to compute a trip plan, which involves a set of intermediate action to reach the final destination. For example, if the agent decides to take the bus to go to the supermarket, he will perform the following actions: 1) walk to the bus stop, 2) wait until the bus arrives,3) commute from the bus stop of departure to the bus stop of arrival, 4) walk from the arrival bus stop to the supermarket. To ensure that the transport model delivers precise estimates of commuting time, it considers the time of day, day of the week with the timetables when an agent is using public transport. Also, the agents can define certain constraints as the maximum walking distance, waiting time and number of bus connexions.

Regarding (ii), commuting by bus or walking through the street may clearly present two different scenarios of exposure where a given set of policies will have a different impact. Models for each transportation type can be modelled by specific place models (as seen in the previous section), or by partial differential equations (PDEs) considering the number of susceptible, exposed and infected subjects and the total area of circulation. Each approach presents its advantages depending on the policies to be tested, tracing detail required and available resources for the simulation.

Lastly, (iii) is addressed by modelling the agent’s behaviour with a stochastic planner. Each time an agent is idle the planner is executed, evaluating theprobability for this agent to take different actions (such as go to work, go to sleep, return home, visit a friend, go shopping) based on his personal attributes (age, occupation, family), history of previous actions and time and day of the week. After all the probabilities have been computed, the agent performs the most probable action. Once the action is finished the agent is idle and the planning starts again.

Note that all the probabilities distributions needed to characterize the agent’s behaviour and the number of agents across the country can be extracted from national census data.

Modelling nation-wide infection: Cluster models

The cluster model is based on the standard SEIR (susceptible-exposed-infected-removed) approach, gathering all transport model data for each city or region and reducing it to a set of ordinary differential equations. The rates and coefficients of this model can be computed by fitting each cluster model with the observations of the corresponding transport model and, at the same time, the cluster model informs the transport among cities serving as a boundary condition of the transport model.

This macro-scale serves as a dashboard of the infection process in large communities. Its simplicity renders tractable and easy to analyze variables, and their variation over time - e.g. infection, mortality and recovery rates - which is extremely useful for specialists and policymakers.

Selected Projects | Mahyar's world 🌏

Temporal Predictive Coding

Overview

Key Contributions

Methodology

Core Principles

Comparison to Kalman Filtering

Results

1. Robust Tracking of Dynamic States

2. Learning Synaptic Weights

3. Biological Receptive Fields

4. Nonlinear Dynamics

Implications

3D Reconstruction of Human Olfactory Projection

Overview

Key Computational Contributions

Methodology

Computational Pipeline

Results

Segmentation Performance

Registration Efficiency

Key Findings

Implications

Technical Advancements

Biological Insights

Computational Tools Used

Spatiotemporal patterns in the embryonic heart

Overview

Key Contributions

Methodology

Experimental Workflow

Computational Workflow

Results

Implications

SFEAL

Overview

Key Contributions

Methodology

Finite Element Mesh Construction

Energy Model Formulation

Pseudo-code for Mesh Fitting

Data Alignment and Statistical Shape Modeling

Alignment Equation

Principal Component Analysis (PCA)

PCA Decomposition via SVD

Results

Implications

The SPARC DRC

Overview

Key Contributions

1. FAIR-Compliant Data Management Platform

2. Knowledge Management System

3. Multi-Scale Anatomical Mapping Framework

4. Computational Modeling Platform

Methodology

Developing Standardized Data Workflows

Creating Neural Connectivity Maps

3D Scaffold Framework for Anatomical Mapping

1. Material Coordinate System Development

2. Cross-Species Anatomical Representation

3. Deformation-Invariant Mapping

4. Integration with Whole-Body Models

Results

Comprehensive ANS Data Resource

Species-Specific Anatomical Scaffolds

Neural Mapping Demonstrations

Computational Modeling Environment

Implications

Advancing Basic Science

Clinical Applications

Beyond SPARC

Conclusion

AI-Driven Radiomics for Precision Prostate Cancer Therapy

AI-Driven Radiomics for Precision Prostate Cancer Therapy

Overview

Key Innovations

Computational Pipeline

1. Multi-modal Image Registration

2. Radiomic Feature Engineering

Machine Learning Architecture