Computational Modeling | Mahyar's world 🌏

Temporal Predictive Coding

Sun, 15 Dec 2024 00:00:00 +0000

Authors: Beren Millidge, Mufeng Tang, Mahyar Osanlouy, Nicol S. Harper, Rafal Bogacz
Publication: PLOS Computational Biology (April 2024)
Code: GitHub Repository
Read the paper: PLOS Computational Biology

Overview

How does the brain process dynamic sensory inputs; like tracking a moving object or predicting the next note in a melody? In this work, we propose temporal predictive coding (tPC), a biologically plausible neural framework that extends classical predictive coding to handle time-varying stimuli. By unifying Bayesian inference, neural dynamics, and synaptic plasticity, tPC explains how cortical circuits might learn to predict and adapt to continuously changing environments.

Key Contributions

A unified model for temporal prediction: tPC bridges predictive coding (for static inputs) and Kalman filtering (for dynamic systems), enabling online inference and learning without complex matrix operations.
Biologically plausible implementation: The model uses only local computations and Hebbian plasticity, aligning with known cortical microcircuit architecture.
Motion-sensitive receptive fields: When trained on natural movies, tPC neurons develop Gabor-like, direction-selective receptive fields—matching properties of early visual cortex neurons.
Nonlinear generalization: The framework extends to nonlinear systems (e.g., pendulum dynamics), outperforming linear models in complex tasks.

Methodology

Core Principles

Generative model: Hidden states evolve via a Markov process, with observations generated from hidden states.

$$ x_k = A f(x_{k-1}) + B u_k + \omega_x $$

$$ y_k = C f(x_k) + \omega_y $$

Free energy minimization: Neurons minimize prediction errors at two levels:
1. Sensory errors: Mismatch between predicted and actual observations.
2. Temporal errors: Mismatch between predicted and inferred hidden states.
Neural implementation:
- Explicit prediction error neurons or dendritic computations.
- Synaptic updates follow local Hebbian rules (e.g., ΔA ∝ prediction error × presynaptic activity).

Comparison to Kalman Filtering

tPC approximates Kalman filtering but avoids tracking posterior uncertainty.
Achieves similar accuracy in linear tracking tasks with simpler computations.

$$ \hat{x}_k^- = A \hat{x}_{k-1} + B u_k $$

$$ \hat{x}_k = \hat{x}_k^- + K(y_k - C\hat{x}_k^-) $$

$$ K = \Sigma_x C^T \left[C \Sigma_x C^T + \Sigma_y \right]^{-1} $$

Results

1. Robust Tracking of Dynamic States

tPC accurately infers position, velocity, and acceleration from noisy observations, even with scrambled sensory inputs.
Matches Kalman filter performance with 5 inference steps (MSE difference < 1%).

2. Learning Synaptic Weights

Hebbian plasticity enables online learning of dynamics (A, B) and observation (C) matrices.
Learns to predict observations even with random initial weights.

3. Biological Receptive Fields

Trained on natural movies, tPC develops:
- Spatiotemporal receptive fields with motion direction selectivity.
- Gabor-like filters resembling V1 simple cells. Representations developed by the model when trained with patches from movies of dynamic natural scenes. A: First 10 frames of 2 example training movies used in our experiments. Patches extracted from movies obtained at websites pexels.com, pixabay.com and commons.wikimedia.org (for wikimedia attributions see https://github.com/C16Mftang/temporal-predictive-coding). B: The projective fields C developed Gabor-like filters after training. C: Space-time receptive fields developed by hidden neurons of the tPC model. https://doi.org/10.1371/journal.pcbi.1011183.g006

4. Nonlinear Dynamics

Outperforms linear models in predicting pendulum motion (20% lower MSE).
Captures nonlinearities where linear approximations fail (e.g., large-angle oscillations). Simulations of the pendulum. A: A free-body diagram of a simple pendulum that has a mass m attached to a string with length $L$. Also shown are the forces applied to the mass. The restoring force $-mg sin \theta$ is a net force toward the equilibrium position. B: A phase portrait of the pendulum simulation showing the result of our linear versus nonlinear models prediction for the ground-truth data. The vector field (i.e. set of small arrows) was created by computing the derivatives of $\frac{d\theta_1}{dt}$ and $\frac{d\theta_2}{dt}$ at $t = 0$ a grid of 30 points over the range of $-\pi$ to $\pi$ and -4 to +4 for $\theta_1$ and $\theta_2$ respectively. C: The barplot shows the difference between the mean prediction errors of the linear model versus the nonlinear model from 100 simulations with varying noise profiles. The mean errors are significantly different $(p \ll 0.001)$ https://doi.org/10.1371/journal.pcbi.1011183.g007

Implications

Theoretical: Provides a unified account of static and dynamic sensory processing under predictive coding.
Experimental: Predicts neural responses to time-varying stimuli and receptive field properties.
Translational: Framework for neuromorphic hardware and efficient machine learning models.

Temporal Predictive Coding: A New Framework for Neural Processing of Dynamic Stimuli

Fri, 15 Nov 2024 00:00:00 +0000

Introduction

One of the most fascinating aspects of the brain is its ability to process and predict dynamic sensory inputs that continuously change over time. From tracking a moving object to predicting the next note in a melody, our brains are remarkably adept at temporal prediction. In a recent paper published in PLOS Computational Biology titled “Predictive coding networks for temporal prediction,” my colleagues and I proposed a new computational framework that may help explain how the brain accomplishes this feat.

Predictive coding has emerged as an influential theoretical model for understanding cortical function. The core idea is deceptively simple: the brain constantly generates predictions of incoming sensory inputs and compares these predictions with actual sensory data. Any mismatch results in prediction errors that drive learning and perceptual processing. This framework has successfully explained many neural phenomena and receptive field properties in visual cortex.

However, most previous predictive coding models have focused on static inputs, neglecting the temporal dimension that is crucial for real-world perception. Our work addresses this gap by extending predictive coding to the temporal domain while maintaining its elegant biological implementation.

The Temporal Predictive Coding Model

Generative Model and Free Energy

At the foundation of our temporal predictive coding (tPC) model is a Hidden Markov Model (HMM) structure, which assumes that observations are generated by hidden states that evolve according to a Markov process. Mathematically, we can express this generative model as:

$$ x_k = A f(x_{k-1}) + B u_k + \omega_x $$

$$ y_k = C f(x_k) + \omega_y $$

Where:

$x_{k}$ is the hidden state at time $k$.
$y_{k}$ is the observed sensory input at time $k$.
$u_{k}$ is the control input at time $k$.
$A$ is the dynamics matrix governing state transitions.
$B$ is the control matrix.
$C$ is the observation matrix.
$f$ is a potentially nonlinear function.
$\omega_x$ and $\omega_y$ are Gaussian process and observation noise.

The goal is to infer the current hidden state $x_{k}$ given the current observation $y_{k}$ and previous state estimate $y_{1:k-1}$. To achieve this, we formulate a variational free energy objective:

$$ \mathcal{F}_k = \frac{1}{2}(y_k - C f(x_k))^T \Sigma_y^{-1} (y_k - C f(x_k)) + \frac{1}{2}(x_k - A f(\hat{x}_{k-1}) - B u_k)^T \Sigma_x^{-1} (x_k - A f(\hat{x}_{k-1}) - B u_k) $$

This free energy can be understood as the sum of two weighted prediction errors:

Sensory prediction errors: The difference between observed and predicted sensory inputs $y_k - Cf(x_k)$.
Temporal prediction errors: The difference between the current state and the prediction from the previous state $x_k - Af(\hat{x}_{k-1}) - Bu_k$.

Each prediction error is weighted by the precision (inverse variance) of the corresponding noise distribution, ensuring that more reliable predictions carry more weight.

Neural Implementation

A crucial contribution of our work is showing how temporal predictive coding can be implemented in neural circuits using biologically plausible mechanisms. The neural dynamics for inferring the hidden state follow gradient descent on the free energy:

$$ \tau \frac{d x_k}{d t} = -\epsilon_x + f'(x_k) \odot C^T \epsilon_y $$

Where $\epsilon_x$ and $\epsilon_y$ are precision-weighted prediction errors:

$$ \epsilon_y = \Sigma_y^{-1} \left( y_k - C f(x_k) \right) $$

$$ \epsilon_x = \Sigma_x^{-1} \left( x_k - A f(\hat{x}_{k-1}) - B u_k \right) $$

We proposed multiple neural circuit implementations of this model:

Network with explicit prediction error neurons: Where dedicated neurons represent prediction errors at each level of processing
Dendritic computing implementation: Where prediction errors are computed as differences between somatic and dendritic potentials
Single-iteration implementation: A simplified version that performs single updates per time step

Neural circuit implementation

Importantly, all of these implementations rely on local information and Hebbian plasticity. The synaptic weights are updated according to:

$$ \Delta A = \eta \epsilon_x f(\hat{x}_{k-1})^T $$

$$ \Delta B = \eta \epsilon_x u_k^T $$

$$ \Delta C = \eta \epsilon_y f(x_k)^T $$

These update rules are Hebbian in nature because they depend only on the activities of pre- and post-synaptic neurons, making them biologically plausible.

Relationship to Kalman Filtering

An intriguing property of our model is its relationship to the Kalman filter, which is the optimal solution for linear Gaussian filtering problems. We demonstrated that both Kalman filtering and temporal predictive coding can be derived as special cases of Bayesian filtering, with the key difference being how they handle uncertainty.

The Kalman filter propagates uncertainty estimates through time, tracking the full posterior covariance at each step. In contrast, tPC approximates this by assuming a point estimate (Dirac distribution) for the previous state. Despite this simplification, our tPC model achieves comparable performance to the Kalman filter in tracking tasks while being computationally simpler and more biologically plausible.

For linear systems, the tPC dynamics at equilibrium yield:

$$ \hat{x}_k^- = A\hat{x}_{k-1} + Bu_k $$

$$ \hat{x}_k = \hat{x}_k^- + K(y_k - C\hat{x}_k^-) $$

$$ K = \Sigma_x C^T \left[C \Sigma_x C^T + \Sigma_y \right]^{-1} $$

This resembles the Kalman filter update equations but with a fixed gain matrix $K$ rather than a dynamically updated one based on posterior uncertainty.

Experimental Results

Performance in Linear Filtering Tasks

We tested our model on classic tracking problems, where the goal is to infer the hidden state (position, velocity, acceleration) of an object undergoing unknown acceleration based on noisy observations. Even with just a few inference steps between observations, tPC achieved performance approaching that of the optimal Kalman filter.

A key advantage of our model is its ability to learn the parameters of the generative model (matrices $A$, $B$, and $C$) using Hebbian plasticity. Even when starting with random matrices, tPC could learn to accurately predict observations. Interestingly, the model also implicitly encoded noise covariance information in its recurrent connections, without needing explicit representation of precision matrices.

Motion-Sensitive Receptive Fields

Perhaps most excitingly, when trained on natural movies, our tPC model developed spatiotemporal receptive fields resembling those observed in the visual cortex. These fields exhibited Gabor-like patterns and direction selectivity, a hallmark of motion-sensitive neurons in early visual areas.

Nonlinear Extensions

We extended the model to handle nonlinear dynamics by incorporating nonlinear activation functions. When tested on a simulated pendulum task, the nonlinear tPC significantly outperformed the linear model, accurately predicting the pendulum’s motion even at extreme angles where nonlinear effects are strongest.

Implications and Future Directions

Our temporal predictive coding framework has several important implications:

It provides a biologically plausible explanation for how the brain processes dynamic stimuli and performs temporal predictions.
It demonstrates that complex temporal filtering operations can be implemented in neural circuits using simple, local computations.
It offers a unified framework that connects normative theories of perception (Bayesian inference) with mechanistic models of neural circuits.
It suggests that the same computational principles might underlie both static and dynamic sensory processing in the brain.

Conclusion

The temporal predictive coding model we’ve developed bridges an important gap in our understanding of how the brain processes dynamic sensory inputs. By extending predictive coding to the temporal domain while maintaining its biological plausibility, our model provides a compelling computational mechanism for temporal prediction in neural circuits.

The fact that our model develops receptive fields resembling those in the visual cortex and approximates optimal filtering solutions suggests that temporal predictive coding may indeed capture fundamental principles of neural computation in the brain. As we continue to refine these models and test them against empirical data, we hope to gain deeper insights into the remarkable predictive capabilities of the brain.

This blog is based on the paper “Predictive coding networks for temporal prediction” by Beren Millidge, Mufeng Tang, Mahyar Osanlouy, Nicol S. Harper, and Rafal Bogacz, published in PLOS Computational Biology, April 2024. Link to the paper

3D Reconstruction of Human Olfactory Projection

Fri, 10 Nov 2023 00:00:00 +0000

Authors: Victoria F Low, Chinchien Lin, Shan Su, Mahyar Osanlouy, Mona Khan, Soroush Safaei, Gonzalo Maso Talou, Maurice A Curtis, Peter Mombaerts
Publication: Communications Biology (November 2024)
Code: Zenodo Repository
Interactive 3D Model: Neuroglancer Platform

Overview

This study presents a computational pipeline to reconstruct the 3D microanatomy of the human olfactory system, from the nasal cavity to the olfactory bulb—using fluorescence histology, deep learning, and high-performance computing (HPC). The workflow addresses challenges in processing terabyte-scale datasets and provides new insights into olfactory sensory neuron (OSN) distribution and axon trajectories.

Key Computational Contributions

First end-to-end pipeline for large-scale 3D reconstruction of human olfactory tissues (~7.45 cm³ specimen, 1234 sections).
CNN-based segmentation achieving Dice scores >0.85 for critical structures (OSNs, glomeruli, vasculature).
HPC-optimized registration reducing banana-effect artifacts via multi-resolution deformable alignment.
Public 3D dataset enabling interactive exploration of olfactory projection via Neuroglancer.

Methodology

Computational Pipeline

From fluorescence histology to 3D visualization – click to expand

Fluorescence Histology
- Quadruple staining: Hoechst (nuclei), UEA1 (OSNs), OMP (mature OSNs), VGLUT2 (axon terminals).
- Whole-slide scanning: 1.097 µm/pixel resolution, ~2.9 TB raw data.
CNN Segmentation
- Architecture: Modified 2D U-Net with 4 input channels (Hoechst + 3 markers).
- Training: Bootstrap approach with iterative ground truth expansion (20-45 sections/structure).
- Key layers:
```
# Simplified U-Net structure
encoder = [Conv2D(8→16→32), MaxPooling2D]
bottleneck = Conv2D(64) + Upsampling2D
decoder = [Conv2D(32→16→8), Concatenate(skip connections)]
```
HPC Registration
- Intra-block: Parallel registration of 247 blocks (5 sections each) using SimpleElastix.
- Inter-block: Banana-effect correction via affine + B-spline transformations between blocks.
- Metrics: Mutual information for intensity alignment, DSC for structural consistency.

Results

Segmentation Performance

Structure	Dice Score	Binary Cross-Entropy
Vasculature	0.808	0.0148
OSNs	0.760	0.0173
Glomeruli	0.779	0.0017

OSN segmentation achieved single-cell resolution in sparse regions but grouped cells in dense zones.

Registration Efficiency

1082 CPU hours on 96 Intel Xeon Gold 6136 cores
16% error reduction vs. sequential registration
Tolerance: ±80 µm axial drift corrected

Key Findings

OSN Count: ~2.7 million OSNs calculated via morphometric extrapolation (90% CI: 2.4–2.9M).
Fila Olfactoria: 34 foramina identified in cribriform plate (17/side).
Non-uniform Distribution: Olfactory epithelium showed serrated borders and posterior-anterior density gradient.

Implications

Technical Advancements

Scalable ML: Method enables processing of whole-brain datasets (~100x mouse brain volume).
Clinical Potential: Pipeline adaptable for Parkinson’s/Alzheimer’s studies via α-synuclein/tau staining.
Open Science: First public 3D olfactory dataset with ~5.8 GB/channel resolution.

Biological Insights

Challenges mouse-to-human extrapolation: ~10x fewer OSNs/glomerulus vs. mice.
Provides baseline for studying SARS-CoV-2 olfactory dysfunction.

Computational Tools Used

Segmentation: TensorFlow U-Net, Fiji for ground truth
Registration: SimpleElastix, ITK
Visualization: ParaView, Neuroglancer
HPC: New Zealand eScience Infrastructure (NeSI)

Interactive 3D exploration

Spatiotemporal patterns in the embryonic heart

Mon, 25 Sep 2023 00:00:00 +0000

Authors: Nazanin Ebrahimi, Mahyar Osanlouy, Chris Bradley, Fabiana Kubke, Dane Gerneke, Peter Hunter
Publication: iScience. (July 2022)
Dataset: Mendeley Data

Overview

This study presents an innovative hybrid experimental-computational pipeline to investigate the spatiotemporal growth patterns underlying cardiac C-looping in embryonic chick hearts. C-looping is a critical phase in heart development where the straight heart tube transforms into a C-shaped structure, setting the stage for proper chamber formation. Abnormalities during this process are linked to congenital heart defects. The work combines multi-scale imaging, deep learning-based cell segmentation, and biomechanical modeling to bridge cellular dynamics with tissue-level deformations, offering new insights into the mechanisms driving heart morphogenesis.

Key Contributions

First 3D multi-scale dataset of C-looping hearts, integrating cell-to-organism level data from confocal microscopy and micro-CT.
Fully automated 3D myocardial cell segmentation using a custom convolutional neural network (CNN), achieving a Dice coefficient of 0.91 ± 0.1.
Finite Element (FE) biomechanical models capturing spatiotemporal heart geometry changes across four developmental timepoints.
Variance-driven analysis revealing how inter-cellular space (ICS) and cellular heterogeneity contribute to tissue growth.
Open-source pipeline for integrating cellular features with tissue-level kinematics, enabling future studies on cardiac morphogenesis.

Methodology

Experimental Workflow

Multi-modal imaging:
- Confocal microscopy: Whole-mount staining (WGA, NCAM-1, DAPI) provided 3D cell-resolution images of myocardial cells.
- Micro-CT scanning: Sub-micron resolution imaging contextualized heart geometry within the entire embryo.
- Optical clearing: Enabled high-resolution imaging while preserving 3D architecture.
Temporal staging: Embryos were ordered developmentally using anatomical landmarks and dorsal detachment metrics.

Computational Workflow

3D Deep Learning for Cell Segmentation:
- A 3D U-Net architecture was trained to segment individual myocardial cells from confocal stacks.
- Achieved 91% Dice similarity compared to manual segmentation, reducing processing time from 8 hours to seconds per 100 cells.
- Addressed memory constraints via a traceable slicing-merging algorithm for large-scale image processing.
Finite Element Modeling:
- Anatomically accurate FE meshes were fitted to heart geometries using OpenCMISS.
- RMS fitting error < 3 μm enabled precise representation of tissue deformation.
Spatiotemporal Analysis:
- Cellular features (volume, anisotropy, orientation) were mapped onto FE meshes.
- General Linear Model (GLM) linked tissue growth to cellular dynamics (cell number, ICS volume, anisotropy).

Results

Differential Growth Patterns:
- Ventral regions showed ~2x higher tissue growth than dorsal regions, driven by increased cell proliferation and ICS expansion.
- Outer curvature cells exhibited circumferential alignment, while inner curvature cells remained isotropic.
Role of Inter-Cellular Space (ICS):
- ICS accounted for 30–45% of tissue volume changes, highlighting its role in accommodating growth.
- Ventral ICS volume increased by 62% during bending phases.
Variance as a Developmental Signal:
- Ring-shaped variance patterns in cell volume/anisotropy emerged around the outer curvature, suggesting mechanical feedback loops.
- High-variance regions correlated with zones of rapid tissue remodeling.
Tissue-Cell Dynamics:
- GLM analysis explained 70% of tissue growth variance, with cell number (β = 0.48, p < 0.001) and ICS (β = 0.32, p < 0.01) as key predictors.
- Cell orientation aligned with tissue deformation vectors during rotation (r > 0.8) but not bending phases.

Implications

Mechanistic Insights: Challenges the “differential growth hypothesis” by showing ICS and cellular heterogeneity are equally critical to looping.
Toolkit for Developmental Biology: The pipeline is extensible to other organs/species, enabling studies on how cellular noise shapes tissue patterning.
Clinical Relevance: Provides a framework to explore how genetic mutations disrupt growth coordination, informing congenital heart defect research.
AI/ML Impact: Demonstrates the power of deep learning in automating large-scale 3D biological image analysis.

SFEAL

Fri, 22 Sep 2023 00:00:00 +0000

Authors: Mahyar Osanlouy, Alys R. Clark, Haribalan Kumar, Clair King, Margaret L. Wilsher, David G. Milne, Ken Whyte, Eric A. Hoffman & Merryn H. Tawhai Publication: Nature Scientific Reports. (September 2020)
Code: Mendeley Data

Overview

This project presents a quantitative investigation into age‐related variations in lung and fissure shape among healthy adults (aged 20–90 years). By integrating advanced finite element mesh fitting techniques with statistical shape modeling and principal component analysis (PCA), the study establishes a robust framework for capturing and quantifying subtle geometric changes in lung morphology. Such a mathematical framework provides a normative baseline against which pathological alterations may be compared.

Key Contributions

Integration of Finite Element Modeling and PCA:
• Developed a high-order finite element (FE) mesh fitted to segmented lung surfaces, incorporating curvature and smoothness constraints.
• Applied PCA to reduce dimensionality, isolating principal modes of shape variation.
Mathematical Modeling of Shape Deformations:
• Formulated an energy minimization problem that couples data fidelity with smoothness constraints.
• Expressed lung shape variations as weighted perturbations from a mean model in a high-dimensional space.
Clinical Insights:
• Revealed strong associations between lung geometry, age, and BMI.
• Enabled detection of subtle fissure shifts and volumetric changes, likely reflecting underlying tissue elasticity.

Methodology

Finite Element Mesh Construction

Each lung is represented by a finite element mesh with a fixed topology (225 nodes in this study). To ensure an accurate and smooth representation, the mesh is fitted to imaging data by minimizing an energy functional that balances data fidelity against surface smoothness.

Energy Model Formulation

The energy function is defined as:

$$ T(u) = \sum_{i=1}^{N} \gamma_i \left| z(\xi_i) - z_d \right|^2 + \int_{\Omega} g(u(\xi)), d\xi, $$

where:

$u$ is the vector of shape (or nodal displacement) parameters.
$z(\xi_i)$ represents the geometric position of a node in local parametric coordinates $\xi_i$.
$z_d$ is the target data point from the segmented lung surface.
$\gamma_i$ is a weight factor controlling data matching for each node.
$g(u(\xi))$ is the smoothness (regularization) term that penalizes rapid or non-physical deformations.
$\Omega$ denotes the domain over which the mesh is defined.

This formulation ensures that the FE mesh adapts to the lung’s surface while preserving a biologically plausible smooth geometry.

Pseudo-code for Mesh Fitting

for each subject:
// Step 1: Initialize template mesh
Initialize template_mesh with 225 nodes
// Step 2: Define anatomical landmarks
Define landmarks:
- Apices
- Diaphragm
- Lung base edges
- Anterior segments
// Step 3: Optimize node positions using energy minimization
Optimize node positions by minimizing the energy function T(u):
T(u) = Σγ ||z(ξ₁, ξ₂) - z_d||² + ∫ g(u(ξ)) dξ
// where:
// z = nodal coordinates (x, y, z)
// g = smoothness constraint (curvature regularization)
// γ = data matching weight factor

Mean fitting error: 5.2 ± 2.3 mm

Data Alignment and Statistical Shape Modeling

Prior to PCA, lung shapes are aligned using General Procrustes Analysis (GPA) to eradicate variations due to translation, rotation, and, in the case of the size-exclusive model, scaling.

Alignment Equation

The alignment is expressed as:

$$ \bar{S} = \alpha R S + T, $$

where:

$S$ is the original shape vector,
$R$ is the rotation matrix,
$T$ is the translation vector,
$\alpha$ is the scaling factor (applied only in size-inclusive models).

Principal Component Analysis (PCA)

PCA is employed to reduce the high-dimensional shape vector into principal modes that capture the primary sources of variance.

PCA Decomposition via SVD

The shape matrix $\mathbf{S}$ is decomposed using singular value decomposition (SVD):

$$ \mathbf{S} = \mathbf{U} {\Sigma} \mathbf{V}^{T}, $$

where:

$\mathbf{U}$ contains the eigenvectors of $\mathbf{S} \mathbf{S}^{T}$,
${\Sigma}$ is the diagonal matrix of singular values $\sigma_i$ (with $\lambda_i = \sigma_i^2$ being the eigenvalues),
$\mathbf{V}^{T}$ holds the principal directions in the data space.

Each lung shape is reconstructed as:

$$ M_{l}(w) = \bar{S}_0 + w, u_{l}, $$

with:

$\bar{S}_0$ being the mean shape,
$u_l$ the $l$th principal component,
$w$ the corresponding weight factor.

The variance explained by each mode is:

$$ \text{Variance Explained} = \frac{\sigma_l^2}{\sum_{i} \sigma_i^2} \times 100%. $$

Results

Correlation with Age:
Specific principal shape modes display strong correlations with age (e.g., $R = -0.75, ; p < 0.001$),
indicating that lung geometries become more “pyramidal” as age increases.
Sex and BMI Associations:
Differences in shape related to lung size manifest in the size-inclusive model, highlighting sex-related volumetric differences. In contrast, the size-exclusive model emphasizes purely geometric deformations, with moderate correlations observed for BMI.

Implications

Clinical Applications:
The detailed mathematical framework enables lung shape to be employed as a biomarker for aging and disease. Deviations from the normative PCA model may assist in early detection or staging of conditions like COPD and IPF.
Technical Advancements:
The fusion of finite element modeling with PCA offers a versatile and robust method for analyzing complex organ shapes. This approach can be adapted to other anatomical structures requiring similar quantitative assessments.
Future Directions:
Enhancing the energy model and incorporating dynamic imaging (e.g., covering breathing cycles) could further refine shape analysis. Additionally, exploring non-linear dimensionality reduction techniques may uncover further subtleties in lung morphology.

The SPARC DRC

Wed, 20 Sep 2023 00:00:00 +0000

Authors: Mahyar Osanlouy, Anita Bandrowski, Bernard de Bono, David Brooks, Antonino M. Cassarà, Richard Christie, Nazanin Ebrahimi, Tom Gillespie, Jeffrey S. Grethe, Leonardo A. Guercio, Maci Heal, Mabelle Lin, Niels Kuster, Maryann E. Martone, Esra Neufeld, David P. Nickerson, Elias G. Soltani, Susan Tappan, Joost B. Wagenaar, Katie Zhuang, Peter J. Hunter
Publication: Front. Physiol. (June 2021)
Website: SPARC Science
Interactive 3D Models & Data: MAP Web App

Overview

The NIH Common Fund’s Stimulating Peripheral Activity to Relieve Conditions (SPARC) program is an ambitious initiative aimed at revolutionizing our understanding of the autonomic nervous system (ANS) and its role in organ function. As the lead developer for the Mapping Core (MAP-Core) of the SPARC Data and Resource Center (DRC), I collaborated with an interdisciplinary team to create a comprehensive framework for curating, visualizing, and analyzing ANS data across different species.

The SPARC DRC serves as the technological foundation for the SPARC program, providing researchers worldwide with access to standardized experimental data, computational models, and visualization tools through a unified web portal (https://sparc.science). Our work addresses a critical gap in biomedical research: the need for standardized, reproducible, and interoperable data and computational resources to advance our understanding of neural control of organ function and develop effective neuromodulation therapies.

Key Contributions

Our team developed several groundbreaking innovations that form the technological foundation of the SPARC program:

1. FAIR-Compliant Data Management Platform

We implemented comprehensive data standards and workflows to ensure all SPARC data adheres to FAIR principles (Findable, Accessible, Interoperable, and Reusable). This includes standardized metadata schemas, dataset organization structures, and persistent identifiers for long-term data discoverability.

2. Knowledge Management System

We created a sophisticated knowledge graph integrating anatomical relationships, neural connectivity, and semantic annotations from multiple species. This system enables powerful cross-species comparisons and semantic searches that were previously impossible with traditional database approaches.

3. Multi-Scale Anatomical Mapping Framework

One of our most significant innovations is the development of 3D material coordinate systems (“scaffolds”) for organs and bodies. These scaffolds provide a revolutionary solution to a complex problem: how to map and compare data from organs that undergo substantial deformation (e.g., beating hearts, inflating lungs) across different experimental conditions and species.

4. Computational Modeling Platform

We developed o²S²PARC (“open, online simulations for SPARC”), an online computational environment that enables researchers to analyze data, develop models, and simulate neuromodulation scenarios using cloud computing resources.

Methodology

Developing Standardized Data Workflows

We established comprehensive curation and data standards to ensure consistency across all SPARC datasets. Key aspects include:

Implementation of the SPARC Dataset Structure (SDS) for consistent file organization
Development of the Minimal Information Standard (MIS) for semantic metadata
Integration with community ontologies and specialized annotation tools
Creation of SODA (Software for Organizing Data Automatically) to assist researchers in preparing data submissions

Creating Neural Connectivity Maps

To represent the complex connectivity of the ANS across different species, we:

Utilized the ApiNATOMY toolkit to create topological and semantic models of neural pathways
Generated interactive flatmap diagrams that visualize neural connectivity in 2D for multiple species
Linked these maps to the SPARC Knowledge Graph for data integration

3D Scaffold Framework for Anatomical Mapping

A cornerstone of our innovation is the development of the 3D scaffold system. This methodology addresses the fundamental challenge of mapping data from organs that undergo significant deformation. Our approach includes:

1. Material Coordinate System Development

We developed Scaffold-Maker, specialized CAD software that creates 3D material coordinate systems for body organs. These scaffolds use finite element meshes with smooth interpolation to represent complex anatomical geometries.

2. Cross-Species Anatomical Representation

We designed the scaffolds to accommodate topological differences between species, allowing for meaningful cross-species comparisons despite anatomical variations. For example, we created heart scaffolds that accommodate different numbers of pulmonary veins in humans (4), pigs (2), and rats (3).

3. Deformation-Invariant Mapping

Our material coordinate system maintains consistent references to tissue locations regardless of organ deformation. This allows data to be mapped and compared across different physiological states (e.g., cardiac contraction phases).

4. Integration with Whole-Body Models

We developed methods to embed organ scaffolds within whole-body coordinate systems, facilitating multi-scale data integration from cellular to organism levels.

Results

Our work has produced several tangible outcomes that demonstrate the power of the SPARC DRC infrastructure:

Comprehensive ANS Data Resource

The SPARC Portal now serves as the world’s most comprehensive resource for standardized ANS data, with:

Curated datasets from multiple species and organ systems
Rich semantic annotations linking experimental data to anatomical locations
Standardized protocols and metadata ensuring reproducibility

Species-Specific Anatomical Scaffolds

We have successfully created detailed 3D scaffolds for key organs across multiple species:

Heart scaffolds for human, pig, and rat with species-specific topologies
Colon scaffolds capturing species variations in haustra and taeniae coli structures
Whole-body scaffolds with embedded organ systems
Bladder, stomach, and lung scaffolds for integrating diverse experimental data

Neural Mapping Demonstrations

We have demonstrated successful registration of neural data to our scaffold systems:

Mapping of intrinsic cardiac neurons (ICNs) from rat hearts to a standardized cardiac scaffold
Integration of data from multiple specimens into “integrative” scaffolds
Preservation of spatial relationships across deformed organ states

Computational Modeling Environment

The o²S²PARC platform now provides:

Online access to sophisticated computational models
Tools for simulating neuromodulation effects
Reproducible workflows for data analysis
Collaborative environments for model development

Implications

The SPARC DRC infrastructure we’ve developed has significant implications for both basic science and clinical applications:

Advancing Basic Science

Our work enables new approaches to understanding the ANS by:

Facilitating cross-species comparisons of neural circuitry
Providing standardized reference systems for integrating multi-scale data
Creating a foundation for quantitative, data-driven neural mapping
Enabling reproducible computational analyses

Clinical Applications

The infrastructure directly supports the development of bioelectronic medicine by:

Providing detailed anatomical maps for targeting neuromodulation
Enabling simulation of device effects on neural activity
Supporting the optimization of stimulation parameters
Facilitating translation between animal models and human applications

Beyond SPARC

The methodologies and tools we’ve developed have applications beyond the ANS:

The scaffold concept can be applied to any deformable biological system
Our FAIR data management approaches set standards for other large-scale initiatives
The o²S²PARC computational platform can support diverse modeling applications

Conclusion

The SPARC DRC represents a paradigm shift in how we approach the study of the autonomic nervous system. By creating standardized frameworks for data organization, anatomical mapping, and computational modeling, we’ve laid the groundwork for a new era of quantitative, integrative neuroscience research that can directly inform the development of bioelectronic therapies. Our 3D scaffold methodology, in particular, solves the fundamental challenge of mapping biological data in deformable systems, enabling unprecedented integration of experimental data across scales, species, and physiological states.

AI-Driven Radiomics for Precision Prostate Cancer Therapy

Sun, 20 Aug 2023 00:00:00 +0000

AI-Driven Radiomics for Precision Prostate Cancer Therapy

Authors: Tsz Him Chan, Annette Haworth, Alan Wang, Mahyar Osanlouy et al.
Publication: EJNMMI Research (2023)
Code: Available on Request

Overview

This pioneering study developed an AI-powered radiomics pipeline to optimize biologically targeted radiation therapy (BiRT) for prostate cancer. By integrating PSMA PET/CT with multiparametric MRI (mpMRI), we created voxel-level predictions of tumor location and grade through advanced image registration and machine learning techniques.

Key Innovations

Multi-modal fusion: First study combining PSMA PET radiomics with DCE MRI perfusion parameters
Voxel-wise prediction: Achieved 0.89 AUC for tumor detection using 3D radiomic features
Grade differentiation: Developed two-stage RFC model separating high/low-grade disease (Accuracy: 0.67-0.99)
HPC-optimized registration: Reduced spatial uncertainty to 3.3mm using deformable histology alignment

Computational Pipeline

Data Integration: Co-registered PSMA PET/CT (5 scanners), mpMRI (2x Siemens 3T), and whole-mount histology
Key Steps:

align_pet_ct() → rigid_registration(mpMRI) →
deformable_registration(ex_vivo_MRI) →
histology_annotation_propagation()

Challenges Solved:
Bladder filling artifacts in PET
Partial volume effects in 3.27mm PET slices
Non-linear prostate deformation post-resection

2. Radiomic Feature Engineering

Modality	Features Extracted	Key Parameters
PSMA PET	3D LoG, LBP, GLCM textures	SUVmax, metabolic tumor volume
DCE MRI	Ktrans, iAUGC60, TTP perfusion maps	Pharmacokinetic modeling
ADC Maps	NGTDM coarseness, percentile values	b=1200 s/mm² diffusion restriction

Feature Selection:

ANOVA filtering → Gini impurity ranking
Final feature set: 50 most discriminative parameters

Machine Learning Architecture

Tumor Detection Model

Two-stage RFC Framework:

Location Detection: 842 sensitivity/804 specificity
Grade Classification: Low vs High Grade (ISUP ≥3)

Performance Comparison:

Model	AUC	Sensitivity	Specificity
PET Alone	0.865	0.781	0.799
mpMRI Alone	0.882	0.802	0.801
Combined	0.890	0.842	0.804

Receiver operating characteristics for different models

Key Radiomic Predictors

PET: 3D LoG(σ=3mm) minimum
ADC: NGTDM Coarseness
DCE MRI: Ktrans 90th percentile

Clinical Implications

Personalized Radiotherapy: Enables voxel-level dose painting based on metabolic/perfusion features
Early Recurrence Prediction: High-grade lesions showed 2.8× higher PSMA uptake (p<0.01)
Technical Impact:
Solved partial volume effects in PET-guided planning
Demonstrated perfusion > diffusion for grade prediction
Open-source registration framework [3D Slicer Plugin]

Future Directions

Multi-institutional validation across PET/MRI scanners
DL Enhancement: Replace handcrafted features with 3D CNNs
Real-time Adaptation: Integrate with MR-Linac systems

This project received funding from Prostate Cancer Foundation of Australia and Health Research Council of New Zealand.

Generalization of predictive coding model to dynamic stimuli

Mon, 26 Apr 2021 00:00:00 +0000

Introduction

Predictive coding is an established model of perceptual inference and learning in hierarchical networks of the brain. It describes a network of neuron-like nodes, which can infer stimulus properties from noisy input using only local computation, i.e. the changes of activity of each neuron in the model is determined only by its inputs and its current activity levels. Furthermore, the network encodes the estimated parameters of a probabilistic model from which the stimuli are generated in its synaptic connections, and learn these parameters employing only local plasticity, where the changes in synaptic weights only depend of activities of pre and post-synaptic neurons. In its original form the predictive coding model assumes static input stimuli. However, most of the stimuli experienced by animals and humans change in time, and it is critical for survival to efficiently interpret such stimuli.

Very soon after developing the predictive coding model, it was pointed that it could be generalized to dynamic stimuli, and the Kalman filter could be employed to infer the states of hidden variables represented by the model. However that work has not described how such computation could be implemented in a biologically plausible network of neuron-like nodes. More recently, a generalization of predictive coding to dynamic stimuli has been proposed, in which different neurons represent not only the hidden variables, but also their temporal derivatives. Although it is possible to implement this model in a network only employing local computation and local plasticity, this network requires a very intricate and specific pattern of connectivity between various neurons, and there is no evidence that such connectivity exists in cortical circuits.

This report outlines a simple generalization of predictive coding model to dynamic stimuli, which does not require more intricate network than the original predictive coding model. A simulation of the proposed generalizations is shown for a toy problem, and directions are suggested in which the work on the model needs to be conducted.

Model

Process generating stimuli

In this report we assume that stimuli are generated from a very simple linear model, which parallels the assumptions about signal made by the Kalman filter. Let us denote an observed stimulus at time $t$ by a vector with elements $y_i(t)$. Let us assume that the stimulus depends on values of hidden variables denoted by $x_j(t)$ according to:

$$ y_i(t) = \sum_j w_{i,j} x_j(t) + \epsilon_{y,i}(t) \quad (1) $$

In the above equation, $w_{i,j}$ form a matrix of parameters, and $\epsilon_{y,i}(t)$ is a noise process (with zero mean). Furthermore, let us assume that the hidden variables evolve according to:

$$ \dot{x}_j = \sum_k v_{j,k} x_k(t) + \epsilon_{x,j}(t) \quad (2) $$

Analogously as above, $v_{j,k}$ form a matrix of parameters, and $\epsilon_{x,j}(t)$ is a noise process. A natural way for estimating $x_j$ from $y_i$ is to employ the Kalman filter, but it involves complex equations, and it is not clear how such computation could be implemented in a network of neurons. Therefore, this report describes a simpler method for estimating $x_j$ that has a more natural neural implementation.

Computations in the model

Given a observed stimuli $y_i$, we will seek to infer the hidden variables $x_j$ and estimate the parameters $w_{i,j}$ and $v_{j,k}$. In the reminder of Section 2, we will use $x_j$, $w_{i,j}$ and $v_{j,k}$ to denote the estimates of corresponding terms in Equations (1) and (2) above. We wish to find $x_j$ such that the stimulus $y_i$ is close to the predicted value $\sum_j w_{i,j} x_j$. Thus we define error in prediction of $y_i$ as:

$$ e_i = y_i - \sum_j w_{i,j} x_j \quad (3) $$

We wish to minimize a squared sum of these errors which we denote by $E_y = \frac{1}{2} \sum_i \varepsilon_{y,i}^2$. Hence we change $x_j$ in the direction opposite to the gradient of $E_y$, but we additionally append this dynamics towards our goal with the natural evolution of $x_j$:

$$ \dot{x}_j = - \frac{\partial E_y}{\partial x_j} + \sum_k v_{j,k} x_k $$

Evaluating the gradient, we obtain the equation describing the dynamics of our estimate of hidden variables:

$$ \dot{x}_j = \sum_i w_{i,j} \varepsilon_{y,i} + \sum_k v_{j,k} x_k $$

In order to learn parameters $w_{i,j}$, which describe how $y_i$ depends on $x_j$, we modify them to minimize $E_y$:

$$ \dot{w}_{i,j} = - \alpha \frac{\partial E_y}{\partial w_{i,j}} = \alpha \varepsilon_{y,i} x_j $$

In the above equation $\alpha$ denotes a learning rate. In order to learn parameters $v_{j,k}$ describing the natural dynamics of hidden variables, we need to define an error in prediction of this dynamics:

$$ \varepsilon_{x,j} = \dot{x}_j - \sum_k v_{j,k} x_k $$

We wish to minimize squared sum of these errors $E_x = \frac{1}{2} \sum_j \varepsilon_{x,j}^2$, and hence we modify the weights in the direction opposite to the gradient of $E_x$ over $v_{j,k}$:

$$ \dot{v}_{j,k} = \alpha \varepsilon_{x,j} x_k $$

In summary, this generalized predictive coding model continuously updates hidden variables and parameters according and recomputes prediction errors.

Possible neural implementations

Inference of hidden variables $x_j$ from sensory input $y_i$ can be easily performed in a network shown in Figure 1A. The bottom layer consists of sensory neurons representing the stimulus. They project to neurons computing prediction error. These errors are then send to the neurons encoding hidden variables which change their activity according to the dynamics equation above. The weights of connections between neurons encoding errors and hidden variables are symmetric, i.e. equal in both direction. This network has an architecture very similar to a standard predictive coding model , but additionally includes recurrent connections between the neurons encoding hidden variables with weights $v_{j,k}$.

Learning parameters $w_{i,j}$ corresponds to local Hebbian plasticity in the network of Figure 1, analogously as in the standard predictive coding networks. However, learning parameters $v_{j,k}$ is less straightforward because the prediction error $\varepsilon_{x,j}$ is not explicitly represented in activity of any neurons in the network. Nevertheless, it is possible to construct models in which $\varepsilon_{x,j}$ would be represented in internal signals (e.g. concentrations of particular ions or proteins) within neurons encoding $x_j$, and let us consider two such possible models.

The first model is illustrated in Figure 1B. In this network, the recurrent inputs from neurons representing hidden variables converge on a separate dendritic branch, which sums them and thus can compute $\sum_k v_{j,k} x_k$. To compute the error $\varepsilon_{x,j}$, the neuron would need to compute the difference between change in its activity and the membrane potential in the dendrite. Since both of these quantities are encoded within the same neuron, it is plausible that such a computation may be performed, and an error encoded in an internal signal. Such signal could then drive local synaptic plasticity.

An alternative way of computing prediction errors $\varepsilon_{x,j}$ relies on an observation that by combining equations describing the dynamics of $\dot{x}_j$ adn error $\varepsilon_{x,j}$, we see that these errors are equal to:

$$ \varepsilon_{x,j} = \sum_i w_{i,j} \varepsilon_{y,i} $$

Such input from the previous layer of prediction error neurons could be computed in dendrites shown in Figure 1C. The membrane potential of such dendrite would need to set level of an internal signal that would govern the plasticity within the entire neuron. This mechanism could be considered biologically plausible as it is analogous to observations that high membrane potential of apical dendrites of pyramidal neurons triggers plateau potentials via calcium influx, leading to a burst of spikes by the neuron. Such bursts of spikes may subsequently induce synaptic plasticity.

Results

I tested the model on a simple problem in which hidden variables and stimuli were 2-dimensional. The hidden variables were generated according to $\dot{x}_j = \sum_k v_{j,k} x_k(t) + \epsilon_{x,j}(t)$ with parameters $v_{j,k}$ set to a rotation matrix visualized in Figure 2C. The stimuli were generated according to $y_i(t) = \sum_j w_{i,j} x_j(t) + \epsilon_{y,i}(t)$ with parameters $w_{i,j}$ set to the identity matrix, so that the stimuli were simply noisy versions of the hidden variables. The stimuli are shown in Figure 2A, and they are noisy periodic signal because parameters $v_{j,k}$ were set to a rotation matrix. The variables and stimuli were generated with a sampling frequency 10, by solving our equations using Euler method with integration step $0.1$. During each step, noise with variance of $0.01$ was added.

At the start of the learning process, weights $w_{i,j}$ were initialized to an identity matrix, while the weights between hidden units were all set to $v_{j,k}=0$. The hidden units were also initialized to $x_j=0$. The hidden variables and parameters were updated according to our equations above using the Euler method with integration step of $0.1$, and learning rate set to $\alpha=0.01$.

Figure 2B shows that as the learning progressed, the error in prediction of stimuli decreased, so the network was able to better predict the stimuli. Figure 2D visualizes learned values of parameters $v_{j,k}$, which are very close to the original parameters used to generate the training data (cf. Figure 2C). Thus the network was able to discover the underlying process generating the stimuli.

Discussion

This report outlines generalization of predictive coding to dynamic stimuli for linear and shallow generative models, so more work would be required to extend this to more complex models and relate it with experimental data. In particular the work can be extended in the following directions:

Introduce the non-linear activation functions to hidden units, and test if the model can learn dynamics of non-linear systems.
Introduce multiple levels of hierarchy and investigate if the model can extract dynamics of stimuli generated by hierarchical dynamical systems.
Test the model performance on real world machine learning problems, e.g. prediction of EEG signal from past history.
Investigate if after training with natural stimuli the receptive fields of neurons in the model have similar properties to the receptive fields in the visual system, analogously as in neural networks trained with the back-propagation algorithm.

COVID-19 Modelling Effort

Tue, 21 Jul 2020 00:00:00 +0000

Authors: Mahyar Osanlouy, Gonzalo D. Maso Talou, David Brooks, Soroush Safaei, Alan Garny, and Peter J. Hunter

Introduction

Amid COVID-19 pandemic, a group of computational scientists and engineers in the ABI decide to set up models of COVID- 19 transmission and infection in New Zealand. The taskforce developed technologies to model/analyze epidemic outbreaks, and a pipeline from data to model to visualization that offers a sandbox environment to test policies and track the infection process with an unprecedented level of details.

Modelling infection in buildings, cities and countries: A multiscale integrative model

Modelling the spread of COVID-19 is of paramount importance to strategize effective policies to reduce the impact of the disease in our health and economic system. By means of this model, we can identify social interactions, places and activities that favour the spread of the virus in order to propose appropriate countermeasures. The model is also a sandbox environment where we can test policies and observe the consequences in the complex environment of a city, to understand the pros and cons of each action.

The model is composed of three scales: i) place/building scale; ii) transport/city scale; iii) cluster/country scale. Each scale is implemented with a different type of model (see Figure 1), and data between these models is communicated by a homogenization process. In this process, data of the smaller scale computes the parameters of the larger scale such that it represents the same phenomenology, e.g., if the number of infections grew by 10 times during 10 hours in a transport scale, then rate of infection for the country scale can be computed as one infection per hour assuming a base 10 exponential growth model.

Finally, the model evolution and statistical analysis of the simulation is visualized on top of a dynamic web interface that allows the users to easily navigate through the results in time and space using cartographic and 3D views powered by Unity. In the current development, simulations can be executed real-time, online (visualizing results as the simulation is taking place) or offline. The offline modality allows the user to navigate forward and backwards on time, turning the model into an ideal tool for contact tracing and understanding the causality of a particular scenario.

Person-to-person transmission: Place model

The place model is implemented by means of agent-based models (ABMs) where the actions and status of each person can be modelled, predicted and, if needed, controlled. These models offer high-resolution inferences of infection transmissions at the level of individuals giving us a clear understanding of the impact of different infection mechanisms (sneeze, direct or via surface transmission, social distancing efficiency, among others).

ABMs capture the interactions of individuals in time and space using real data and detailed transmission pathways, providing insight into settings where transmission is occurring and how clusters emerge. Various hypotheses can be tested by modelling and evaluating social behaviour within different scenarios giving insights about which elements are key to draw effective policies. In modelling disease transmission, these scenarios may include different pandemic intervention strategies deployed by government and/or policymakers such as containment, mitigation, and suppression. Every individual (agent) also has specific attributes which may contribute to the behaviour of the model and, as a result, to the result of the disease transmission. Therefore, ABMs simulate the dynamics of a system where individual-level characteristics, environmental factors, and the interaction of the two are of importance.

As an example, we have developed an ABM to simulate the transmission of COVID- 19 across shoppers in a supermarket (see Figure 2). Here, shoppers are the agents of the model which have particular attributes including basic demographics (as accurate as possible), health states and characteristics (e.g. susceptible, immunity to the disease, etc.), probability of practising social distancing and basic hygiene, etc. These are what we call the specifics of the local transmission dynamics. Additionally, a natural disease history model for COVID-19, i.e., the infectivity profile from the onset of infection, over an incubation period, to the peak of infectivity, to recovery, for a single symptomatic or asymptomatic individual becomes extremely important in longitudinal simulations. The supermarket is the spatial environment where the simulation is confined. It is precisely this environment where the rules and policies are defined and the mutual interaction with the shoppers occurs. The environment and its rules have direct and indirect implications on the shoppers and, ultimately, on the probability of the disease transmission. The same is also true for the effects of the shoppers and their behaviour on the environment.

Circulation and infection through the city arteries: Transport model

Cities are a complex aggregate of place models, where people navigate between them at specific times of day using specific means of transportation, among which infection is also occurring. The proposed transportation model communicates numerous place models and computes infections while agents commute between places.

This model deals with three complexities: (i) How to transport an agent from place A to B for a given type of transportation; (ii) Which models of infection are appropriate for each type of transportation mechanism (bus, private service,personal transport, walking, cycling, running, aeroplane and ferry); (iii) How an agent decides the next action to perform after visiting a place.

To properly address (i), we need to consider the available path when the agent is using a specific transport type. The route for commuting to work will be quite different for bus, cycling and walking, as some spaces can be traversed for one mode but not for the remaining ones. Then, specific transportation graphs are derived for each type of transportation. Once the route is known, the agent will have to compute a trip plan, which involves a set of intermediate action to reach the final destination. For example, if the agent decides to take the bus to go to the supermarket, he will perform the following actions: 1) walk to the bus stop, 2) wait until the bus arrives,3) commute from the bus stop of departure to the bus stop of arrival, 4) walk from the arrival bus stop to the supermarket. To ensure that the transport model delivers precise estimates of commuting time, it considers the time of day, day of the week with the timetables when an agent is using public transport. Also, the agents can define certain constraints as the maximum walking distance, waiting time and number of bus connexions.

Regarding (ii), commuting by bus or walking through the street may clearly present two different scenarios of exposure where a given set of policies will have a different impact. Models for each transportation type can be modelled by specific place models (as seen in the previous section), or by partial differential equations (PDEs) considering the number of susceptible, exposed and infected subjects and the total area of circulation. Each approach presents its advantages depending on the policies to be tested, tracing detail required and available resources for the simulation.

Lastly, (iii) is addressed by modelling the agent’s behaviour with a stochastic planner. Each time an agent is idle the planner is executed, evaluating theprobability for this agent to take different actions (such as go to work, go to sleep, return home, visit a friend, go shopping) based on his personal attributes (age, occupation, family), history of previous actions and time and day of the week. After all the probabilities have been computed, the agent performs the most probable action. Once the action is finished the agent is idle and the planning starts again.

Note that all the probabilities distributions needed to characterize the agent’s behaviour and the number of agents across the country can be extracted from national census data.

Modelling nation-wide infection: Cluster models

The cluster model is based on the standard SEIR (susceptible-exposed-infected-removed) approach, gathering all transport model data for each city or region and reducing it to a set of ordinary differential equations. The rates and coefficients of this model can be computed by fitting each cluster model with the observations of the corresponding transport model and, at the same time, the cluster model informs the transport among cities serving as a boundary condition of the transport model.

This macro-scale serves as a dashboard of the infection process in large communities. Its simplicity renders tractable and easy to analyze variables, and their variation over time - e.g. infection, mortality and recovery rates - which is extremely useful for specialists and policymakers.