Generative 3D Cardiac Shape Modelling for in-silico Trials

Andrei Gasparovici
(proceedings), html, Mathematical modeling, models in biomathematics, Numerical Analysis, paper

Abstract


We propose a deep learning method to model and generate synthetic aortic shapes based on representing shapes as the zero-level set of a neural signed distance field, conditioned by a family of trainable embedding vectors with encode the geometric features of each shape. The network is trained on a dataset of aortic root meshes reconstructed from CT images by making the neural field vanish on sampled surface points and enforcing its spatial gradient to have unit norm. Empirical results show that our model can represent aortic shapes with high fidelity. Moreover, by sampling from the learned embedding vectors, we can generate novel shapes that resemble real patient anatomies, which can be used for in-silico trials.

Authors

Andrei Gasparovici
Babeș-Bolyai University, Cluj-Napoca, Romania
Advanta, Siemens SRL, Brașov, Romania
Tiberiu Popoviciu Institute of Numerical Analysis, Cluj-Napoca, Romania

Alex Serban
Advanta, Siemens SRL, Brașov, Romania
Transilvania University of Brașov, Romania

Keywords

Synthetic shape generation; in-silico trials, deep learning

Paper coordinates

A. Gasparovici, A. Serban, Generative 3D Cardiac Shape Modelling for In-Silico Trials, Proceedings of the EFMI Special Topic Conference 2024, Series Studies in Health Technology and Informatics, vol. 321, published 2024, Editors: L. Stoicu-Tivadar, A. Benis, T.M. Deserno, S.D. Bolboacă, K. Saranto, M. Crişan-Vida, P. Gallos, O.S. Chirila, P.Weber, G. Mihalaş, O. Tamburis, ISBN 978-1-64368-554-0 (online), pp. 190 – 194, DOI: http://doi.org/10.3233/SHTI241090

PDF

http://SHTI-321-SHTI241090.pdf

About this paper

Journal
Publisher Name
DOI

http://doi.org/10.3233/SHTI241090

Print ISSN
Online ISSN

google scholar link

Paper (preprint) in HTML form

SHTI-321-SHTI241090

Generative 3D Cardiac Shape Modelling for in-silico Trials

Andrei GASPAROVICI a , b , c , 1 a , b , c , 1 ^(a,b,c,1){ }^{\mathrm{a}, \mathrm{b}, \mathrm{c}, 1}a,b,c,1 and Alex SERBAN b , d b , d ^(b,d){ }^{\mathrm{b}, \mathrm{d}}b,d a a ^(a){ }^{\mathrm{a}}a Babeş-Bolyai University, Cluj-Napoca, Romania b b ^(b){ }^{\mathrm{b}}b Advanta, Siemens SRL, Braşov, Romania c c ^(c){ }^{\mathrm{c}}c Tiberiu Popoviciu Institute of Numerical Analysis, Cluj-Napoca, Romania d d ^(d){ }^{\mathrm{d}}d Transilvania University of Brașov, RomaniaORCiD ID: Andrei Gasparovici https://orcid.org/0009-0009-6150-5540,Alex Serban https://orcid.org/0000-0001-9571-7968

Abstract

We propose a deep learning method to model and generate synthetic aortic shapes based on representing shapes as the zero-level set of a neural signed distance field, conditioned by a family of trainable embedding vectors with encode the geometric features of each shape. The network is trained on a dataset of aortic root meshes reconstructed from CT images by making the neural field vanish on sampled surface points and enforcing its spatial gradient to have unit norm. Empirical results show that our model can represent aortic shapes with high fidelity. Moreover, by sampling from the learned embedding vectors, we can generate novel shapes that resemble real patient anatomies, which can be used for in-silico trials.

Keywords. Synthetic shape generation, in-silico trials, deep learning

1. Introduction

In-silico trials provide an opportunity to accelerate testing of medical devices, accounting for a wider range of patients compared to in-vitro trials. To enable in-silico trials, it is necessary to have access to a pool of diverse patient data or have the ability to generate synthetic data. For example, the simulation of Transcatheter Aortic Valve Implantation procedures can be performed computationally on distinct aortic shapes representing diverse human anatomies. In this paper, we introduce a deep learning (DL) based method for generating 3D aortic valve shapes. DL has significant advantages over conventional methods such as statistical shape models (SSMs), allowing the reuse of learned representations for distinct tasks, including conditioning other generative processes, such as calcium deposit generation, and predicting hemodynamic measurements using physics-informed DL models.
Previously, Hoeijmakers et al. [3] used a SSM to reconstruct aortic geometries from CT images and to predict the pressure drop across the aortic valve by performing computational fluid dynamics simulations. Similarly, Verstraeten et al. [7] used a SSM to generate new aortic shapes, represented as deformations of a template shape. The synthetic shapes are used to generate virtual cohorts which are similar to real cohorts and considered anatomically plausible for in-silico trials.
A three-dimensional shape S S SSS can be represented as the zero-level set of a function f : R 3 → R f : R 3 → R f:R^(3)rarrRf: \mathbb{R}^{3} \rightarrow \mathbb{R}f:R3→R, that is:
(1) S = { x ∈ R 3 : f ( x ) = 0 } (1) S = x ∈ R 3 : f ( x ) = 0 {:(1)S={x inR^(3):f(x)=0}:}\begin{equation*} S=\left\{x \in \mathbb{R}^{3}: f(x)=0\right\} \tag{1} \end{equation*}(1)S={x∈R3:f(x)=0}
For a watertight shape (i.e., a three-dimensional surface having a well-defined interior and no holes), a suitable choice for f f fff is the signed distance function (SDF), which is a continuous mapping that assigns to each point in space the distance to the surface, with the sign indicating whether the point is inside (positive) or outside (negative) the shape (Park et al. [5]). If f f fff is known, the zero-level set can be recovered using the well-established Marching Cubes meshing algorithm (Lorensen and Cline [4]).
Using this method to represent shapes, Park et al. [5] introduced a DL-based method for shape modeling and generation by representing a family of watertight shapes S ( k ) S ( k ) S^((k))S^{(k)}S(k) as the zero-level set of a neural network f θ : R d × R 3 → R f θ : R d × R 3 → R f_(theta):R^(d)xxR^(3)rarrRf_{\theta}: \mathbb{R}^{d} \times \mathbb{R}^{3} \rightarrow \mathbb{R}fθ:Rd×R3→R :
(2) S ( k ) = { x ∈ R 3 : f θ ( z k ; x ) = 0 } , (2) S ( k ) = x ∈ R 3 : f θ z k ; x = 0 , {:(2)S^((k))={x inR^(3):f_(theta)(z_(k);x)=0}",":}\begin{equation*} S^{(k)}=\left\{x \in \mathbb{R}^{3}: f_{\theta}\left(z_{k} ; x\right)=0\right\}, \tag{2} \end{equation*}(2)S(k)={x∈R3:fθ(zk;x)=0},
where θ θ theta\thetaθ are trainable parameters and z k ∈ R d z k ∈ R d z_(k)inR^(d)z_{k} \in \mathbb{R}^{d}zk∈Rd are trainable embedding vectors, which encode the geometric properties of each shape. The proposed network has an autodecoder architecture and is trained on a set of pairs consisting of points and corresponding SDF values sampled from a neighborhood of the surface and the ambient.
For smooth surfaces, the signed distance field f f fff is differentiable and satisfies the eikonal PDE ‖ ∇ f ( x ) ‖ = 1 PDE ‖ ∇ f ( x ) ‖ = 1 PDE||grad f(x)||=1\mathrm{PDE}\|\nabla f(x)\|=1PDE‖∇f(x)‖=1. Taking advantage of this property, Gropp et al. [1] improved the previously mentioned technique by introducing an implicit geometric regularization method, which makes the network f θ f θ f_(theta)f_{\theta}fθ produce a signed distance field by encouraging the spatial gradient of f θ f θ f_(theta)f_{\theta}fθ to have unit norm. This regularization allows for ground truth points to be sampled only from the surface.

2. Methods

Based on the works of Park et al. [5] and Gropp et al. [1], we develop a DL-based aortic shape generation model described together with the training dataset as follows.
Model. Following Park et al. [5] and Gropp et al. [1], we represent each of the n n nnn shapes as the zero-level set of a neural signed distance field f θ f θ f_(theta)f_{\theta}fθ. Our network is an 8 -layer fully connected auto-decoder, with softplus activation functions. As illustrated in Figure 1, we add a skip-connection after the fourth layer, which improves stability and allows the network to utilize high-dimensional embedding vectors effectively (Rebain et. al. [6]).
Figure 1. Auto-decoder neural network architecture for aortic shape modeling and generation.
The loss function consists of three terms. The first term is responsible for making the network f θ f θ f_(theta)f_{\theta}fθ vanish on points sampled from the surface and enforcing the consistency
of normal vectors. The second term is the implicit geometric regularization, which makes f θ f θ f_(theta)f_{\theta}fθ produce a signed distance field by encouraging the spatial gradient of f θ f θ f_(theta)f_{\theta}fθ to have unit norm, and the last term is an L 2 L 2 L^(2)L^{2}L2 regularization for the embedding vectors. Concretely, the loss has the expression:
(3) L ( θ , z 1 , … , z n ) = 1 n ∑ k = 1 n L ( k ) ( θ , z k ) (3) L θ , z 1 , … , z n = 1 n ∑ k = 1 n   L ( k ) θ , z k {:(3)L(theta,z_(1),dots,z_(n))=(1)/(n)sum_(k=1)^(n)L^((k))(theta,z_(k)):}\begin{equation*} \mathcal{L}\left(\theta, z_{1}, \ldots, z_{n}\right)=\frac{1}{n} \sum_{k=1}^{n} \mathcal{L}^{(k)}\left(\theta, z_{k}\right) \tag{3} \end{equation*}(3)L(θ,z1,…,zn)=1n∑k=1nL(k)(θ,zk)
where L ( k ) L ( k ) L^((k))\mathcal{L}^{(k)}L(k) is the loss corresponding to the k k kkk-th shape, given by
(4) L ( k ) ( θ , z k ) = 1 N ∑ i = 1 N ( | f θ ( z k ; x i ( k ) ) | + ‖ ∇ x f θ ( z k ; x i ( k ) ) − n i ( k ) ‖ 2 ) + τ E x ∼ D [ ( ‖ ∇ x f θ ( z k ; x ) ‖ − 1 ) 2 ] + λ ‖ z k ‖ , k = 1 , n ― (4) L ( k ) θ , z k = 1 N ∑ i = 1 N   f θ z k ; x i ( k ) + ∇ x f θ z k ; x i ( k ) − n i ( k ) 2 + τ E x ∼ D ∇ x f θ z k ; x − 1 2 + λ z k , k = 1 , n ¯ {:[(4)L^((k))(theta,z_(k))=(1)/(N)sum_(i=1)^(N)(|f_(theta)(z_(k);x_(i)^((k)))|+||grad_(x)f_(theta)(z_(k);x_(i)^((k)))-n_(i)^((k))||^(2))],[+tauE_(x∼D)[(||grad_(x)f_(theta)(z_(k);x)||-1)^(2)]+lambda||z_(k)||","k= bar(1,n)]:}\begin{align*} \mathcal{L}^{(k)}\left(\theta, z_{k}\right)= & \frac{1}{N} \sum_{i=1}^{N}\left(\left|f_{\theta}\left(z_{k} ; x_{i}^{(k)}\right)\right|+\left\|\nabla_{x} f_{\theta}\left(z_{k} ; x_{i}^{(k)}\right)-n_{i}^{(k)}\right\|^{2}\right) \tag{4}\\ & +\tau \mathbb{E}_{x \sim D}\left[\left(\left\|\nabla_{x} f_{\theta}\left(z_{k} ; x\right)\right\|-1\right)^{2}\right]+\lambda\left\|z_{k}\right\|, k=\overline{1, n} \end{align*}(4)L(k)(θ,zk)=1N∑i=1N(|fθ(zk;xi(k))|+‖∇xfθ(zk;xi(k))−ni(k)‖2)+τEx∼D[(‖∇xfθ(zk;x)‖−1)2]+λ‖zk‖,k=1,n―
The expectation in (4) is estimated by sampling points x ∈ R 3 x ∈ R 3 x inR^(3)x \in \mathbb{R}^{3}x∈R3 from a distribution D D DDD which is the average between a sum of Gaussian distributions centered at the sampled points and a uniform distribution.
For training the network, we use the Adam optimizer for 5000 epochs with an initial learning rate of 10 − 3 10 − 3 10^(-3)10^{-3}10−3, which is decreased by a factor of 0.5 every 500 epochs. The embedding vectors have dimension d = 256 d = 256 d=256d=256d=256 and are initialized from a normal distribution with mean 0 and standard deviation 10 − 2 10 − 2 10^(-2)10^{-2}10−2. The coefficients in (4) are chosen τ = 0.5 τ = 0.5 tau=0.5\tau=0.5τ=0.5 and λ = 10 − 4 λ = 10 − 4 lambda=10^(-4)\lambda=10^{-4}λ=10−4.
Dataset. The training dataset consists of n = 97 n = 97 n=97n=97n=97 aortic root (non-watertight) meshes normalized to a unit ball. For each mesh, we sample N = 500000 N = 500000 N=500000N=500000N=500000 surface points along with unit normal vectors { ( x i ( k ) , n i ( k ) ) ∈ R 3 × R 3 , i = 1 , N ― } , k = 1 , n ― x i ( k ) , n i ( k ) ∈ R 3 × R 3 , i = 1 , N ¯ , k = 1 , n ¯ {(x_(i)^((k)),n_(i)^((k)))inR^(3)xxR^(3),i= bar(1,N)},k= bar(1,n)\left\{\left(x_{i}^{(k)}, n_{i}^{(k)}\right) \in \mathbb{R}^{3} \times \mathbb{R}^{3}, i=\overline{1, N}\right\}, k=\overline{1, n}{(xi(k),ni(k))∈R3×R3,i=1,N―},k=1,n―.
Shape reconstruction and generation. For an embedding vector z z zzz, the corresponding mesh can be recovered by running Marching Cubes on the SDF values f θ ( z , x ) f θ ( z , x ) f_(theta)(z,x)f_{\theta}(z, x)fθ(z,x) produced by the network on a uniform 256 3 256 3 256^(3)256^{3}2563 grid. In addition, new shapes can be generated by taking z z zzz to be a convex combination (weighted average) of the learned embedding vectors:
(5) z = α 1 z 1 + α 2 z 2 + ⋯ + α n z n (5) z = α 1 z 1 + α 2 z 2 + ⋯ + α n z n {:(5)z=alpha_(1)z_(1)+alpha_(2)z_(2)+cdots+alpha_(n)z_(n):}\begin{equation*} z=\alpha_{1} z_{1}+\alpha_{2} z_{2}+\cdots+\alpha_{n} z_{n} \tag{5} \end{equation*}(5)z=α1z1+α2z2+⋯+αnzn
with α 1 , … , α n ≥ 0 α 1 , … , α n ≥ 0 alpha_(1),dots,alpha_(n) >= 0\alpha_{1}, \ldots, \alpha_{n} \geq 0α1,…,αn≥0 and α 1 + ⋯ + α n = 1 α 1 + ⋯ + α n = 1 alpha_(1)+cdots+alpha_(n)=1\alpha_{1}+\cdots+\alpha_{n}=1α1+⋯+αn=1.

3. Results

Shape reconstruction. For verifying the fidelity of the learned representations, we reconstruct the meshes from the training set using the learned embedding vectors and evaluate the similarity using the Chamfer distance (Fan et al. [2]) computed on sets of points sampled from each mesh.
The distribution of Chamfer distances between the ground truth and reconstructed shapes is illustrated in Figure 2. We observe that almost all shapes can be accurately reconstructed, except for one outlier, which upon manual inspection revealed a reconstruction artifact near the outlet.
Shape generation. To evaluate the novelty of the generated shapes, we study the distribution of pairwise Chamfer distances (illustrated in Figure 3) on a set of 100 new shapes, generated by interpolating between 2,4 , and 8 embedding vectors. We observe
Figure 2. Reconstruction Chamfer distance distribution.
Figure 3. Pairwise Chamfer distance distribution.
that increasing the number of interpolated vectors decreases the variance of the distribution, because the novel shapes better resemble existing shapes in the dataset. When using minimum two vectors for interpolation, the distribution trades less variance, and the shapes generated are more diverse. An illustration of 3 generated samples is provided in Figure 4. Each shape is generated from a new embedding vector constructed by interpolating between two learned embedding vectors z 1 z 1 z_(1)z_{1}z1 (corresponding to the shape in Figure 4a) and z 2 z 2 z_(2)z_{2}z2, with α 1 = α 2 = 0.5 α 1 = α 2 = 0.5 alpha_(1)=alpha_(2)=0.5\alpha_{1}=\alpha_{2}=0.5α1=α2=0.5.
Figure 4. Examples of shapes generated with the proposed method.

4. Discussion

Our initial experiments show that the shapes produced by this method exhibit less variance, and thus less diversity, compared to the original training data. This reduced variance results from the generative process, which relies on interpolating pre-trained embedding vectors. Adjusting the number of embedding vectors involved in the generation process can help manage the trade-off between variance and diversity. Our findings suggest that the number of samples chosen for interpolation significantly impacts the diversity of the generated cohorts.
Exploring innovative generative techniques represents a promising direction for future research. For instance, selecting vectors based on their variance relative to the
initial training distribution could enhance the variance in synthetic datasets. Moreover, introducing noise during the generation process may further increase cohort diversity. Clinicians can then fine-tune these hyperparameters to customize the cohorts for specific in-silico trials. Additionally, incorporating anatomical measurements [7] may provide a more precise way to assess the variance of synthetic cohorts and could be used to generate shapes with specific anatomical features.

5. Conclusions

Our study demonstrates the potential of using DL-based methods to generate aortic shapes for cardiovascular in-silico trials. While our method effectively generates synthetic cohorts, it may exhibit limitation in variance and diversity due to its reliance on interpolating pre-trained embedding vectors. To address these drawbacks, future work could explore more advanced generative techniques such as generative adversarial networks or diffusion models and incorporate attention mechanisms for conditioning the neural field. Additionally, expanding the scope of our model to include or generate features such as calcifications or predictions of velocity and pressure could significantly enhance its accuracy and efficacy. These improvements would enable clinicians to tailor synthetic cohorts for specific in-silico trials, leading to more personalized outcomes.

Acknowledgements

This project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 101017578.

References

[1] Gropp A, Yariv L, Haim N, Atzmon M, Lipman Y. Implicit geometric regularization for learning shapes. In: Proceedings of the 37th International Conference on Machine Learning. ICML'20. c2020. [cited 16.06.2024] Available from http://proceedings.mlr.press/v119/gropp20a/gropp20a.pdf
[2] Fan H, Su H, Guibas LJ. A Point Set Generation Network for 3D Object Reconstruction From a Single Image [Internet]. c2017 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 2017 [cited 16.06.2024] Available from https://openaccess.thecvf.com/content_cvpr_2017/papers/Fan_A_Point_Set_CVPR_2017_paper.pdf
[3] Hoeijmakers MJMM, Waechter-Stehle I, Weese J, Van de Vosse FN. Combining statistical shape modeling, CFD, and meta-modeling to approximate the patient-specific pressure-drop across the aortic valve in real-time. Int J Numer Method Biomed Eng. 2020 Oct;36(10):e3387, doi:10.1002/cnm.3387.
[4] Lorensen WE, Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics 1987;21(4):163-169, doi:10.1145/37402.37422
[5] Park JJ, Florence P, Straub J, Newcombe R, Lovegrove S. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In: 2019 IEEE/CVF CVPR. IEEE, June 2019. [cited 16.06.2024] Available from https://openaccess.thecvf.com/content_CVPR_2019/papers/Park_DeepSDF_Learning_Continuous_Sig ned_Distance_Functions_for_Shape_Representation_CVPR_2019_paper.pdf
[6] Rebain D, Matthews MJ, Yi KM, Sharma G, Lagun D, Tagliasacchi A. Attention Beats Concatenation for Conditioning Neural Fields. Transactions on Machine Learning Research 2022;11:1-17.
[7] Verstraeten S, Hoeijmakers M, Tonino P, Brüning J, Capelli C, van de Vosse F, Huberts W. Generation of synthetic aortic valve stenosis geometries for in silico trials. Int J Numer Method Biomed Eng. 2024 Jan;40(1):e3778, doi: 10.1002/cnm. 3778.
  1. 1 1 ^(1){ }^{1}1 Corresponding Author: Andrei Gasparovici; E-mail: andrei.gasparovici@ubbcluj.ro.

[1] Gropp A, Yariv L, Haim N, Atzmon M, Lipman Y. Implicit geometric regularization for learning shapes. In: Proceedings of the 37th International Conference on Machine Learning. ICML’20. c2020. [cited 16.06.2024] Available from http://proceedings.mlr.press/v119/gropp20a/gropp20a.pdf
[2] Fan H, Su H, Guibas LJ. A Point Set Generation Network for 3D Object Reconstruction From a Single Image [Internet]. c2017 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 2017 [cited 16.06.2024] Available from https://openaccess.thecvf.com/content_cvpr_2017/papers/Fan_A_Point_Set_CVPR_2017_paper.pdf
[3] Hoeijmakers MJMM, Waechter-Stehle I, Weese J, Van de Vosse FN. Combining statistical shape modeling, CFD, and meta-modeling to approximate the patient specific pressure-drop across the aortic valve in real-time. Int J Numer Method Biomed Eng. 2020 Oct;36(10):e3387, doi:10.1002/cnm.3387.
[4] Lorensen WE , Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics 1987;21(4):163-169, doi:10.1145/37402.37422
[5] Park JJ, Florence P, Straub J, Newcombe R, Lovegrove S. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In: 2019 IEEE/CVF CVPR. IEEE, June 2019. [cited 16.06.2024] Available from https://openaccess.thecvf.com/content_CVPR_2019/papers/Park_DeepSDF_Learning_Continuous_Sig ned_Distance_Functions_for_Shape_Representation_CVPR_2019_paper.pdf
[6] Rebain D, Matthews MJ, Yi KM, Sharma G, Lagun D, Tagliasacchi A. Attention Beats Concatenation for Conditioning Neural Fields. Transactions on Machine Learning Research 2022;11:1-17.
[7] Verstraeten S, Hoeijmakers M, Tonino P, Brüning J, Capelli C, van de Vosse F, Huberts W. Generation of synthetic aortic valve stenosis geometries for in silico trials. Int J Numer Method Biomed Eng. 2024 Jan;40(1):e3778, doi: 10.1002/cnm.3778.

2024

Related Posts