3D reconstruction of patient-specific carotid artery geometry using clinical ultrasound imaging

- One of the techniques used to diagnose carotid artery disease is the ultrasound (US) examination. The initiation and development of vascular diseases depends also on the flow conditions in the artery. Additional parameters that cannot be directly measured can be obtained by performing numerical simulations using patient-specific geometry. In this study, the Finite Element Method (FEM) is used to analyze the distribution of relevant blood flow characteristics. Images obtained from the US examinations are used to adapt the generalized carotid bifurcation model to the specific patient. The approach presented in this study combines the deep learning approach for the image segmentation and automated 3D reconstruction method to create a semi-generic geometrical model of the carotid artery that is adapted to the specific patient, using data obtained from only several US images. The presented methodology enables efficient segmentation, extraction of the morphological parameters and creation of 3D meshed volume models that can be also used for the further computational simulations of blood flow.


Introduction
One of the diseases in human cardiovascular system is carotid artery stenosis (CAS). Early detection of this disease is very important because if it is not adequately treated, it may potentially have deteriorating consequences, such as a debilitating stroke. These serious accidents occur when atherosclerotic plaques in the arteries suddenly rupture, leading to the obstruction of the blood flow to the heart or to the brain. There are several examination techniques that can be applied to analyze the state of patient's carotid artery, including both 3D imaging techniques like computed tomography (CT) and magnetic resonance imaging (MRI) [1], as well as ultrasound (US) examination [2][3][4]. US technique is accurate, noninvasive and inexpensive and is hence one of the first methods that is applied in diagnosing carotid artery disease. The US combines two procedures: traditional B-mode (gray-scale) ultrasound and color-Doppler ultrasound. Using the first procedure, images of the vessels in question at rest are created from the reflected sound waves. The second procedure provides information about the motion of the blood and enables the visualization of blood flow and measurement of flow velocity. Both these procedures produce two-dimensional (2D) cross-sectional images.
Deep learning is a promising machine learning tool for the automatic classification and interpretation of medical image data. This technique has been applied extensively in several medical imaging applications, such as for brain, lung, and breast imaging. Its application to arterial structures such as the carotid artery and for noisy image data as those found in US images of the carotid bifurcation has not been reported yet [5]. Analysis of a carotid US images requires segmentation of the vessel wall, lumen, and plaque of the carotid artery. Convolutional neural networks are one of the most common tools used in image segmentation. In [6], the U-Net convolutional neural network was used for lumen segmentation from US images of the entire carotid system.
After obtaining the US images, additional analysis can be performed. Since the initiation and development of vascular diseases depends also on the flow conditions in the artery, it is useful to analyze the distribution of relevant blood flow characteristics. The US examination can be used to measure wall thickness and blood velocities in patients. Additional parameters that cannot be directly measured can be obtained using numerical simulations, more precisely computational fluid dynamics (CFD) methods. In this study, the Finite Element Method (FEM) is used to analyze blood velocity, pressure and wall-shear stress distribution [7][8][9].
The simulations can be performed using idealized carotid bifurcation, but in order to determine the correct patient-specific diagnosis that also considers the individual anatomy of the particular patient, it is necessary to perform simulations using a patient-specific geometry. In this study, images obtained using the US examinations are used to adapt the generalized carotid bifurcation model to the specific patient.
The paper is organized as follows: The applied methods and the used clinical data are discussed in Section 2. The results of the training of the neural network and the blood flow simulation results for a specific patient are presented in Section 3. Section 4 discusses relevant work in literature and concludes the paper.

Materials and methods
In this Section, the details of the applied techniques will be given.

Clinical data
In order to develop and validate the tool for 3D reconstruction of carotid artery based on the US images as input, a dataset of original and annotated US images obtained in the Serbian Clinical Centre, was used. The annotated images were used for processing and training, while the original images were used for the validation. The dataset consisted of 108 patients who underwent the US examination (baseline time point). For each patient the common carotid artery, the branches and carotid bifurcation in transversal and longitudinal projections are captured. The examination was performed in B-mode and Color-Doppler mode. All imaging data were anonymized respecting the data protection and safety.

Extraction of vessel lumen from US images using deep learning
The preprocessing of US dataset included following actions:  Annotation of carotid lumen area  Resizing/Cropping of US images to 512x512 pixels  Classification of longitudinal and transversal US images  Classification of B-mode and Color-Doppler mode images The automatic carotid artery segmentation is done using U-Net [10] based deep convolutional networks. U-Net is a convolutional neural network for image segmentation with the most important application being in segmentation of medical images. It is based on encoder-decoder model. The variant of U-Net in this study is slightly modified from the original. It has two additional blocks in both encoder and decoder. Each block in encoder has two convolutional layers with 3×3 filters, followed by 2×2 max pooling. In each decoder block, 2×2 upconvolution and skip connection are followed by three more convolutional layers with 3×3 filters, and the last decoder block produces the segmentation mask with 1×1 convolution and sigmoid activation function. All convolutional layers are padded so that the resulting activation map preserves the same height and width. In this way, the resulting segmentation map has the same resolution as the input image. Also, the variant of U-Net applied in this study uses batch normalization after each convolutional layer which proves to work a lot better on our data than the original U-Net model [11]. All batch normalization layers are followed by a ReLU activation.
The model is trained with a combination of binary cross-entropy and soft dice coefficient as a loss function, which is expressed as: where pred y and true y denote the flattened predicted probabilities and the flattened ground truths of the image. Soft Dice coefficient loss is described in [12,13].
From the entire dataset explained in Section 2.1, all subfolders corresponding to the patients are randomly divided into training, validation and testing sets by a ratio of 8:1:1 at the carotid artery level (either for the left or for the right arterial model), such that a total of 700 images have been taken out for training purposes and the remaining is used for validation and testing.

3D reconstruction of the carotid artery
The 3D reconstruction of patient-specific carotid artery is performed using the available clinical imaging data for the particular patient. One of the main problems with the used patient data set is the limited number of 2D transversal cut that is available. In order to overcome the problem with the missing cuts, the generalized model was used as the basis, that is then improved with the available data. This process is illustrated in Fig. 1. The generalized model was defined according to data presented in literature [14,15]. The transversal cut of the common carotid artery (CCA -annotated by the B line in Fig. 1) and the external carotid artery (ECA -the lower segment annotated by the C line in Fig. 1) is used to define the shapes of cross-sections of the carotid artery in these segments. The longitudinal cut of the internal carotid artery (ICA -annotated by the A segment in Fig. 1) is used to extract the centerline of the ICA and the diameters in this segment, while the transversal cuts of the ICA (the upper segment annotated by the C line and the cross-section annotated by the D line in Fig. 1) are also used to more accurately define these particular cross-sections. The lengths of the branches are defined according to the information obtained from the cuts. Length of the ECA was taken to be half of the length of the ICA that was extracted from the longitudinal cut. The length of the CCA was set to be 1.2 times higher than the diameter of the CCA that was extracted from the appropriate transversal cut. The 3D reconstruction process is performed using the algorithm similar to the one presented in literature [16] and the overview of this process is shown in Fig. 2. First, the extracted segments are obtained using the procedure explained in Section 2.2 ( Fig. 2A). Then, the smoothing of the obtained curves is performed (Fig. 2B), by converting them to nonuniform B-spline curves. After that, the cross-sections along the centerline are defined (Fig. 2C) and finally, the 3D FE mesh is generated (Fig. 2D). Details of these steps are given in the sequel.
The tube-like surface for each branch is defined with crosssections positioned along the parameterized centerline. The cross-sections are either circles with extracted diameters, or the extracted segments. In both cases, the points of the cross-sections are projected onto the trihedron normal-binormal plane of each centerline to obtain the patches of the surface. In order to ensure the regularity of points and to prevent twisting of individual patches, all points were converted to polar coordinates in normal-binormal plane and sorted in circular direction. These patches in 3D space are used for NURBS surface representation that can be defined as:  2), according to the Cox-de Boor recursive algorithm [17].
Using the parameterized vessel centerline it is possible to perform the discretization of the vessel. The introduction of the parameterization enabled to control the meshing process by defining the density of nodes in the mesh in both longitudinal (u) and circular (v) directions. Details of the meshing procedure are described in [16]. The main idea is to decompose the branches of the carotid artery into connected hexahedra elements, which were composed of quadrilateral patches (that are actually the vessel cross-sections). Finally, the individual branches of the carotid artery are then connected into a bifurcation, following the procedure described in [16].

Image segmentation results for the lumen
In order to present the results of the neural network training, the binary classification task for image segmentation was considered. Four common classification metrics are considered for quantitative evaluation, including accuracy (ACC), precision (P), recall (R), and Dice coefficient. The results for the test set are shown in Table 1. The method used in this study is compared with thresholding technique and FCN-8s model with VGG16 as a backbone classifier [18]. Similar to U-Net, FCN model proved to work better with batch normalization. However, FCN has almost twice as many trainable parameters than U-Net network, so the U-Net can be trained faster and is more memory efficient.

Blood flow through the 3D reconstructed carotid artery
The results of the 3D reconstruction and blood flow simulations are presented in this Section. Clinical data for a patient that belonged to the testing set is used for the 3D simulations. The reconstruction is performed using the procedure described in Section 2.3, while the blood flow simulations were performed using the software PakF [8,9]. The fluid is observed as a Newtonian fluid and its characteristics were defined according to the literature [8,14]. The fluid density is set to 1.05 g/cm 3 and the kinematic viscosity is set to 0.035 cm 2 /s. The initial condition is defined using the inlet velocity (at the inlet of CCA) that was set to be equal to 96 cm/s, for all cases. The outflow boundary condition is defined at the outlet branches (ICA and ECA).

Discussion and conclusion
There are many papers in literature that use B-mode US images to analyze the state of the patient's carotid arteries. Kristensen et al. [66] performed US examinations on 1390 patients in a clinical study, which provided fast diagnostics and enabled better clinical decision-making. The US imaging was also used to monitor the progression of atherosclerotic plaque [19]. In combination with the Color-Doppler US, it is possible to obtain quantitative estimates of luminal narrowing and other flow measurements. Visualizing and analyzing the morphological structure of ca-rotid bifurcations are important for understanding the etiology of carotid atherosclerosis, which is a major cause of stroke and transient ischemic attack. However, conventional 2D ultrasound imaging has technical limitations in observing the complicated 3D shapes and asymmetric vasodilation of bifurcations, since it is not possible to obtain a composite view of the vessel wall, lumen and plaque. This problem was solved with 3D US imaging that was successfully used in literature to pre-surgically evaluate the carotid stenosis [20].
The three-dimensional (3D) visualization of the carotid artery from the 2D US images would also be very useful. This would be possible if a series of crosssectional B-mode US images is acquired. Rosenfield et al. [2] used B-mode US images to perform 3D reconstruction of the carotid bifurcation from its inward flow. Authors of a more recent paper [21] also performed 3D reconstruction from 2D transversal US images.
Another limitation of US imaging is that it requires skills, expertise and knowledge from the sonographer when making data acquisition. It also influences the quality of the images. Therefore, data taken by different sonographer has the possibility of producing different images. The difference in image results can affect the process of locating and segmenting CA areas. Automatic determination of the location of the CA area can be used to avoid the subjectivity of the sonographer. However, there are not many methods proposed in literature that automatically reconstruct carotid stenosis and/or bifurcation with vessel course alterations from US images. The proposed methodology includes training of a deep learning model (U-net) for the image segmentation phase and a computer-based automated 3D reconstruction method that is capable of generating a semi-generic geometrical model of the carotid artery that is adapted to the specific patient, using data obtained from only several transversal US images.
The presented approach and methodology which combines the data mining and 3D reconstruction of carotid artery enables efficient segmentation, extraction of the morphological parameters and creation of 3D meshed volume models that can be also used for the further computational simulations of blood flow.