An integrated system for the segmentation of atherosclerotic carotid plaque ultrasound video

The robust border identification of atherosclerotic carotid plaque, the corresponding degree of stenosis of the common carotid artery (CCA), and also the characteristics of the arterial wall, including plaque size, composition, and elasticity, have significant clinical relevance for the assessment of future cardiovascular events. To facilitate the follow-up and analysis of the carotid stenosis in serial clinical investigations, we propose and evaluate an integrated system for the segmentation of atherosclerotic carotid plaque in ultrasound videos of the CCA based on video frame normalization, speckle reduction filtering, M-mode state-based identification, parametric active contours, and snake segmentation. Initially, the cardiac cycle in each video is identified and the video M-mode is generated, thus identifying systolic and diastolic states. The video is then segmented for a time period of at least one full cardiac cycle. The algorithm is initialized in the first video frame of the cardiac cycle, with human assistance if needed, and the moving atherosclerotic plaque borders are tracked and segmented in the subsequent frames. Two different initialization methods are investigated in which initial contours are estimated every 20 video frames. In the first initialization method, the initial snake contour is estimated using morphology operators; in the second initialization method, the Chan-Vese active contour model is used. The performance of the algorithm is evaluated on 43 real CCA digitized videos from B-mode longitudinal ultrasound segments and is compared with the manual segmentations of an expert, available every 20 frames in a time span of 3 to 5 s, covering, in general, 2 cardiac cycles. The segmentation results were very satisfactory, according to the expert objective evaluation, for the two different methods investigated, with true-negative fractions (TNF-specificity) of 83.7 ± 7.6% and 84.3 ± 7.5%; true-positive fractions (TPF-sensitivity) of 85.42 ± 8.1% and 86.1 ± 8.0%; and between the ground truth and the proposed segmentation method, kappa indices (KI) of 84.6% and 85.3% and overlap indices of 74.7% and 75.4%. The segmentation contours were also used to compute the cardiac state identification and radial, longitudinal, and shear strain indices for the CCA wall and plaque between the asymptomatic and symptomatic groups were investigated. The results of this study show that the integrated system investigated in this study can be successfully used for the automated video segmentation of the CCA plaque in ultrasound videos.


I. Introduction
T he size and composition of carotid atherosclerotic plaque have been shown to be independent predictors of future cardiovascular events [1], with vulnerable plaques described as containing a large lipid core, a thin fibrous cap, and dense macrophage inflammation in or beneath its surface, whereas stable plaques are characterized by larger quantities of calcium and collagen and less lipids and inflammatory cells [2]. accurate and precise segmentation of the atherosclerotic carotid plaque in ultrasound B-mode video [see Fig. 1(a)] allows for the extraction of different anatomical properties of the artery wall, and plaque that can be most useful to the physicians for the evaluation of plaque development and the process of atherosclerosis in modeling and evaluating the risks for future cardiovascular events. Ultrasound, elastography, and other imaging methods such as spectroscopic photoacoustic imaging can be integrated to advance the characterization of the plaques and corresponding vulnerability [3] because the variation in the mechanical properties of the different tissue types is huge [4]. For example, the elasticity modulus of calcified plaque material is 50 times higher than the modulus of cellular plaque tissue [1]. over the past few years, different quantitative methods have been developed for the common carotid artery (cca) segmentation [5], [6], as well as plaque heterogeneity and plaque echogenicity characterization [2]- [5].
The availability and evaluation of different characteristics of carotid plaque is of outmost importance especially because it is widely accepted that the experts are only moderately confident that there is adequate evidence to determine which interventions to use for different patient populations [3]. However the manual segmentation of the plaque in the cca in ultrasound video [see also Fig. 1(h)], or over large sequences of images is not only a tedious and time consuming task, but also suffers from large inter-and intra-observer variability and it is difficult to trace boundaries for each time frame of the cardiac cycle and for large volume of data sets [6], [7]. Polak et al. [8] demonstrated that when expert operators manually delineate the intima and adventitia borders of the cca, errors can be considerable and are different for the intima and the adventitia borders. The errors are further amplified when experts manually delineate atherosclerotic carotid plaques. Thus, the need exists for the development of new automated or Manuscript   , (c) binary image frame after dilation with a square window shape of size 9 × 9 after removing edges, (d) edge image frame after removal of erroneous edges, (e) roI manual positioning and interpolating spline and detected initial contour with contour selection, (f) initial snake contour mapped on the first video frame, (g) final snake contour after snake deformation, and (h) manual delineation of plaque.
semi-automated tools and techniques for the assessment of the risk of stroke from ultrasound videos of the cca incorporating plaque segmentation.
The predictive ability to identify which patients will have a stroke is poor, where the current practice of assessing the risk of stroke relies on measuring the thickness of the cca wall [intima-media-thickness (IMT)] [9], [10] or the artery lumen stenosis by identifying the plaque borders in the carotid artery [6]. although the IMT and the degree of stenosis can reliably be delineated in still B-mode ultrasound images [1], [6], [9], [10], the moving borders of the atherosclerotic carotid plaque and the identification of the systolic and diastolic video states [7] may provide additional information of an individual's stroke risk, and treatment of asymptomatic patients may be improved [11].
We propose in this study an integrated system for the video segmentation of the cca based on our previous work [6], [10], to further facilitate the quantitative assessment of atherosclerosis disease. The main extension to our previous work in [6], which was based on still image segmentation, is the consideration of moving frames, the identification of the states diagram of the video, the extraction of systolic and diastolic states [7], and the automated re-initialization of the snake contour every 20 frames based on recent work presented in [10]. Preliminary results of this study were also published in [12], where we proposed an improved initialization method for the positioning of the initial snake contour. The snake contour was repositioned using the level-set formulation of chan and Vese [13], using random initialization, which provided a segmentation of the cca ultrasound image sequences into different distinct regions, one of which corresponds to the carotid wall region above the lumen, and another which corresponds to the carotid wall region below the lumen, including the plaque borders. The key point of the proposed framework is the accuracy of the initialization.
To the best of the authors' knowledge there is only one published study for the segmentation of atherosclerotic carotid plaque in ultrasound cca videos [7]. The method by destrempes et al. [7] is based on a Bayesian segmentation model and is evaluated on 33 video sequences. still, several other studies investigated the segmentation of atherosclerotic carotid plaque in ultrasound images. an overview of these techniques is given in Table I. Hamou and El-sakka [14] proposed a method based on the canny edge detector to detect the plaque in longitudinal cca ultrasound images. a morphological-based approach for the carotid contour extraction was proposed in [15] for longitudinal ultrasound images of the cca, incorporating speckle reduction filtering, contour quantization, morphological contour detection, and a contour enhancement stage. Mao et al. [16] proposed a discrete dynamic contour model for extracting the carotid artery lumen in 2-d transversal ultrasound images, whereas abolmaesumi et al. [17] introduced a method, based on the star algorithm improved by Kalman filtering, for plaque segmentation in transversal carotid ultrasound images. a semi-automatic method for plaque segmentation in 3-d images of the cca using the balloon model introduced in [18] was proposed by Gill et al. [19]. loizou et al. [6] proposed a plaque segmentation method, based on the Williams and shah snake [20], for the extraction of the cca using an automated contour estimation and applied on 80 ultrasound images of the cca. The atherosclerotic carotid plaque in [21], was segmented in 56 2-d longitudinal ultrasound images using a gradient-based snake segmentation method and fuzzy K-means algorithm with an initialization based on pixel intensity. In [22], the Hough transform was applied to perform segmentation of plaque in four 2-d cross-sectional ultrasound images of the cca. In [23], an automated segmentation method for 3-d ultrasound carotid plaque based on a geometrically deformable model, taking advantage of both the local and regional detectors, was proposed. More recently, in [24], a 3-d semi-automated segmentation method using sparse field level sets, where the users choose anchor points on each boundary, was proposed for the segmentation of 3-d cca images. all of these methods, with the exception of [7] (in which ultrasound video segmentation of the plaque in the cca was proposed), have investigated and  [14] canny edge detection no abdel-dayen [15] Morphological based no -Mao [16] discrete dynamic contour no 7 abolmaesumi [17] Kalman filtering no 1 Gill [19] Balloon no 2 loizou [6] active contour model yes 80 delsanto [21] Gradient based snake with fuzzy k-means yes 56 Golemati [22] Hough transform no 4 Ultrasound video destrempes [7] Bayesian model no 33 Ultrasound imaging 3-d Zahalka [23] Geometrically deformable model yes 1 Ukwatta [24] level sets no 21 aIc = automatic initial contour; N = number of cases investigated.
proposed different solutions for the segmentation of the atherosclerotic carotid plaque in ultrasound images of the cca. as shown in Table I, different methods were investigated for the segmentation of the atherosclerotic carotid plaque in ultrasound images, yet these studies were evaluated on a limited number of subjects. Therefore, the need still exists for the development, implementation, and evaluation of an integrated system enabling the automated segmentation of ultrasound imaging carotid plaque.
In this paper, an integrated system for the segmentation of the atherosclerotic carotid plaque in 2-d ultrasound video of the common carotid artery (cca) is presented and evaluated. The system builds on some of the authors' previous work [6], and incorporates image frame normalization, speckle reduction filtering, initial contour estimation, M-mode generation, and snake segmentation for the advancement of evaluation and treatment of the carotid atherosclerosis.

A. Recording of Ultrasound Videos
a total of 43 B-mode longitudinal ultrasound videos (from 38 asymptomatic subjects, aged 56 ± 12, and 5 symptomatic subjects, aged 53 ± 16; there were 17 female and 26 male subjects) of the cca bifurcation of subjects were recorded representing different types of atherosclerotic plaque formation with irregular geometry typically found in this blood vessel. almost all subjects demonstrated a left or a right cca stenosis of 30% or larger. The videos were acquired by the aTl HdI-5000 ultrasound scanner (advanced Technology laboratories, seattle, Wa) and were recorded digitally on a magneto optical drive, with a size of 576 × 768 pixels with 256 gray levels, a pixel size of 59 µm (17 pixels/mm) and having a frame rate of 100 frames per second. This frame rate is high; however, it was used because these videos will also be analyzed for plaque motion estimation. The aTl HdI-5000 ultrasound scanner is equipped with a 256-element fine-pitch high-resolution 50-mm linear array and a multi-element ultrasound scan head with an extended operating frequency range of 5 to 12 MHz, and it offers real spatial compound imaging. The video segmentations were performed for 3 to 5 s intervals, covering, in general, 2 to 3 cardiac cycles.

B. Manual Plaque Segmentation
an expert neurologist with more than 20 years of clinical experience (coauthor dr. M. Pantziaris) manually delineated the plaque borders between plaque and artery wall, and the borders between plaque and blood, every 20 frames on 43 longitudinal B-mode ultrasound videos of the cca, after image normalization and speckle reduction fil-tering (see sections II-c and II-d), using Matlab software developed by our group [see Fig. 1(h)]. The manual segmentations traced in the first frame could be transferred to the second frame and then could be readjusted by the expert accordingly. In total, 538 (10 750 frames/20 = 538) ultrasound frames of the cca were delineated. on average, 13 frames per video were manually delineated by the expert. The procedure used for carrying out the manual delineation was the one established and documented in the asymptomatic carotid stenosis and risk of stroke (acsrs) project protocol [25] for still images of the cca. The correctness of the work carried out by a single expert was monitored and verified by at least one other expert. In cases in which several plaques were located in the cca, comprising multiple components, each plaque was segmented independently so that each contour initialization represents a different plaque.

C. Video Normalization of Ultrasound Videos
Brightness adjustments of ultrasound videos were carried out based on the method introduced in [26]. This improves image compatibility by reducing the variability introduced by different gain settings, different operators, different equipment, and facilitates ultrasound tissue comparability [26]. algebraic (linear) scaling of the first video frame was manually performed by linearly adjusting the image so that the median gray level value of the blood was 0 to 5 and the median gray level of the adventitia (artery wall) was 180 to 190 [26]. The intensity of the gray level values on the video frames ranged from 0 to 255. Thus, the brightness of all pixels in the video frame was readjusted according to the linear scale defined by selecting the two reference regions [see also Fig. 1(b)]. The subsequent frames of the video were then normalized based on the selection of the first frame. It is noted that a key point to maintaining a high reproducibility was to ensure that the ultrasound beam was at right angles to the adventitia, adventitia was visible adjacent to the plaque and that for image normalization, a standard sample consisting of half of the width of the brightest area of adventitia was obtained.

D. Speckle Reduction Filtering (DsFlsmv)
For speckle reduction, the filter dsFlsmv, introduced in [27], was applied to each consecutive frame prior to the plaque segmentation. The filters of this type utilize firstorder statistics such as the variance and the mean of a pixel neighborhood and may be described with a multiplicative noise model [27], [28]. The moving window size for the despeckle filter dsFlsmv was 5 × 5 and the number of iterations applied to each video frame was two; a complete description of the filter and its parameters can be found in [27]- [30]. an example of the application of the dsFlsmv filter is shown in Fig. 1(b) after image normalization.

E. Plaque Contour Initialization
Before running the video plaque snake segmentation algorithm, two different plaque initialization procedures (see also Figs. 1 and 2) were investigated for positioning the initial snake contour in the first frame of the video, as close to the area of interest (plaque borders) as possible. The two different initialization methods are explained in detail in the appendix. Prior to both initialization methods, all the video frames were normalized (see section II-B) and despeckled (see section II-c). The normalized despeckled first video frame of a cca video is shown in Fig. 1(b). It is also assumed that the carotid artery is properly imaged in the video according to the standard clinical guidelines.

F. Snake Segmentation
The Williams and shah snake segmentation method [20] was used to deform the snake and segment the plaque borders in each video frame. The snake contour, v(s), adapts itself by a dynamic process that minimizes an energy function [E snake (v, s)] defined as [20] , and E curv (v(s)) are the internal, image, external, continuity, and curvature energies of the snake, and α s , β s , and γ s are the strength, tension, and stiffness parameters, respectively. The method was proposed and evaluated in [6] on 80 ultrasound images of the cca and more details about the model can be found there. For the Williams and shah snake, the strength, tension, and stiffness parameters were equal to α s = 0.6, β s =0.4, and γ s = 2, respectively. This procedure was applied in cases where there was only one plaque present in the cca video. In cases in which multiple plaques were located at the near or far wall of the cca (22 out of 43 videos), thus forming multiple connected regions (see Fig. 3, left column), the proposed segmentation method was applied independently on each plaque component. The sampled contour points resulting from the previous step are used for the active contour initialization. The first and last points on the contour resulting from the initialization procedure presented in the previous section are connected to form a closed initial snake contour [see Figs. 1(g) and 2(f)]. The final plaque contour resulting from the snake deformation is mapped from the first video frame to the second and is sampled to provide the initialization for the snake deformation. The segmentation is carried out for a full cardiac cycle, which is identified using the procedure in section II-G. Every 20 video frames, the entire initialization procedure is repeated to detect the plaque borders again, reposition the snake and reassure that the snake contour will not converge away from the area of interest. If there were multiple plaques in the cca video, then the entire methodology is repeated for each plaque. Two more examples of video plaque segmentation are shown in Fig. 3.

G. M-Mode Image Generation, Boundary Extraction, State Identification, and Manual Delineation
The M-mode image [see Fig. 4(d)] can be generated in such a way that it crosses all plaque borders having maximum motion in opposite directions [31]. The procedure of the M-mode generation is illustrated in Fig. 4 and also described in [32]. Fig. 4(a) presents the first video frame of the cardiac cycle from a B-mode ultrasound video of the left cca, after normalization and despeckle filtering acquired from a male symptomatic subject at the age of 64 (having a stent on the right cca and a stenosis of 40% to 50%). Fig. 4(b) presents the automated snake segmentations of the plaque boundaries at the far wall of the cca and the lumen segmentation at the near wall. The segmented atherosclerotic plaque is shown in Fig. 4(c), whereas Fig. 4(d) presents the despeckled M-mode image generated from the cca video for the first 1200 frames (12 s). Perpendicular lines that cross the major axis of the plaque [from Fig. 4(c)], were placed automatically at the major axis quintiles (20%, 40%, 60%, and 80%). By scanning the intensity values along the straight perpendicular line selected by the user, the M-mode image is generated, by taking this line as the y-axis, and each frame of the video as the x-axis [see Fig. 4(d)]. Four M-mode images were generated for each of the corresponding four perpendicular lines. The manual delineations and all other measurements were performed (by M. Pantziaris) using a system implemented in Matlab from our group. The M-mode images were converted to binary images and morphological operators were applied to smooth the edges. Then, edge detection was applied on each M-mode image to derive the initial near and far wall boundaries and the rate of change [32]. The snake segmentation system [6] was also used to refine the derived snake contours [see From the final state diagram of the video in Fig. 4(f), a full cardiac cycle was selected by identifying the starting and ending frames of the cycle. The frames of the video corresponding to the identified cardiac cycle were then extracted. The segmentation algorithm (see section II-F) was then applied on the frames representing a full cardiac cycle.

H. Evaluation of the Segmentation Method, and State and Strain Metrics
The video segmentation methods were evaluated using the true-positive fraction (TPF) and the true-negative fraction (TnF), corresponding to sensitivity and specificity, respectively [33]. ratios of overlapping areas, can also be assessed by applying the similarity kappa index, KI, and the overlap index as used in [6]. These indices were computed at each video frame as where | | denotes the total number of pixels within the enclosed boundary (or cardinality, which is the number of elements in the set), ∩ denotes the intersection (the number of common pixels in the manual and snake-segmented areas), and ∪ the union (the number of all pixels defined by the manual and snake-segmented areas, where the common pixels are considered only once). GT denotes the number of pixels defined by the segmented area, representing ground truth delineated by the expert, and GT its complement. as denotes the number of pixels belonging in the area obtained by the snake segmentation method, and AS its complement. The intersection gives the probability that both as and GT occur and the union is the probability that either as or GT occurs.
To further evaluate the state-based identification algorithm, the following metrics between the automated and the manual state diagram timings were computed for each subject (video).   (a), we illustrate the points adventitia wall (aW), plaque wall (PW), media-adventitia wall (MaW), plaque at maximum point (P max ), and plaque at minimum point (P min ) (see [34] for definition).
The root mean square error (rMsE): where i represents the frame number, A and M represent the automated and manual state diagrams, and N is the number of states (2× the heart rhythm). The normalized mean square error (nMsE): where std N is the standard deviation over one subject (or video), which can be calculated as the root of the mean variance.
The mean average error (MaE) is The mean average relative error is The carotid diameter during contraction (cdc) and the carotid diameter during distension (cdd), and the percentage of the carotid wall distension (%cWd): The radial and longitudinal movements in these videos have been also investigated, where regions of interest (roIs) were selected for each cca video [see Fig. 4(a)] on the first normalized despeckled frame [34]. More specifically, three roIs, namely, the adventitia wall (aW) at the near wall, the plaque wall (PW), and the media-adventitia wall (MaW), were selected at the far wall of the cca, and two roIs, namely, the maximum plaque border (P max ) and the minimum plaque border (P min ) were selected. The following equations were used to calculate the radial (rs), longitudinal (ls), and shear strains (ss) from the radial and longitudinal displacement indices as documented in [34]: where rP and lP denote radial and longitudinal displacements, respectively, for a pair of roIs, and (ed) corresponds to end diastole. Based on (8)-(10), the radial strain at wall (rs), the longitudinal strain (ls), and the shear strain (ss) were calculated. Fig. 3 presents two different examples of video plaque segmentation with atherosclerotic plaques appearing at the cca in the left and right column. The segmentations were performed for the first and second cardiac cycle, for the 50th, 60th, 70th, and 150th video frames of the video. In the left column, a case from a 61-year-old female asymptomatic subject is presented with three different plaques at the near and far wall of the cca, with a left cca stenosis of 35% to 40%. In the right column, one can see a case from a 64-year-old male symptomatic subject with a stent on the right cca and with a stenosis of 40% to 50%. It is observed that the proposed segmentation algorithm is able to follow the plaque borders consistently as a result of the initialization procedures followed.

B. Evaluation of the Plaque Segmentation Method
Table II tabulates the quantitative results of the statistical analysis based on TnF, TPF, KI, and overlap index for the proposed video segmentation method performed for 1 to 2 cardiac cycles, on 43 ultrasound videos of the cca for the two different initialization methods (see two last rows in Table II). The results of the automated segmentation method are compared with the manual tracings of the expert, which are considered to be the ground truth. The results show that the proposed method using the two different initialization methods (mean ± std method 1 and mean ± std method 2) agrees with the expert by correctly detecting no plaque (TnF) in (83.7 ± 7.6% and 84.3 ± 7.5%) of the cases and by correctly detecting a plaque (TPF) in (85.42 ± 8.1% and 86.1 ± 8.0%) of the cases. The similarity kappa index, KI, and the overlap index for the proposed video snake segmentation method were equal to 84.6% and 85.3% and 74.7% and 75.4% using the two different initialization techniques, respectively. There was no significant difference between the two methods for all metrics investigated using a pair t-test at p < 0.05. However, a small improvement was observed in almost all evaluation metrics when the segmentations were performed with the second initialization method. It should further be noted that the snake contour may be attracted occasionally to local minima and converge to a wrong location. This occurred in less than 10% of the cases (i.e., in 4 videos).  Table III presents the results of the evaluation metrics between the manual and the automated state diagrams (mean ± std) in microseconds for the asymptomatic and the symptomatic subjects. The rMsE, nrMsE, MaE, and the MarE were for all videos 125.41 ± 11.77 µs, 121.16 ± 10.38 µs, 119.06 ± 11.96 µs, and 5.43 ± 0.95 µs, respectively. The t-test test gave statistically significant differences between the asymptomatic and symptomatic cases for all evaluation metrics tabulated in Table III. Table IV illustrates the results of the cdc, cdd, and the %cWd for the asymptomatic and the symptomatic subjects (mean ± std). The cdc, cdd, and the %cWd for the asymptomatic and the symptomatic groups were 5.13 ± 0.34 and 5.48 ± 0.29 mm, 5.89 ± 0.31 and 6.27 ± 0.35 mm, and 10.2 ± 1.03 and 16.09 ± 0.83%, respectively. The t-test, shown in the last row of Table IV, performed between the values obtained for the asymptomatic and symptomatic cases at p < 0.05 gave no statistically significant differences between them.

D. Strain Metrics
Table V presents the strain indices (mean ± std) over two consecutive cardiac cycles for the asymptomatic and symptomatic groups. More specifically, the following measures were evaluated, a radial strain at wall (rsW) of 5.13 ± 0.34%, a longitudinal strain of 2.84 ± 0.65%, a shear strain at wall (ssW) of 0.42 ± 0.12 rad, a shear strain at plaque (ssP) of 0.06 ± 0.01 rad, and a radial strain at plaque (rsP) of 3.13 ± 1.21 rad. all strain indices between the asymptomatic and symptomatic subjects were found to be not significantly different.

IV. discussion
The results of the proposed integrated video segmentation method can be favorably compared with the results of the segmentation of the carotid plaque in ultrasound images presented in [6], and also with the results presented in [7] and [22] (see also Table II). comparing the proposed video segmentation method with the one presented in [6], one may observe that the TnF, TPF, KI, and overlap index are larger in this study. This may be attributed to the fact that the videos used here were of better quality than the images used in [6]. The results reported in [22] are better than the ones reported in [6] and in this study, but it should also be taken into consideration that the number of images used in [22] was very small (only 4 ultrasound images of the cca) when compared with the study in [6] and [7], in which 80 and 33 videos were used, respectively. Furthermore, in [7], where video segmentation of atherosclerotic carotid plaque in 33 video sequences was proposed, similar results were obtained for the KI and overlap index (see also Table II). also, the specificity was 94.1 ± 4.2% in [7] versus 84.3 ± 7.5% here. It should also be noted that the standard deviations obtained for the presented work are larger than the ones obtained in [7].
The results of Table III show that all error measurements (rMsE, nrMsE, MaE, and MarE) calculated in this study are much smaller, with much smaller standard deviations, than the ones obtained in a recent study [32] in which M-mode state based identification in 10 cca videos was investigated. The smaller deviations obtained in this study could be attributed to the fact that a heterogeneous group of videos was investigated in [32]. The results of Table IV are also consistent with [32] (cdc = 5.26 ± 0.52 mm, cdd = 5.81 ± 0.59 mm, %cWd = 10.32 ± 4.71%), in which M-mode state-based identification in 10 ultrasound videos of the cca was performed.
strain indices, similar to the ones reported in Table  V, were also estimated for asymptomatic and symptomatic subjects in [34] (rsW = 5.50 ± 2.90%, ls = 2.15 ± 1.36%, ssW = 0.37 ± 0.20 rad, ssP = 0.10 ± 0.07 rad, and rsr = 4.32 ± 2.34 rad for the asymptomatic and rsW = 5.09 ± 1.73%, ls = 2.79 ± 1.71%, ssW = 0.39 ± 0.18 rad, ssP = 0.07 ± 0.04 rad, and rsr = 3.33 ± 1.82 rad for the symptomatic subjects) to quantify the mechanical behavior of the cca artery in the radial and longitudinal directions. More specifically, four different motion estimation techniques of the cca in ultrasound videos were investigated and applied on synthetic videos with known properties. It was also shown in [34], as can be seen by the presented results, that the errors were higher in the longitudinal direction than in the radial. no statistically significant differences were found for all indices between the asymptomatic and symptomatic subjects in [34] or in the present study.
In [21] and [22], in which image segmentation of the atherosclerotic carotid plaque was performed, images were neither normalized nor despeckled, nor was an automatic contour estimation used as in this study. The video normalization method presented in this study ensures that the segmentation method and the results are not dependent on the equipment used. This is not the case for the video segmentation method proposed in [7], in which the region-based segmentation was established using the statistical distribution of the gray level of the frames and speckle noise was not taken into consideration. Furthermore, the results of this study showed that the proposed video segmentation method performs very closely to the method for atherosclerotic plaque segmentation presented in [6]. The presented work can also be favorably compared with the recent video segmentation study in [7], performed on 33 subjects (see also Table II).

A. Plaque Contour Initialization
The key point of the proposed framework is the accuracy of the initialization. Two different methods were investigated in this work for initial contour estimation. However, both methods gave similar segmentation evaluation performance. There was no significant difference for all evaluation metrics investigated. In [7], the method for video segmentation proposed was also semiautomatic, in the sense that it required an initial manual segmentation of the plaque in the first frame, and then proceeded to the segmentation of the entire sequence without further user interaction. In [21] and [22], the initialization of the segmentation algorithm was performed automatically based on the echogenicity of the lumen and the plaque, or the Hough transform, respectively. These methods were tested on image sequences, but were processed individually. In the presented methodology it was assumed that if the method performed satisfactorily for an individual image, it might also perform well for a whole sequence of images, with each image processed individually. However, a more sophisticated tracking procedure, taking into consideration the interrelations between neighboring frames by utilization of motion estimation, could be taken into consideration in future work. such a method could achieve automated video segmentation with even less manual intervention.
Initial contour estimation for segmenting ultrasound images was also proposed in [35], derived from the polar image by combining information extracted from the probability function of the contour position, and more specifically from the function maximum location and the first zero crossing of its derivative. Then, starting from the initial contour, a region of interest was automatically selected and the process iterated until the snake contour 3.13 ± 1.21 3.9 ± 0.98 ns (0.59) rsW = radial strain at wall; ls = longitudinal strain at wall; ssW = shear strain at wall; ssP = shear strain at plaque; rsP = radial strain at plaque. 1 Test carried out at p < 0.05, ns = not significantly different. evolution could be ignored. In [36], a cost function was calculated and used as an input to the image plaque segmentation algorithm. In [19], a dynamic balloon model [18] represented by a triangular mesh was applied for detecting the plaque borders on two 3-d ultrasound carotid images for which the initial contour was placed manually. In all of these studies and in the present study, the significance of the initial contour placement was not exhaustively investigated to estimate how this influences the final segmentation result.

B. Video States Identification
By identifying the states of the cca, the normal and abnormal plaque motion can be investigated. It was shown in [7] and [31], and in the present study, that M-mode state-based modeling derived from B-mode videos can be used successfully to derive the carotid states and assess the corresponding wall changes. This work also shows that state-based video modeling can be used to identify video segments' dynamic behavior in ultrasound videos of the cca. However, further work is needed for validating the proposed method and for differentiation between normal and abnormal state-based plaque motion analysis.
There are several studies reported in the literature, in which state-based identification of the cca was performed. More specifically, Golemati et al. [22], used blockmatching-based techniques to estimate arterial motion from B-mode cca ultrasound images. They found that arterial wall distensibility in the radial direction was significantly higher than distensibility in the longitudinal direction (10.2 ± 4.5% versus 2.5 ± 0.89%). In another study [34], the same group performed a comparison of block-matching differential methods for motion analysis of the cca in ultrasound videos. Their results were very similar with the ones presented here (see Table V), but a slightly different distensibility of 13.15 ± 0.93% in the radial direction (see Table IV). differences could be easily attributed to the nature of the different video data sets investigated.
In [37], it was shown that M-mode ultrasound analysis can be a sensitive tool for measurements of strain decrease with increasing age and male sex and strain increase with smoking and obesity when evaluated on the right cca. Therefore, it might be used for screening of unsuccessful vascular aging and still potentially reversible subclinical carotid atherosclerosis, thereby lowering the cut-off value of vascular damage toward the values of unaffected population. It provides opportunities for introduction of different therapeutic or life-style change strategies for reduction or retardation of overt clinical manifestation of cerebrovascular disease.
In [31], the states of echocardiogram videos were identified based on a state classifier detector, which first detects the view boundaries using histogram-based comparison and edge change ratio. The accuracy of the classifier was 97.19%, which was higher than two existing approaches [38], [39]. Misclassification error of state detection was less than 13%, which is reasonably low.
Future research perspectives may be orientated toward to the development of new M-mode identification algorithms, and motion field of the arterial wall. Further, the separation between healthy and diseased arterial wall remains to be studied in detail.

C. Limitations of the Video Segmentation Method
There are some limitations in the proposed methodology that are summarized below. cases of plaque type I and type V [6], [25], [26] (see section II-B) were not considered for segmentation. If the plaque is of type I, borders are not clearly visible. Plaques of type V produce acoustic shadowing and the plaque is also not clearly visible [25]. However, there are also inherent difficulties for segmentation of the carotid videos and the corresponding plaques that arise not only from the use of different protocols but also because of differences in anatomy as well as the extent of the atherosclerosis and plaque presence. The presence of extensive and severe carotid stenosis may cause the initialization to completely fail because there will be difficulty in establishing the lumen. In addition, the presence of extensive plaques in the near wall may cause inaccurate segmentation of the far-wall adventitia. It should be noted that the parameters for each processing step (for example, the size of the moving pixel window, the number of iterations, the number of frames for which the contour was reinitialized) were selected for maximum performance. However, there were about 10% of the cases where the final snake contour did not converge properly.
The average computation time for the snake contour was about 9 s per frame, using a Pentium III (Intel corp., santa clara, ca) desktop computer with 3.2 GHz of raM memory, whereas in [7] it was 30 s per frame. It is noted that the expert clinician required more than 25 s per frame for the manual delineation. an average processing time of 20 s per image in case of the segmentation of the cca images is reported in [21, p. 1267, end of section II], whereas in [7], the average processing time per image was 30 s. It should also be noted that the accuracy of the proposed method has been further improved (see last two rows of Table II) when the first frame of the video was initialized with the second initialization method proposed in this study (see section II-E).
Further efficiency can be achieved in a commercial version, in which the proposed algorithm may be designed to perform in parallel with the clinician's evaluation, so the clinician might carry on with the patient examination while the segmentation of the whole sequence is performed. Thus, there is a reasonable expectation that a clinically applicable version will be created in the future. Further improvement could be achieved to enable the use of the proposed video segmentation technique in the real clinical practice if the initialization of the snake contours (see section II-E) is applied every 30 or 40 frames so that the snake contour is repositioned closer to the plaque borders.
In everyday clinical practice, the ultrasonographer manipulates the transducer and mentally transforms the 2-d images into anatomical volume, or structure, or lesion, to make a diagnosis. Three-dimensional ultrasound attempts to provide the ultrasonographer with a more realistic reconstruction and visualization of the 3-d structure under investigation, thus overcoming the limitations of 2-d imaging, reducing the variability of assessment and aiding in improved diagnosis. The additional information may also enable evaluation of changes in the volume but also in the surface morphology, providing additional information for stroke risk analysis [23]. The use of 3-d techniques has primarily focused on measuring volume changes through time to monitor disease progression [40]- [42]. landry et al. [40] demonstrated that plaque volume change can be reliably measured using 3-d ultrasound. They showed that a 20% to 35% change can be measured with 95% confidence for plaques of volume <100 mm 3 . For larger plaques (volume >100 mm 3 ), with 95% confidence, they showed that we can measure finer changes of the order of 10% to 20%, respectively. chiu et al. [41] developed a 3-d segmentation method for measuring the combined thickness of the plaque, the intima, and media (vessel wall plus plaque thickness or VWT). The authors proposed the use of 3-d VWT and VWT-change maps for identifying disease progression in relations to disturbances of flow. The authors extended their work in [42], in which they measured VWT volume changes for assessing and monitoring carotid artery disease.

V. concluding remarks
The method presented in this study proposes an integrated system for the segmentation of atherosclerotic carotid plaque in longitudinal ultrasound image sequences. The method integrates video frame normalization, despeckling, and accurate segmentation using active contours. such a system can reduce the time required for the video analysis, and also the subjectivity that accompanies manual delineations and measurements. The method will be further evaluated on a larger number of ultrasound videos and with ground truth from multiple experts. Future work will focus on improving the segmentation procedure, such that it can satisfactorily process special plaque cases and difficult-to-segment videos, taking into consideration, for example, a statistical approach to initialize and teach the snake about the blood flow and plaque area [37], as was also investigated in [6]. Furthermore, the segmentation system proposed in this study will be incorporated into a computer-aided diagnostic system that supports the texture analysis of the segmented plaque, as documented in [43] and [44], providing an automated system for the early diagnosis and the assessment of the risk of stroke. It will also be of great interest to measure the transversal as well the longitudinal movements at the two arterial sides of the cca (left and right cca) to find differences between the two sides. another interesting application would be to investigate the longitudinal movement of the arterial wall, as in [45], but also apply the aforementioned technique at different arterial sides that are known to be susceptible to the atherosclerosis disease, such as the carotid bifurcation and proximal internal carotid artery. Furthermore, the integrated plaque video segmentation method proposed will greatly facilitate the automated plaque motion analysis [34], [46]. Finally, it can be seen that an optimized implementation based on parallelization could provide close to real-time video processing. appendix a supplementary Material on the Two segmentation Initialization Methods The first initialization method for establishing the positioning of the initial snake contour uses intensity information, morphological operations, and edge detection to find the initial outline of the plaque. The steps of the first initialization procedure are described in what follows and can also be seen in Fig. 1. To begin, the normalized and despeckled first video frame of the cardiac cycle is thresholded-the threshold is calculated from the despeckled grayscale image frame according to otsu's method and the corresponding normalization [26]. Pixels that have smaller intensity values than this threshold are set to zero, whereas pixels with larger intensity values are set to one, resulting in a binary image [see Fig. 1(c)]. The binary image is dilated [see Fig. 1(c)]. Here, a square window shape structuring element with a size of 9 × 9 pixels (0.53 × 0.53 mm) was applied to the binary image frame. This morphological operation is performed to close small gaps and form a continuous boundary. on the dilated area, erroneous small edges that might trap the snake must be removed. This is carried out by labeling connecting components in the image, where the number of connecting components was chosen to be eight. small segments that have an area less than 20 pixels (1.2 mm), and do not belong in the boundary are therefore removed [see Fig.  1(c)]. It should be noted that the choice of the square window structuring element and the number of connected components was chosen after several experiments made using the guidance of the expert neurologist (coauthor M. Pantziaris). next, the contour matrix of the remaining objects is extracted [see Fig. 1(d)], which should represent the lumen and the near and far carotid wall boundaries. The areas of interest in the image, where the plaque will be detected are manually selected [via the positioning of an roI in the first video frame, see Fig. 1(e)], and the contour matrix of the corresponding regions is extracted by locating points and their coordinates on the plaque borders (plaque-lumen boundary, contour). To form the initial plaque-adventitia boundary contour, the two far ends of the contour matrix are connected together. Interpolating B-splines are used to smooth the corresponding boundaries [see Fig. 1(f)] and these smoothed boundaries are sampled in 30 equal segments. The 30 sampled points are the ones used to initialize the active contours plaque segmentation. In a case in which there are multiple plaques in the cca [see Figs. 3(a)-3(d)] the selection of the plaque could be repeated manually by placing a different area of interest, and the procedure could be performed for each plaque independently.
The second initialization method for establishing the positioning of the initial snake contour is based on the use of active contours without edges [13] by chan and Vese and can be seen in Fig. 2. We use in Fig. 2 the same image as in Fig. 1(a) so that direct comparisons can be made between the two initialization methods. level-set methods [47] work in several spatial dimensions, but more importantly they can handle topological changes naturally. Using the level-set formulation of the active contours without edges by chan and Vese [13] (chan-Vese model), the regions corresponding to the lumen and the carotid wall (including the plaque) can be automatically segmented. It is also noted that the chan-Vese formulation can use the corresponding energy terms used in the snake formulation.
The chan-Vese model [13] corresponds to a regionbased level-set method which uses the Mumford-shah functional in the level-set framework for a piecewise constant representation of an image. The evolution of the curve is governed by properties of the region of the image u 0 (x, y) enclosed by the curve. The model tries to separate the image into regions based on pixel intensities and introduces the following energy functional: where μ ≥ 0, ν ≥ 0, and λ 1 , λ 2 > 0 are fixed parameters. Without loss of generality, the initialization contour is a disk with radius of 2.1 mm randomly placed in the image background (segmented from simple thresholding with a threshold close to zero). For the evaluation of the levelset segmentation, in addition to downsampling the original ultrasound image frame by a factor of 4 to accelerate the segmentation, the values of the different parameters used are μ = 0.2; h, the space step, is set to 1; and Δt, the time step, is set to 0.25. For computation of the regularized versions of H and δ 0 , ε = 1, as used in [13]. The values for the different parameters for the chan-Vese model were empirically verified. The empirical verification also showed that slight variation in the value of the parameters would not have a significant effect on the final segmentation. The first two terms in (a.1) control the regularity by penalizing the length and the enclosed area of C. The choice of μ = 0.2 allows the segmentation to accurately match the data, while still resulting in a relatively smooth curve. The term ν provides a penalty on the foreground area resulting from the segmentation, and here is set to 0. For segmenting the lumen and the adventitia, the energy contributed by the intensity variance of the two regions is equally weighted to λ 1 = λ 2 = 1. In a large number of applications [13], including this implementation, λ 1 = λ 2 = 1 and ν = 0; however, parameters of the energy functional may need to be modified if new smoothness constraints are needed or if data sets with different visual quality are used [49].
The chan-Vese model is first applied to the normalized, despeckled first video frame. To initialize the active contours without edges, a rough outline of the lumen is used, resulting in faster convergence. This outline is found by incorporating clinical and image intensity information. according to the clinical guidelines, the lumen should be imaged longitudinally around the middle of the frame, and following normalization it has a limited range of intensity values. Thus, after thresholding the despekcled, normalized first video frame, for intensities less than 4 (which is below the maximum of the lumen intensity value after normalization [25]) and morphological opening with a circular element of 0.3 × 0.3 mm to close any holes, the largest region in the middle of the frame corresponds to a roughly segmented lumen [see Fig. 2(b)]. The outline of this rough segmentation is used to initialize the chan-Vese model. Figs. 2(c) and 2(d) present the chan-Vese model segmentation, showing only the outline at the far and near wall including the plaque of the cca, respectively. The final result of the chan-Vese model is shown in Fig. 2(e). next, the region corresponding to the lumen is eroded using a circular element of 0.82 × 0.82 mm to ensure that the boundaries that will be used for initialising the final segmentation will be in the lumen but outside the carotid wall. This segmentation provides initialization of boundaries of the near and far wall of the carotid. These boundaries are sampled and 30 equally spaced sampled points are used to initialize the active contours for atherosclerotic carotid plaque segmentation [see Fig. 2(e)]. The boundaries area across the entire length of the wall will include all plaques present. references