Quality evaluation of ultrasound imaging in the carotid artery

Image quality assessment plays an important role in various image-processing applications. In this work, we evaluate image quality criteria based on mean square error, signal to noise error, and quality and structural similarity indices on ultrasound imaging of the carotid artery. These criteria as well as statistical and texture features were computed on 80 images recorded from two different ultrasound scanners before and after despeckle filtering, and after despeckle filtering and normalization. Results showed that image quality was improved after despeckle filtering and normalization for both scanners. This finding is also in agreement with the optical perception evaluation carried out by two vascular experts.

INTRODUCTION Ultrasound imaging is a powerful non-invasive diagnostic tool in medicine [1], [2]. However, like all medical imaging modalities, ultrasound is subject to a number of artifacts that degrade image quality and compromise diagnostic confidence [1]. For medical images, quality can be objectively defined in terms of performance in clinically relevant tasks such as lesion detection and classification [3]. Most studies today have assessed the equipment performance by testing diagnostic performance of multiple observers, which also suffers from intra-and inter-observer variability. Although this is the most important method of assessing the results of image degradation, few studies have attempted to perform physical measurements of degradation [4]. The aim is to measure the amount of degradation within a system, using a number of methods, in order to locate the most useful indicators of loss of image quality. A good quality index for the comparison of two images should be: a) extracted form structural information, b) calculated on a small region such that small variations can be detected, and c) based on the regional gray level distribution.
The major performance-limiting factor in visual lesion detection in ultrasound imaging is a multiplicative noise called speckle, that makes the signal or lesion difficult to detect [2]- [4]. In a recent study [2], we have shown that de-speckle filtering improves the optical perception of the physician in the assessment of ultrasound imaging of the carotid artery. The objective of this study is to evaluate the image quality of ultrasound imaging of the carotid artery on two different ultrasound scanners before and, after de-speckle filtering, and after despeckle filtering and image normalization, by statistical and texture analysis, by optical perception evaluation, as well as by image quality metrics.
In the following Section, theoretical concepts on image quality are introduced. In Section III. A and Section III. B, the two different ultrasound scanners, and the de-speckle filter used are presented respectively. In Section III. C image normalization is presented, whereas in Section III.-D and Section III. E, statistical and texture analysis and the evaluation metrics are presented respectively. Sections IV and V present the results and concluding remarks.
II. IMAGE QUALITY In order to be able to design accurate and reliable quality metrics, it is necessary to understand what quality means to the observer. An observer's satisfaction when watching an image depends on many factors. One of the most important is of course image content and material. Research made in the area of image quality showed, that this depends on many parameters, such as: viewing distance, display size, resolution, brightness, contrast, sharpness, colorfulness, naturalness, and other factors [5].
It is also important to note that there is often a difference between fidelity (the accurate reproduction of the original on the display) and perceived quality. Sharp images with high contrast are usually more appealing to the average observer. Likewise, subjects prefer slightly more colorful and saturated images despite realizing that they look somewhat unnatural [6]. For studying visual quality some of the definitions above should be related to the human-visual system. Unfortunately, subjective quality may not be described by an exact figure, due to its inherent subjectivity, it can only be described statistically. Even in psychological threshold experiments, where the task of the observer is to give a yes or no answer, there exists a significant variation between observer's contrast sensitivity functions and other critical low-level visual parameters. When speckle noise is apparent in the image, the observer's differing experiences with noise are bound to lead to different weightings of the artifact [5]. Researchers showed that experts and non-experts, with respect to image quality, examine different critical image characteristics to form their final opinion [7].
In the light of the above difficulties, testing procedures for subjective quality assessment are discussed in the next Section.

A. Ultrasound Imaging Scanners
Two different ultrasound scanners were used in our study, the ATL HDI-3000 and the ATL HDI-5000.
The ATL HDI-3000 ultrasound scanner is equipped with 64 elements fine pitch high-resolution, 38 mm broadband array, a multi element ultrasound scan head with an operating frequency range of 4-7 MHz, an acoustic aperture of 10x8 mm and a transmission focal range of 0.8-11 cm [8].
The ATL HDI-5000 ultrasound scanner is equipped with 256-element fine pitch high-resolution 50 mm linear array, a multi element ultrasound scan head with an extended operating frequency range of 5-12 MHz and real spatial compound imaging. The scanner increases the image clarity using SonoCT imaging by enhancing the resolution and borders, and interface margins are better displayed. Several tests made by the manufacturer [8] showed that, the ATL HDI-5000 scanner was overally superior to conventional 2D imaging, primarily because of the reduction of speckle, contrast resolution, tissue differentiation, and the image was optically better.
In this study 80 ultrasound images of the carotid artery were acquired from each scanner. The images were recorded digitally on a magneto optical drive with a resolution of 768x756 pixels with 256 gray levels.

B. Despeckling
The working principle of the de-speckle filter lsmv used in this study, which was applied six times iteratively on each image, may be described by a weighted average calculation using sub region statistics to estimate statistical measures over pixel windows (typically 5x5, 7x7, or 9x9 sliding pixel windows). It assumes that the speckle noise model has the following multiplicative form [2] where j i g , represents the noise pixel in the middle of the moving window, j i f , represents the noise-free pixel and j i n , is a Rayleigh distributed noise on pixel. Hence the algorithms in this class may be traced back to the following equation [2], [9] ) ( where j i f , is the new estimated pixel, , and j i, the absolute pixel coordinates. The factor j i k , is a function of the local statistics in a moving window. It is found in the literature and is derived as [9] ) /(

C. Image Normalization
The images were normalized manually by linearly adjusting the image so that the median gray level value of the blood was 15-20, and the median gray level of the adventitia (artery wall) was 180-200. The scale of the gray level of the images ranged form 0-255 [10]. This normalization using blood and adventitia as reference points was necessary in order to extract comparable measurements in case of processing images obtained by different operators or different equipment [10].

D. Statistical and Texture Analysis
Texture contains important information, which is used by humans for the interpretation and analysis of many types of images. It may provide useful information about object characterization in ultrasound images [11]. The following statistical and texture features (SF) [11], were extracted from the original, and the processed images to evaluate quantitatively the performance of each scanner:

E. Image Quality and Evaluation Metrics
Differences between images were evaluated using the following image quality and evaluation metrics, which were used as statistical measures, between the original ( which measures the quality change between the original and processed image [12]. b) The randomized mean square error, which is the square root of the squared error averaged over the MxN array [13] c) The error summation in the form of the Minkowski metric, which is the norm of the dissimilarity between two images, as follows [4], [14] β β d) The geometric average error ( GAE ) is a measure, which shows if the transformed image is very bad [15], and it is used to replace or complete RMSE . It is positive only if every pixel value is different between the original and the transformed image. The GAE is zero, if there is a very good transformation between the original and the transformed image, and high if the transformation with the original is extremely bad. This measure is also used for tele-ultrasound, when transmitting US images and is defined as e) The signal-to-noise radio [16] ∑∑ . (8) f) The peak signal to noise radio [16] 2 10 log 10 s where s , is the maximum pixel value in the image.
The PSNR is higher for a better-transformed image and lower for a poorly transformed image. It measures image fidelity that is how closely an image (despeckled) resembles usually the corrupted original. g) The mathematically defined universal quality index [4] models any distortion as a combination of three different factors, which are: 1) loss of correlation, 2) luminance distortion, and 3) contrast distortion. Its highest value is 1 for , while the lowest value is -1 and is defined as where fg σ represents the covariance between images and g f σ σ , represents the standard deviations of the pixel values inside the window. It is calculated for a window size 8x8 using a sliding window without overlapping.
h) The structural similarity index between two images [14], which is a generalization of (10) ) )( ( ) 2 )( 2 ( i) The optical perception evaluation, in our study, was carried out according to the ITU-R recommendations with the Double Stimulus Continuous Quality Scale (DSCQS) procedure [15]. Two vascular experts were used, an angiologist and a neurovascular specialist. They evaluated 80 ultrasound images recorded from each ultrasound scanner before and after despeckle filtering, after image normalization, and after despeckling and normalization. They assigned a score in the one to five scale based on subjective criteria by inspecting the area around the distal common carotid artery, 2-3 cm before the bifurcation, as the objective of the experts was to visualize the intima media complex. The experts were allowed to give equal scores to more than one image in each case.
IV. RESULTS Figure 1 illustrates the original, despeckled, normalized, and normalized despeckled images of the two ultrasound scanners. It is shown that the images for the ATL HDI-3000 scanner have greater speckle noise compared to the ATL HDI-5000 images. Moreover the lumen borders and the intima-media thickness are more easily identified in the ATL HDI-5000 normalized and normalized despeckled images. Table I shows the results of the statistical and texture features for the 80 images recorded from each image scanner. As shown the effect of despeckle filtering for both scanners was similar, that is the speckle index was reduced, the mean and the median were preserved, the Original 3000 Original 5000 Despeckled 3000 Despeckled 5000 Normalized 3000 Normalized 5000 Normalized despeckled 3000 Normalized despeckled 5000 Figure 1. Original, despeckled, normalized, and normalized despeckled images of the ATL HDI-3000, and ATL HDI-5000 images shown in the left and right columns respectively. skewness and the kurtosis were reduced, thus making the image histogram more symmetric and less flattened. The entropy was increased and the contrast was reduced. The ASM was reduced for the despeckled images for both scanners and for the normalized despeckled images for the HDI-5000 scanner. No statistically significant difference was found for all features in Table I Table II shows the results, in percentage (%) format for the optical perception evaluation made by the two vascular experts. It is clearly shown that the highest scores were obtained for the normalized despeckled images for both scanners from both experts. Table III tabulates the image quality evaluation metrics, presented in Section III. E, for the 80-ultrasound images recorded from each image scanner, between the O-D, the O-N, and the O-ND images. Best values were obtained for the ATL HDI-3000 scanner for the O-D images with lower MSE, RMSE, and Err3, and higher PSNR. The GAE was 0.00 for all cases, and this can be attributed to the fact that the information between the original and the processed images, remains unchanged. Best values for Q and SSIN were obtained for the O-N images for both scanners, whereas best values for SNR were obtained for the ATL HDI-3000 scanner on the O-N images. V. CONCLUDING REMARKS Image quality is very important in the assessment of atherosclerosis and the evaluation of the risk of stroke. In this work we have evaluated two different ultrasound scanners on 80 ultrasound images acquired from the carotid artery before, and after despeckle filtering, after normalization, and after despeckle filtering and normalization. We have evaluated the results based on statistical and texture features, on optical evaluation by two experts as well as based on image quality metrics. Table I shows that brighter images were recorded for the ATL HDI-5000. Moreover it is shown that the entropy that is a measure of the information content of the image is higher for the ATL HDI-5000 in the cases of the D, N, and ND images. The ASM that is a measure of the inhomogeneity of the image is lower for the ATL HDI-5000 in the cases of the D and ND images. Despeckle filtering reduced in both scanners the speckle index, the mean and the median were preserved, the skewness and the kurtosis were reduced, thus making the image histogram more symmetric and less flattened.
The optical perception evaluation given in Table II, showed that the ND images were rated better than the O, D, and N images by both experts.
In previous studies [3]- [7], [14], [15] researchers evaluated image quality on real world images using either only the visual perception by experts or some of the evaluation metrics presented in Table III. The usefulness of these measures was not investigated on ultrasound images.
It is shown from Table III, that the effect of despeckle filtering is more obvious on the ATL HDI-3000 scanner, which shows that the ATL HDI-5000 scanner produces images with lower noise and distortion. Moreover, it is obvious that all quality metrics presented here are equally important for image quality evaluation.
Concluding, image quality was improved after normalization and despeckle filtering for both scanners, as well as the ATL HDI-5000 produces images with higher quality.