Assessing skin tone heterogeneity under various light sources

In this paper, skin tone heterogeneity in five facial areas (forehead, right cheekbone, left cheekbone, nose tip and chin) was investigated under six light sources with correlated color temperature (CCT) of 2850 K, 3500 K, 5000 K, 5500 K, 6500 K and 9000 K. Firstly, a facial image capturing protocol was developed and applied to five female participants, and their facial skin tone was analyzed based on the captured images. Through color characterization of the camera, XYZ values in each facial area were converted by a matrix from the extracted RGB data and then transformed to CAM02-UCS color space. MCDM with CAM02-UCS color difference was used to quantify skin tone heterogeneity in each facial area. The results under different light sources indicated that larger heterogeneity exists under the light source with lower CCT, and when the CCT of the light source ranges from 5000 K to 9000 K, there was smaller skin tone heterogeneity in each facial area.


Introduction
The color of skin is probably one of the colors that we see most often in our daily lives and it plays an important role in many multidisciplinary applications. For example, skin color directly connects with the visual perception of facial attractiveness as well as age and therefore it is important in the cosmetics industry [1][2]. In medical applications, facial skin color is related to the perception of health. Different skin coloration in facial areas can be used to judge the physical status of a person as healthy or unhealthy, particularly in the periorbital, cheek, and forehead areas [3]. In clinical utilization, skin color information from images can be a useful tool for diagnostics and surveillance when cross-or parallelpolarization is applied to photography [4]. In the mobile phone industry, different filters are introduced in the cameras to enhance facial skin color reproduction, especially for portraits.
The measurement of skin color and its appearance had been investigated in previous studies. Xiao et al. [5] conducted a study to measure the skin color of a large numbers of subjects (four ethnic groups and four body locations) using a spectrophotometer. All skin data were compared in CIELAB uniform color space. Results showed that skin color distributions for the four ethnic groups overlap significantly and major differences exist along the yellowness dimension between ethnic groups. Using average skin spectral reflectance of Caucasian and Chinese subjects, Chauhan et al. [6] investigated just noticeable color difference of facial image under different illumination and concluded facial image in CWF has a larger discrimination threshold than the face under D65. Melgosa et al. [7] conducted a study to investigate how facial contrast is affected by different white light-emitting diode (LED) sources and CIE recommended traditional illuminants. It was found that both the body area differences and the discrepancies of color difference between Caucasian and Oriental subjects were small for the 18 illuminants tested.
Skin is a complicated multi-layer material with a non-flat, uneven and spatially non-uniform surface. Color heterogeneity is one of the key proprieties of human skin and it affects overall skin color appearance significantly. The perception of color heterogeneity is contributed by skin texture and spectral reflectance. It can also be largely affected by the illumination. It is thus desirable to quantify skin color heterogeneity for many applications. However, conventional color measurement techniques often fail to predict skin color heterogeneity since skin pores are too small to measure. It is still unknown how illumination and skin texture affect the perception of skin color heterogeneity.
The aim of this study is to develop a method to objectively quantity skin color heterogeneity using color difference units. We specially focus on an investigation of the effect of illumination and the texture on skin color heterogeneity.

Methodology
In this study, to quantify skin color heterogeneity, an image measurement method, as developed in our previous study [8], was used to predict skin color from each pixel of the camera images. Firstly, a protocol for capturing facial images using a RGB DSLR camera was developed. In order to investigate the effect of different light sources, a facial image of the same subject was captured using six different light sources respectively, each with a different Correlated Color Temperature (CCT). Then, a camera color characterization was performed that was specially designed for human skin color. Subsequently, the color of each pixel of the facial image was converted to CIE XYZ tristimulus values from raw camera RGB values. Finally, CAM02 UCS was used to transform each facial color to human perceptual attributes and quantify skin color heterogeneity by the mean color difference from the mean (MCDM) for a small skin area. The details of each step are described in below.

Facial image acquisition
Six light sources with different CCT (2850 K, 3500 K, 5000 K, 5500 K, 6500 K and 9000 K) were chosen using two THOUSLITE LED Cubes. The relative spectral power distributions (SPDs) of the six light sources are shown in Figure  1. A Canon EOS 6D Mark II camera was used to capture the facial images under each of the different light sources. Additionally, in order to obtain uniform light and reduce the effect of highlights in the images [4], a linear polarizer was placed in front of each LED Cube with a further polarizer placed over the lens of the camera. The positions of two LED Cubes were aligned such that they directly illuminated each face as in Figure 2.
The facial color images were captured against a black background. Each participant sat in front of the camera at a distance about 120 cm and their faces were fixed in the same position without any makeup on their faces. RAW images without post-processing were obtained. The camera settings were 1/8 second exposure time, f/5.6 aperture and 640 ISO. In this study, five female participants from different ethnic groups were involved: one each of Indonesian, Caucasian, Chinese, Mexican and African. All the participants were young women with ages in the range of 20-30 years old.   and chin (CH), were located in each captured image with a size of 50 * 50 pixels and RGB data from the three channels in each pixel were obtained using MATLAB software.

Camera color characterization
Camera color characterization was performed to transform camera raw RGB values to CIE XYZ tristimulus values for each facial image under each light source. To get better predictive accuracy, a Spectromatch Skin Color Chart (Figure 4) was used as the training samples which included 95 color patches representing different skin colors. These charts were made using silicone material to give large similarity to human skin. The Skin Color Chart was captured by the camera with the same settings above under each light source and the raw RGB data for each color sample were extracted from the captured images in MATLAB. A Konica Minolta CM-2600d spectrophotometer with 3 mm aperture size was used to measure the spectral reflectance of each color patch and the XYZ values were then computed from the measured spectral reflectance and SPD of each light source. In order to establish a matrix to convert RGB values to XYZ values, polynomial regressions based on third-order polynomials [9] were then performed for each light source.
The skin data of the five participants were used as the test data for camera color characterization. For each light source, CIE XYZ values of each participant in each facial area were predicted using the camera characterization model. CAM02-UCS was then used to predict the color difference between the model predicted CIEXYZ values and the corresponding CIE XYZ values calculated from the spectrophotometer measurement. The average results for each illuminant are shown in Table 1. The predictive error ranges from 2.03 to 2.44 DE under the six light sources and these results suggest that it is reasonable to predict colorimetric values of human skin using this method as also shown in a previous study [8].

Color difference prediction
In this study, CAM02-UCS [10] was used to predict the color difference on the face of each participant under different light sources. For each skin color under each light source, the CIE XYZ tristimulus values were converted to CAM02 parameters J', a' and b'. The color difference between the two sets of skin color were represented by DE using Equation (1).

E
(1) where J', a' and b' are the difference of the J', a' and b' values between the two sets of skin color. KL is a lightness parameter which is equal to 1 in CAM02-UCS.
Comparing with CIELAB uniform color space, CAM02 UCS has a better model of chromatic adaptation and transforms skin color under a specific illumination to the equivalent under an equal-energy white condition (illuminant E). Based on the color characterization of the camera, XYZ values of each facial area/pixel were obtained using the derived matrix for each light source. The XYZ values were then transformed to CAM02-UCS color space and J'a'b' values were obtained.

Skin color heterogeneity
Skin color heterogeneity is an indicator of the color spatial distributions in a given facial area. Using CAM02 UCS, it is represented by the Mean Color Difference from the Mean (MCDM), as defined by Equation (2). The larger the MCDM values, the greater the skin color heterogeneity. (2) where N is the number of pixels (N=50*50); Vi is a set of J'a'b' values for a given pixel i; Vave is a set of J'a'b' values averaged over all the pixels, fE is a function to calculate color differences. In this study, the CAM02-UCS color difference formula was used in the calculation of MCDM.

Results and Discussion
Skin color heterogeneity of five subjects in five facial area under six illumination was calculated in MCDM in CAM02 UCS color space. The average results for each facial area under each light source are given in Table 2 Figure 5 illustrates the average color heterogeneity trend for each participant under each light source. As shown in Figure 5, under the light sources with the CCT ranges from 2850 K to 5000 K, the values decrease with an increase in the CCT of the light source. Under light sources with the CCT of 5000 K to 9000 K, there were slight changes in skin color heterogeneity. Thus it can be concluded that larger skin color heterogeneity exists under the light sources with lower CCT and when the CCT of the light source is higher than 5000 K, the color heterogeneity differs little with the change of CCT.
From the results obtained, it appears that larger skin color heterogeneity exists under the light source with a lower CCT, considering skin color is reddish as well as yellowish and the light sources with lower CCT appears warmer (reddish), maybe the color variation is magnified under the light source with similar color appearance. In order to further investigate the effect of the CCT of light source on skin color heterogeneity, especially the skin color distribution, taking the area of the chin as an example, each color of 2500 pixels under six light sources was drawn in J'a'b' color space. Their distributions on a'b' and J'M' planes and the skin color boundaries under each light sources were illustrated in Figure 6. The color of red, green, blue, cyan, magenta and yellow represent the results under the light source with CCT of 2850 K, 3500 K, 5000 K, 5500 K, 6500 K and 9000 K, correspondingly. From the color distributions of the chin area on an a'b' plane in Figure 6, it can be clearly seen that the skin color boundary under the 2850 K light source (the red line) is the largest among the six light sources, followed by the boundary under the 3500 K light source (the green line); the skin color boundaries under the other four light sources are much smaller. This trend is similar to the results shown in Table 2. It also indicates that lower color temperatures result in larger color gamut. With the increasing CCT of the light sources, the color distribution of the chin area shifts to the -a' axis and -b' axis, which means the colors become less reddish and less yellowish. From the color distributions of the chin area on the J'M' plane in Figure 6, it is shown that the changes of lightness (J') under the six light sources has little change while the colorfulness (M') decrease with increasing CCT. But actually, the variation of three attributes has a large range, the reason maybe the influence of skin texture, pores or fine hair in facial areas.  Table 3. The chin color gamut is largest under 2850 K illuminant and then followed by the results under 3500 K illuminant. Besides, the chin color gamut on the a'b' plane is larger than that on the J'M' plane by a factor of 2.0/1.5 under the different light sources, which indicates that the lightness changes less than other chromatic attributes with the change of CCT. Furthermore, the chin color gamut of the African subject is relatively larger than the others. It seems to be caused by the ethnicity, the genes and the biophysical properties of the different ethnic groups [11].

Conclusions and future work
Skin color heterogeneity can be assessed using skin image measurement and color difference analysis. From the analysis of the captured facial color images under six light sources with different CCT, we have reported on the skin color heterogeneity in terms of CAM02-UCS color difference. The MCDM values were computed for each facial area under each light sources, with the following results: (1) the skin color heterogeneity decreases with the increase in CCT when the light source has a relatively low CCT; (2) there is little change in skin color heterogeneity when the CCT ranges from 5000 K to 9000 K; (3) when different CCT of illuminants were used, for same skin area, color gamut become larger under the low CCT than the high CCT.
The ethnicity also has an influence on skin color heterogeneity of facial area, but due to the limited number of participants, more female as well as male subjects from different ethnic groups will be involved and the effect of facial size on color heterogeneity will be also investigated in future work.