An Efficient Metric for Evaluating the Quality ofColor Image Enhancement

M. C Hanumantharaju, M. Ravishankar, D. R Rameshbabu,V. N Manjunath Aradhya

Department of Information Science and Engineering,Dayananda Sagar College of Engineering,

Bangalore, India{mchanumantharaju,ravishankarmcn,bobrammysore,aradhya.mysore}@gmail.com

Abstract. This paper describes a new metric for evaluating the qualityof the color image enhancement. The image enhancement achieved byalgorithms such as histogram equalization, adaptive histogram equaliza-tion, homomorphic filtering and Multiscale Retinex with Color Restora-tion (MSRCR) etc., improves the contrast, luminance and dynamic rangecompression. However, these algorithms introduces halo artifacts, vio-lates the gray world assumption and edge details of an image is lost.A major concern is that there is no accurate objective criterion for as-sessing the performance of image enhancement algorithms. Researchersdeveloped some objective criteria such as Peak Signal to Noise Ratio(PSNR), contrast enhancement performance, luminance enhancementperformance and Structural Similarity Measurement (SSIM) etc. How-ever, they could not solve the problem satisfactorily, and it often usesvisual inspection and subjective criteria to evaluate the performance ofthese image enhancement algorithms. This paper proposes an objectiveand effective assessment criterion, Wavelet Energy (WE), to evaluate theperformance of the color image enhancement algorithms. The proposedscheme describes both textual and structural information changes in anenhanced image. Numerous experiments were conducted to demonstratethe superiority of the proposed metric over the existing metrics. Exper-imental results confirm that the WE is an efficient and useful metric forevaluating the quality of the color image enhancement.

Keywords: Objective Metric, Color Image Enhancement, Wavelet En-ergy, Peak Signal to Noise Ratio, Structural Similarity Measure.

1 Introduction

Image enhancement is one of the important preprocessing step in digital imageprocessing. Nowadays color image enhancement has become significant researcharea due to the widespread use of color images in many applications. The goal

of the color image enhancement is to improve the image details, contrast andsharpness for better visual quality. Numerous algorithms have been developed forcolor image enhancement which are applied to medical images, natural images,micro array images, and Synthetic Aperture Radar (SAR) images etc. Imageenhancement techniques may have significant impact on image quality and thusaffect human interpretation. The image quality metric [1] plays an importantrole in the design and evaluation of imaging and image processing systems. Inan image compression, image quality evaluation [2] algorithm can be used toevaluate the performance of different image compression algorithms that attemptto minimize the number of bits require to store an image, while maintainingsufficiently high image quality. Similarly, image quality assessment algorithmscan be used to evaluate the image acquisition and display systems.

Nowadays images and videos are frequently transported over optic fiber,packet switched networks such as internet, wireless systems etc. For the applica-tions such as video conferencing and video on demand the bandwidth efficiencycan be improved using image quality assessment techniques. Further, image qual-ity assessment algorithms can be used in optimized design of various componentsof an image processing systems. Finally the image quality measurement and hu-man vision system are closely related disciplines. Therefore, research on imageand video quality assessment may provide deep insights into the functioning ofHuman Visual System (HVS) [2]. The HVS model exhibit good correlation withthe perception of the human visual system. The HVS model is not only used byimage processing, video processing researchers but also computer vision expertsto deal with biological and psychological visual perception.

Image quality assessment methods [3] are broadly classified as subjective andobjective techniques. The first approach evaluates the image quality based onsubjective image quality testing however, observers evaluate the image quality.These approaches are time complex, expensive and have a very strict definitionof observational conditions. The second approach uses objective image qualitytesting technique based on mathematical computations. Further, the objectivebased image assessment method is divided into three category: Full Reference(FR) criteria, Reduced Reference (RR) criteria, and No Reference (NR) criteria.These criteria’s are the function of the distorted image. The FR criteria assumethat original image is free from distortion. The objective measures of referencetechniques used for image quality assessment are PSNR, SSIM [4], and the VisualInformation Fidelity (VIF). An RR criterion is fully generic, requires a partialknowledge of the reference image and is an elaborate model of HVS. The NRcriteria do not require any information about reference image.

The quantitative evaluation [5] of image enhancement methods focuses onseveral criteria such as edges, sharpness and details of an image. In addition,color of the processed image and its constancy through out the image is a sig-nificant factor for evaluating the quality of the image. Many traditional metrics[6] measure the image enhancement performance by comparing the distance be-tween the original and the processed images. However, these techniques are notcorrelated with the perceived quality well, and their limitations have been widely

recognized. The proposed wavelet energy overcomes the drawback of other meth-ods and provides both structural and textual information of an enhanced imagein a single stretch.

This paper is organized as follows: The most commonly used algorithms forcolor image quality assesment is presented in Section 2. The proposed imageenhancement quality measure is described in Section 3. Experimental resultsand analysis is presented in Section 4. Finally conclusion is provided in Section5.

2 Color Image Enhancement Quality Measures

In this section, some of the commonly used image quality assessment metricsused for color image enhancement is discussed.

2.1 Peak Signal to Noise Ratio

The most commonly used objective fidelity measure is the PSNR. The PSNR isestimated using Mean Squared Error (MSE) which is the arithmetic differencebetween enhanced and original images. It is defined as:

MSE =1

M ×N

M∑i=1

N∑j=1

[I(i, j)− I(i, j)

]2

(1)

where I(i, j) and I(i, j) are the original and enhanced images of size M ×Npixels respectively. It can be easily found that some enhanced image with thesame MSE may have different error severities, and some of them are much betterthan others, visually. The PSNR is expressed as:

PSNR = 10 log10

[L2

MSE

](2)

where L is the intensity levels and is equal to 255, for an unsigned 8-bit pixelrepresentation.

2.2 Contrast Enhancement Perfromance

The contrast enhancement performance [7] is an objective criterion used formeasuring the contrast of an image. The contrast enhancement is defined as:

C =σout − σin

σin(3)

where σout is the luminance variance of an enhanced image and σin is theluminance variance of an original image. The contrast enhancement of 0.088implies that the contrast improvement is 8.8 percent.

2.3 Luminance Enhancement Perfromance

The luminance enhancement performance [7] is an another objective criterionused for measuring the luminance of an enhanced image. The luminance en-hancement can be expressed as:

L =Iout − Iin

Iin(4)

where Iout is the luminance mean of an enhanced image and Iin is the lumi-nance mean of an original image. The luminance enhancement of 0.044 indicatesthat enhanced image has 4.4 percent luminance improvement.

2.4 Structural Similarity Measurement

A significant metric proposed recently for image quality assessment [8] is theSSIM measurement. The SSIM compares the structures of the reference anddistorted images. The structural similarity index [9] is defined as a function ofluminance l(x, y), contrast c(x, y) and structure comparison s(x, y).

SSIM(x, y) = [l(x, y)]α [c(x, y)]β [s(x, y)]γ (5)

where α, β, and γ are the parameters defining the relative importance of thethree components. Specifically, these are set to unity assuming that the threecomponents posses equal importance. These schemes are based on HVS, extractsstructural information of the image and consider point by point distortions thatmay not relevant to the perception quality. The basic SSIM index is a realnumber in the range [-1 1] and is calculated using the second order statistics ofthe referenced and distorted images:

SSIM(x, y) =(2µxµy + C1) + (2σxy + C2)(µ2

x + µ2y + C1)(δ2

x + δ2y + C2)

(6)

where x and y are two non-negative image signals µx and µy are the meanintensities, δ2

x and δ2y are the variances, σxy is the covariance of x and y. C1 and

C2 are small real constants to avoid instability. SSIM has introduced a way toinvestigate image fidelity [10] however, this metric also have some limitations.

2.5 Color Colorfullness and Color Naturalness

Kai-Qi-Huang et al. [2] proposed a natural color image enhancement and evalua-tion algorithm based on human visual system. In this scheme, enhanced image isevaluated using metrics such as Color Colorfulness Index (CCI) and Color Nat-uralness Index (CNI). The hue pixels in HSV space is divided into three classes:skin pixels (hue value of 25 to 70), grass pixels (95 to 135) and sky pixels (185to 260). The CNI values are computed using the expression:

CNI =nskin ∗Nskin + ngrass ∗Ngrass + nsky ∗Nsky

nskin + ngrass + nsky(7)

where nskin, ngrass, and nsky is the total number of skin, grass, and skypixels, Nskin, Ngrass, and Nsky represents local CNI values for skin, grass andsky pixels. The CNI value varies from 0 to 1. The CCI can be expressed as:

CCI = S + σ (8)

where S is the average saturation of the image and σ is the standard deviation.The CCI value of zero indicates that, the image is achromatic and maximumvalue of CCI indicates the most colorful image. The best range of CNI humanperception is close to unity and for CCI is about 16 to 20.

3 Proposed Method

The proposed method uses wavelet domain in order to evaluate the quality of thecolor image enhancement. The orthogonal wavelet transform preserves energysuch that sum of the normalized squared of the wavelet coefficients by levelplus the normalized squared of the final level approximation coefficients equalsthe squared normalized of the signal. In the proposed scheme, the energy ofthe enhanced image is analysed using both approximate and detailed waveletcoefficients. The approximate wavelet coefficients provides the information aboutglobal image enhancement, where as the detailed wavelet coefficients providesthe information on the image details. Finally, these enhanced image waveletcoefficients are compared with the original image wavelet coefficients and thanquality assesment of an image is done. The details of the proposed method isdiscussed in the following section.

3.1 Wavelet Energy

Wavelet Transform (WT) is a technique for analyzing the time frequency domainthat is most suited for non stationary signal. The importance of WT is used tocapture the localized features of the signal. A Continuous Wavelet Transform(CWT) maps a given function in time domain into two dimensional functionof s and t. The parameter s is called the scale and corresponds to frequency inFourier transform and t is the translation of the wavelet function. The CWT isdefined by

CWT (x, y) =1√s

∫S(T )ϕ

(T − t

s

)dt (9)

where S(T) is the signal and ϕ(T ) is the basic wavelet and ϕ(

T−ts

)1√s

isthe wavelet basis function. The Discrete Wavelet Transform (DWT) is normallyused for short time analysis. The DWT for a signal can be written as

DWT (m,n) =1

2m

N∑i=1

S(I, i)φ[2−m (i− n)

](10)

Wavelet energy is a method for finding wavelet energy for 1-D wavelet decom-position. The 1-D wavelet decomposition, wavelet energy provides percentage ofenergy corresponding to the approximation and the vector containing the per-centage of energy corresponding details. The Wavelet Energy is computed asfollows

WE =1

2−m/2

N∑i=1

S(I, i)φ[2−m (i− n)

](11)

In the proposed method, the WE is used as an effective and an efficient met-ric in order to evaluate the quality of the enhanced image. In most of the qualityassesment metrics, high frequency information of an image is not sensitive tolightning changes or more prominent features, to descriminate different objectsin an image. The WE is computed using a linear combination of high frequencycoefficients after a Daubechies wavelet transform. The foreground object withpoor contrast then the background of an image has continuous regular coefficientvalues, the Probability Density Function (PDF) for such image have relativelylower wavelet energy, lower skewness of the WE than a background. Thus thePDFs for such a poor contrast object is distinct when compared with the back-ground. The wavelet energy computed for enhanced image and original imageare compared and the following observations are made. Higher the detailed WEcoefficients, details of the enhanced image is better. Alternatively, the detailsof the enhanced image are better for large detailed WE coefficients. However,the approximate WE coefficients provides information about the enhancement.Higher the approximate WE coefficients, globally the enhanced image is better.

3.2 Comparing Wavelet Energy with the Exising Metrics

Wavelet energy metric is compared with the commonly used metrics such asPSNR, contrast enhancement performance, luminance enhancement perfromance,SSIM, CNI and CCI. The simplest and most widely used full reference qualitymetric are MSE and PSNR. The MSE is computed by averaging the squaredintensity differences of distorted and reference image pixels, along with the re-lated quantity of PSNR. MSE and PSNR are simple to calculate, have clearphysical meanings, and are mathematically easy to deal with for optimizationprocess (For instance MSE is differentiable). But they are not well matched toperceived visual quality [11]. In the last few years, a great deal of effort has goneinto the development of quality metric techniques that take advantage of knowncharacterstics of HVS.

The image quality evaluated using contrast and luminance measurements pro-vide contrast and luminance improvement with respect to original image. How-ever, these techniques may not endow with adequate information about theimage details. The SSIM is based on the structure comparison of original imageand enhanced image. Although, SSIM uses luminance, contrast and structure

information in image quality evaluation but the method fails to produce sat-isfactory results for poorly blurred images. The CNI and CCI metrics providebetter results when compared with other methods. But the algorithm is not ef-ficient from computation point of view. In addition, image quality evaluationbased on CNI and CCI works for certain class of images and may not offer sat-isfactory results for medical images, facial images, and aerial images. Based onthe above discussions it is clear that PSNR, contrast, luminance, SSIM, CNI,and CCI are not appropriate and effective to evaluate the performance of thecolor image. Thus, the proposed Wavelet energy metric is more advanced andovercomes the drawback of other image quality assessment methods.

4 Experimental Results

We conduct numerous experiments on different images in order to evaluate theperformance of the proposed image quality metric. The proposed wavelet energymetric overcomes the drawback of other researcher methods [11] [12] in an effi-cient way. We have chosen three image enhancement methods such as histogramequalization, MSRCR, and improved MSRCR with three test images in orderto verify the proposed method. Image quality assesment metrics such as PSNR,contrast, luminance are measured and is compared with the proposed technique.The results are as shown in Table 1, Table 2, and Table 3 for the Fig. (1), Fig. (2),and Fig. (3) respectively. Table 1 shows the perfromance comparison of the colorimage enhancement for the ’palace’ image using histogram equalization, MSRCRand improved MSRCR. Table 2 shows the perfromance comparison of the colorimage enhancement for the ’house’ image using histogram equalization, MSRCRand improved MSRCR. Table 3 shows the perfromance comparison of the colorimage enhancement for the ’white house’ image using histogram equalization,MSRCR and improved MSRCR.

It is clear that for the ’palace’ image shown in Fig. (1), histogram equalizationmethod has poor details and overall image enhancement is not satisfactory. Theimage details and overall image enhancement using MSRCR approach is goodbut not efficient. The image enhancement using improved MSRCR is best inenhancing image details and overall image enhancement since, WE (approx) is99.57 and WE (detailed) is 0.419. We observe that approximate and detailed co-efficients for the enhanced image is larger than the original image (WE(approx):99.55, and WE (detailed): 0.439) therefore, the improved MSRCR is better com-pared to other methods.

The MSRCR based color image enhancement shown in Fig. (2)c has clearerdetails and overall image enhancement is excellant when compared with othermethods. The approximate (99.69) and detailed (0.302) wavelet energy of en-hanced MSRCR image is greater than the original image approximate (99.84)and detailed (0.152). Therefore, the MSRCR based image enhancement is betterthan other methods.

Finally in Fig. (3)c. the MSRCR technique offers best detail enhancementbut overall image enhancement is not satisfatory. However, the overall imageenhancement using improved MSRCR of Fig. (3)d is better but details are notsatisfactory. The above observations are made based on the results tabulted.

(a) Original Image of Resolution of1024×768 pixels

(b) Image Enhancement using HistogramEqualization

(c) Image Enhancement using MSRCR (d) Image Enhancement using ImprovedMSRCR

Fig. 1. Image Enhancement Methods Comparison based on Quality Assessment

Table 1. Metric Comparison for the Color Image Enhancement shown in Fig. (1)

Original Image : WE (Approx.) : 99.55, WE (Detl.) : 0.439

Method PSNR Contrast Luminance WE(Appr.) WE (Detl.)

Histogram Equalization 34.48 0.546 0.025 99.70 0.285

MSRCR 36.77 0.118 0.132 99.59 0.397

Improved MSRCR 36.95 0.034 0.017 99.57 0.419

(a) Original Image of Resolution of1024×768 pixels

(b) Image Enhancement using HistogramEqualization

(c) Image Enhancement using MSRCR (d) Image Enhancement using ImprovedMSRCR

Fig. 2. Image Enhancement Methods Comparison based on Quality Assessment

Table 2. Metric Comparison for the Color Image Enhancement shown in Fig. (2)

Original Image : WE (Approx.) : 99.84, WE (Detl.) : 0.152

Method PSNR Contrast Luminance WE(Appr.) WE (Detl.)

Histogram Equalization 30.14 4.100 0.032 99.83 0.166

MSRCR 32.82 0.072 0.002 99.69 0.302

Improved MSRCR 32.11 0.028 0.002 99.89 0.102

Table 3. Metric Comparison for the Color Image Enhancement shown in Fig. (3)

Original Image : WE (Approx.) : 99.50, WE (Detl.) : 0.487

Method PSNR Contrast Luminance WE(Appr.) WE (Detl.)

Histogram Equalization 30.39 0.028 0.558 99.69 0.300

MSRCR 35.94 0.397 0.071 99.10 0.893

Improved MSRCR 32.86 0.011 0.004 99.64 0.343

5 Conclusion

In this paper, we propose a novel effective and objective metric for evaluatingthe performance of color image enhancement. The existing metrics are reviewed

(a) Original Image of Resolution of1024×768 pixels

(b) Image Enhancement using HistogramEqualization

(c) Image Enhancement using MSRCR (d) Image Enhancement using ImprovedMSRCR

Fig. 3. Image Enhancement Methods Comparison based on Quality Assessment

and limitations are investigated. The wavelet energy is computed using a linearcombination of high frequency coefficients after a Daubechies wavelet transform.The probability density function of the enhanced image has relatively higherwavelet energy. Therefore, proposed wavelet energy can be used as an effectiveand efficient metric in order to evaluate the quality of the enhanced image. Theapproximate wavelet coefficient provides the information about the global imageenhancement and detailed coefficients of wavelet energy provides statics on theimage details. Numerous experiments demonstrate and validate the superiorityand effectiveness of the proposed metric, wavelet energy, which may find wideapplications in image processing and computer vision.

References

1. Zhou Wang, Alan C Bovik.: Modern Image Quality Assessment: Synthesis Lectureson Image, Video & Multimedia Processing. Morgan & Claypool Publishers(2006)

2. Kai-Qi Huang, Qiao Wang, and Zhen-Yang Wu.: Natural Color Image Enhancementand Evaluation Algorithm based on human visual system. Journal of ComputerVision and Image Understanding. 52–63 (2006)

3. A. S Zianou, and Hachouf, F.: Image Quality Assessment Measure based on VisualRegion and Displacement with distorted pixels for compressed images. Journal ofAutomation and Systems Engineering. 196–211 (2010)

4. Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero Simoncelli, P.:Image Quality Assessment: from Error Visibility to Structural Similarity. IEEETranscaction on Image Processing. Vol. 13, No. 14, 600–612 (2004)

5. Zhou Wang, Alan Conrad and Bovik.: A Universal Image Quality Index. In: IEEESignal Processing Letters, Vol. 9, No. 3, pp. 81–84(2002)

6. Czajkowski, K., Fitzgerald, S., and Foster, I., Kesselman, C.: Color Image Qualityon the Internet. In: Proceedings of IS & T/SPIE Electronic Imaging, Vol. 5304, pp.118–131. (2004)

7. Hanumantharaju, M.C., Ravishankar, M., Rameshbabu, D.R., Ramachandran,S.:Adaptive Color Image Enhancement based on Geometric Mean filter. In: Pro-ceedings of International Conference on Communication, Computing and Security(ICCCS-2011), pp. 403–408. (2011)

8. Yue Wang, Tingting Jiang, Siwei Ma, and Wengao.: Image Quality Assesmentbased on Local Orientation Distributions. In: Proceedings of Picture Coding Sym-posium,pp. 274–278. (2010)

9. Jun Feng Li, Wen Zhan Dai, and Hui Jiao Wang.: Image Quality Assessment Basedon Fuzzy Similarity Measure and Wavelet Transform. Journal of Advanced MaterialsResearch,pp. 31–36. (2011)

10. Lin Zhang, Lei Zhange,Xuanqin Mou, and David Zhang.: FSIM: A Feature Simi-larity Index for Image Quality Assessment. IEEE Transactions on Image Processing,Issue 99, pp. 1–10. (2011)

11. Weisi Lin, and Jay Kuo, C. C.: Perceptual Visual Quality Metrics: A Survey Jour-nal of Visual Communication and Image Representation, Vol. 22, Issue 4, pp. 297–312. (2011)

12. Xuelong Li, Dacheng Tao, Xinbo Gao, and Wen Lu C.: A Natural Image QualityEvaluation Metric. Journal of Signal Processing, Vol. 89, Issue 4, pp. 548–555. (2009)