USF Libraries
USF Digital Collections

An algorithm for image quality assessment

MISSING IMAGE

Material Information

Title:
An algorithm for image quality assessment
Physical Description:
Book
Language:
English
Creator:
Ivkovic, Goran
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
image coding
image quality measure
human visual system
contrast sensitivity function
brightness perception
Dissertations, Academic -- ELECTRICAL ENGINEERING -- Masters -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: Image quality measures are used to optimize image processing algorithms and evaluate their performances. The only reliable way to assess image quality is subjective evaluation by human observers, where the mean value of their scores is used as the quality measure. This is known as mean opinion score (MOS). In addition to this measure there are various objective (quantitative) measures. Most widely used quantitative measures are: mean squared error (MSE), peak signal to noise ratio (PSNR) and signal to noise ratio (SNR). Since these simple measures do not always produce results that are in agreement with subjective evaluation, many other quality measures have been proposed. They are mostly various modifications of MSE, which try to take into account some properties of human visual system (HVS) such as nonlinear character of brightness perception, contrast sensitivity function (CSF) and texture masking. In these approaches quality measure is computed as MSE of input image intensities or frequency domain coefficients obtained after some transform (DFT, DCT etc.), weighted by some coefficients which account for the mentioned properties of HVS. These measures have some advantages over MSE, but their ability to predict image quality is still limited. A different approach is presented here. Quality measure proposed here uses simple model of HVS, which has one user-defined parameter, whose value depends on the reference image. This quality measure is based on the average value of locally computed correlation coefficients. This takes into account structural similarity between original and distorted images, which cannot be measured by MSE or any kind of weighted MSE. The proposed measure also differentiates between random and signal dependant distortion, because these two have different effect on human observer. This is achieved by computing the average correlation coefficient between reference image and error image. Performance of the proposed quality measure is illustrated by examples involving images with different types of degradation.
Thesis:
Thesis (M.S.E.E.)--University of South Florida, 2003.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Goran Ivkovic.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 82 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001416911
oclc - 52786340
notis - AJJ4763
usfldc doi - E14-SFE0000049
usfldc handle - e14.49
System ID:
SFS0024745:00001


This item is only available as the following downloads:


Full Text

PAGE 1

An Algorithm for Image Quality Assessment by Goran Ivkovic A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering Department of Electrical Engineering College of Engineering University of South Florida Major Professor: Ravi Sankar, Ph.D. Wei Qian, Ph.D. Arthur David Snider, Ph.D. Date of Approval: July 10, 2003 Keywords: brightness perception, contrast se nsitivity function, human visual system, image quality measure, image coding Copyright 2003, Goran Ivkovic

PAGE 2

i TABLE OF CONTENTS LIST OF TABLES ii LIST OF FIGURES iii ABSTRACT vii 1. INTRODUCTION 1 1.1. Background 1 1.2. Overview of image quality measures 1 1.3. Descript ion and organization of thesis 5 2. HVS MODEL 7 2.1. Brightness perception 7 2.2. Contrast sensitivity function 12 2.3. Simple HVS model 22 3. IMAGE QUALITY MEASURE 23 3.1. MSE as image quality measure 23 3.2. Definition of image quality measure 29 4. RESULTS 36 4.1. “Lena” image with various types of distortion 36 4.2. “Goldhill” image with va rious types of distortion 37 4.3. “Couple” image with vari ous types of distortion 42 4.4. “Tiffany”, “Lake”and ”Mandrill” images with JPEG coding 48 4.5. “Woman”, “Man” and “Barba ra” images with blurring 51 4.6. “Lena” image with inverted grayscale 55 4.7. “Lena” image with blurring and additive noise 57 4.8. Comparison of JPEG and JPEG2000 coding using “Lena” image 61 4.9. Performance of quality measure on two different images 70 5. CONCLUSION 71 REFERENCES 73

PAGE 3

ii LIST OF TABLES Table 1: “Lena” images with various types of distortion 36 Table 2: “Goldhill” images with various types of distortion 37 Table 3: “Couple” images with various types of distortion 43 Table 4: “Tiffany”, “Lake” and “Mandr ill” images with JPEG coding 48 Table 5: “Woman”, “Man” and “Bar bara” images with blurring 51 Table 6: “Lena” images with inverted grayscale 55 Table 7: “Lena” images with blurring and additive noise 58 Table 8: Comparison of JPEG and JPEG2000 coding 62 Table 9: Performance of quality measure on two different images 70

PAGE 4

iii LIST OF FIGURES Figure 1: Nonlinear function modeling brightness perception 10 Figure 2: “Lena” image with additive noise 10 Figure 3: Error image 11 Figure 4: Error image computed after original and noisy images were modified by (3) 11 Figure 5: Ideal magnitude frequency response 14 Figure 6: Magnitude frequency response of FIR filter 14 Figure 7: “Lena” image with filtered noise (fmin=0, fmax=0.025) 15 Figure 8: “Lena” image with filtered noise (fmin=0.05, fmax=0.1) 15 Figure 9: “Lena” image with filtered noise (fmin=0.1, fmax=0.15) 16 Figure 10: “Lena” image with filtered noise (fmin=0.15, fmax=0.2) 16 Figure 11: “Lena” image with filtered noise (fmin=0.3, fmax=0.35) 17 Figure 12: “Lena” image with filtered noise (fmin=0.35, fmax=0.4) 17 Figure 13: “Lena” image with filtered noise (fmin=0.45, fmax=0.5) 18 Figure 14: “Lena” image with filtered noise (fmin=0.5, fmax=0.55) 18 Figure 15: Contrast sensitivity function 19 Figure 16: Computation of vertical visual angle 20 Figure 17: Filtering of an image by CSF 22 Figure 18: Simple model of HVS 22 Figure 19: Original “Lena” image 25 Figure 20: “Lena” image with im pulsive salt-pepper noise 25

PAGE 5

iv Figure 21: “Lena” image with additive white noise 26 Figure 22: “Lena” image with multiplicative speckle noise 26 Figure 23: “Lena” image with mean shift 27 Figure 24: “Lena” image with contrast stretching 27 Figure 25: “Lena” image with blurring 28 Figure 26: JPEG coded “Lena” image 28 Figure 27: JPEG2000 coded “Lena” image 29 Figure 28: Function modeling the different eff ect of random and signa l dependent noise 35 Figure 29: Original “Goldhill” image 38 Figure 30: “Goldhill” image with impulsive salt-pepper noise 38 Figure 31: “Goldhill” image with additive white noise 39 Figure 32: “Goldhill” image with multiplicative speckle noise 39 Figure 33: “Goldhill” image with mean shift 40 Figure 34: “Goldhill” image w ith contrast stretching 40 Figure 35: “Goldhill” im age with blurring 41 Figure 36: JPEG coded “Goldhill” image 41 Figure 37: JPEG2000 coded “Goldhill” image 42 Figure 38: Original “Couple” image 43 Figure 39: “Couple” image with im pulsive salt-pepper noise 44 Figure 40: “Couple” image with additive white noise 44 Figure 41: “Couple” image with mu ltiplicative speckle noise 45 Figure 42: “Couple” image with mean shift 45 Figure 43: “Couple” image with contrast stretching 46

PAGE 6

v Figure 44: “Couple” image with blurring 46 Figure 45: JPEG coded “Couple” image 47 Figure 46: JPEG2000 coded “Couple” image 47 Figure 47: Original “Tiffany” image 48 Figure 48: JPEG coded “Tiffany” image 49 Figure 49: Original “Lake” image 49 Figure 50: JPEG coded “Lake” image 50 Figure 51: Original “Mandrill” image 50 Figure 52: JPEG coded “Mandrill” image 51 Figure 53: Original “Woman” image 52 Figure 54: “Woman” imag e with blurring 52 Figure 55: Original “Man” image 53 Figure 56: “Man” image with blurring 53 Figure 57: Original “Barbara” image 54 Figure 58: “Barbara” im age with blurring 54 Figure 59: Inverted “Lena” image 56 Figure 60: Inverted “Lena” image w ith impulsive salt-pepper noise 56 Figure 61: Inverted “Lena” image with additive noise 57 Figure 62: “Lena” image with blurring 58 Figure 63: “Lena” image with blurri ng and additive noise of sd=6.8 59 Figure 64: “Lena” image with blurri ng and additive noise of sd=9.4 59 Figure 65: “Lena” image with blurri ng and additive noise of sd=11.5 60 Figure 66: “Lena” image with blurri ng and additive noise of sd=13.3 60

PAGE 7

vi Figure 67: “Lena” image with additive noise of sd=15 61 Figure 68: JPEG coding at 1.1561 bits/pixel 63 Figure 69: JPEG2000 coding at 1.1561 bits/pixel 63 Figure 70: JPEG coding at 0.5501 bits/pixel 64 Figure 71: JPEG2000 coding at 0.5501 bits/pixel 64 Figure 72: JPEG coding at 0.3647 bits/pixel 65 Figure 73: JPEG2000 coding at 0.3647 bits/pixel 65 Figure 74: JPEG coding at 0.246 bits/pixel 66 Figure 75: JPEG2000 coding at 0.246 bits/pixel 66 Figure 76: JPEG coding at 0.1747 bits/pixel 67 Figure 77: JPEG2000 coding at 0.1747 bits/pixel 67 Figure 78: JPEG coding at 0.1428 bits/pixel 68 Figure 79: JPEG2000 coding at 0.1428 bits/pixel 68 Figure 80: JPEG coding at 0.1129 bits/pixel 69 Figure 81: JPEG2000 coding at 0.1129 bits/pixel 69

PAGE 8

vii AN ALGORITHM FOR IMAGE QUALITY ASSESSMENT GORAN IVKOVIC ABSTRACT Image quality measures are used to optimize image pro cessing algorithms and evaluate their performances. The only reliable way to assess image quality is subjective evaluation by human observers, where the mean value of their scores is used as the quality measure. This is known as mean opinion score (MOS). In addition to this measure there are various objective (quantitative) measures. Most widely used quantitative measures are: mean squared error (MSE), peak signal to noise ratio (PSNR) and signal to noise ratio (SNR). Since these simple measur es do not always produce results that are in agreement with subjective evaluation, many ot her quality measures have been proposed. They are mostly various modifications of MSE, which try to take into account some properties of human visual system (HVS) such as nonlinear character of brightness perception, contrast sensitivity function (CSF) and texture masking. In these approaches quality measure is computed as MSE of input image intensities or frequency domain coefficients obtained after some transfor m (DFT, DCT etc.), weighted by some coefficients which account for the mentioned properties of HVS. These measures have some advantages over MSE, but their ability to predict image quality is still limited. A different approach is presented he re. Quality measure proposed here uses simple model of HVS, which has one user-defined parameter, w hose value depends on the reference image. This quality measure is base d on the average value of locally computed

PAGE 9

viii correlation coefficients. This takes into account structural similarity between original and distorted images, which cannot be measured by MSE or any kind of weighted MSE. The proposed measure also differentiates between random and signal dependant distortion, because these two have different effect on human observer. This is achieved by computing the average correlation coefficient between reference image and error image. Performance of the proposed qua lity measure is illustrated by examples involving images with different types of degradation.

PAGE 10

1 1. INTRODUCTION 1.1. Background Methods for image quality assessment can be divided into two groups: subjective and objective. The only subjective measure is mean opinion score (MOS) where quality of an image is judged by several individuals and then the mean value of their scores is used as the measure. Since human observer is ultimate receiver of the information contained in an image, this is the best way to assess image quality. Objective measures try to assess im age quality and express it as a number. The goal in developing objective (quantitative) measures is to find the one that correlates well with image quality as perceived by human visual sy stem (HVS). These measures are used to assess performance of different algorithms for image enhancement and coding. Quality measures can also be used to optimize image processing algorithms. For example in image coding the goal is to represent images with as few bits as possible with the minimal loss of image quality. Quality measure is used as a criterion in a sense that the bits that contribute more significantly to the used quality measure are coded first. 1.2. Overview of image quality measures Most widely used objective measur es are mean squared e rror (MSE) and MSE-based measures: peak signal to noise ratio (PSNR) and signal to noise ratio (SNR). These simple measures work well when we compare images with the same type of degradation.

PAGE 11

2 In this case distorted image with smaller MSE will be perceived closer to the original image than the one with greater MSE. However, when we compare images with different types of degradation MSE does not produce re sults that correlate well with subjective quality assessment. Images with different types of degradations with the same MSE values can have very different subjective visual qualities. In order to find a criterion, which agr ees with subjective assessment, many other algorithms have been developed. These algorithms try to assess image quality not just by using some simple formula but also by taki ng into account the prope rties of HVS. Most of these algorithms are trying to model the following three properties of HVS: nonlinear relationship between image intensities and pe rceived brightness, frequency response of HVS (contrast sensitivity function) and texture masking. One such algorithm is described in [1]. In this approach input intensities are first modified by a nonlinear function modeling brightness perception and then filtered by a 2D filter modeling frequency response of HVS. After this, processed images are decomposed into different subbands using a sy stem of filters. Then perceptual threshold is computed for every spatial location in every subband. Using the computed threshold probability that the error is visible is comput ed again for every spatial location in each subband. Finally, probabilities at same spa tial location from diffe rent subbands are combined to compute probability of visible di fference at that spatia l location. The output of this algorithm is map of probabilities whic h has the same size as input images. If the value at some spatial location is 1, then ther e is visible difference at that location between input images. If the value is less than 1, then the difference is not clearly visible, it is somewhere in the threshold region. This appr oach can identify the pixels with visible

PAGE 12

3 distortion, but it cannot quantify the extent of easily visible distortion. Also, this algorithm does not produce one numbe r that indicates image quality. Image quality measures are sometime s developed as a part of image processing algorithms. An example of this can be found in [2], where properties of HVS are used to design an algorithm for image c oding. In this approach input image is first decomposed into 16 subbands using generalized quadr ature mirror filters. Each subband is downsampled by a factor of four in horizontal and vertical directions so the total number of subband coefficients is equal to the num ber of pixels in th e input image. Then perceptual threshold is computed for every subband coefficient at each spatial location. Computation of the perceptual thresholds ta kes into account the same HVS properties as in [1], but here they are mode led in a different way. If the coefficient in some subband at some location is above the computed perceptu al threshold then it is visible to human observer, otherwise it is not visible. Each subband is then coded using differential pulse code modulation (DPCM), where the quantizer step for a subband is equal to the minimal perceptual threshold in that s ubband. Image quality measure in this case can be defined as MSE of weighted subband coefficients of the original image and distorted image, where each subband coefficient is normalized by the perceptual threshold for that subband at that spatial location. Another approach is presented in [3]. It is assumed here that input image is degraded by two sources of degradation: linear distortion (for example as a result of filtering) and additive noise. In the first step linear distor tion and additive noise are separated and then two quality measures are computed: distor tion measure and noise quality measure. Distortion measure is computed using discrete Fourier tran sform (DFT) coefficients of

PAGE 13

4 the original image and original image dist orted by linear distor tion only. Noise quality measure is computed using the original image degraded by linear distortion and noise and the original image degraded by linear distor tion only. These two images are decomposed into subbands using a system of filters and perceptual threshold is computed for every subband and every spatial location. If the diffe rence between the two images is below the perceptual threshold it is not considered vi sible and that differenc e is ignored, otherwise the difference is taken into account. Finally, noise quality measure is compute as SNR of the modified input images, which contain onl y visible difference between the two images. This approach requires as i nput not only two images but also the algorithm that causes linear distortion and produces two nu mbers, which assess image quality. An overview of image quality measures is presented in [4]. Performance of several proposed quality measures is tested in [5]. Distorted images ar e created using four different image coding algorithms, which prod uce different types of degradation. Then tested quality measures are computed for each distorted image and these values are compared with subjective assessment. The tested measures included MSE and the measure proposed in [2]. The quality measur e from [2] showed best agreement with subjective evaluation compared with other tested measures. However, none of the tested measures was able to predict subjec tive image quality consistently. In [6] and [7] previously proposed quali ty measures are discusse d and it is noted that these measures frequently use some problema tic assumptions. Also it is pointed out that MSE or some modifications of MSE (such as th e one described in [2]) completely fail to measure structural similarity between two images, which is very important for a human observer. In other words MSE computes only error between the corre sponding pixels but

PAGE 14

5 does not take into account corre lation between that pixel and surrounding pixels in any of the two input images. It is possible to in troduce a very large MSE by shifting the mean value of image intensities or by contrast st retching and preserve structural similarity between the two images. On the other hand if the same amount of MSE is introduced as a result of low-pass filtering or image coding, the resulting image can be severely distorted. Algorithms described in [1], [2] and [3] do not measure structural similarity at all. Based on this discussion a new image quality measure is proposed in [6] and [7]. This measure is computed locally and it is defined as a pr oduct of three components. Most important of these components is correlation coefficient, which measures the degree of linear relationship between the corresponding blocks of pixels. Quality measure for the whole image is the average value of locally comput ed quality measures. This approach does not use HVS model at all. 1.3. Description and organization of the thesis The approach presented here will use some ideas from [6] and [7], but it will not use image quality measure as it is defined there (onl y correlation coefficient will be used). In the first step reference image and distorted image are processed by simple model of HVS which consists of a nonlinear function modeli ng brightness perception and a 2-D filter modeling frequency response of HVS (contrast sensitivity function). The 2-D filter used in this HVS model is not fi xed. It contains one user-def ined parameter which can be changed depending on the content of the re ference image. After this, the correlation coefficient is computed on a block-by-block basis for the processed input images. Finally, quality measure is computed as the average correlation coefficient between the reference

PAGE 15

6 image and distorted image, adjusted by av erage correlation coefficient between the reference image and error image. The thesis is organized as follows. Chapter two describes HVS model used to process input images. Chapter three discusses problems with MSE and presents the proposed algorithm. Performance of the proposed al gorithm will be illustrated by examples involving images with different types of distortion in Chapter four. Most of these images (but not all) are the same ones used to test quality measure developed in [6] and [7]. Chapter five concludes the thesis.

PAGE 16

7 2. HVS MODEL 2.1. Brightness perception Digital images are represented using a finite number of intens ity levels. For example, gray scale images with 8 bits/pixel are represented by 256 inte nsity levels, where 0 corresponds to the darkest leve l and 255 corresponds to the brightest level. Intensities between these two extremes represent various shades of gray level. Brightness perceived by HVS is not a linear function of intensity represented by integers from 0 to 255. This can be shown by the following experiment. Firs t we create an image which consists of uniform background of intensity I and a sma ll square in the center of intensity I+ I, and then we repeat this for different values of background intensity I while keeping I constant. When the background intensity is ve ry low (for example I=10) or very high (I=240) the difference in percei ved brightness is between th e background and the center square is smaller than in the case when the background in tensity is in the mid range (I=125). Since the intensity difference between the background and the center square is the same in each case, perceived brightness mu st be a nonlinear function of intensity. In other words equal increases in intensity do not correspond to equal increases in perceived brightness. This phenomenon is well known but it is very difficult to model it because we do not know how to measure perceived br ightness. Usually, this is modeled by transforming input intensities by some nonlinear function, such as logarithmic or cube

PAGE 17

8 root function. If we denote perceived bright ness by B and input intensity by I, then by using logarithmic functio n they are related as: I K B log (1) where K is a constant. This is known as Weber’s law and is c ited in many books on image processing (for example [4 ] and [8]). The second formula, which is used in [9], is obtained using cube root function: 33 0) ( I K B (2) where K is again some constant. None of th ese two formulas models the perception of brightness accurately, especially at low intensities. According to them the sensitivity to intensity increases is greatest at low intensit ies and decreases at mi d and high intensities. This is clearly wrong, because an increase in intensity result in a very small increase in perceived brightness, when th e intensity is low. Theref ore, the following nonlinear function is used here to model brightness perception: MAX MAX MAX MAX MAX MAX MAX MAX MAXI I I I I I B B I I I I B B 2 / if ) ( 2 2 2 / 0 if 2 25 2 5 2 (3) where IMAX=255 is the maximum value of intensity and BMAX=100 is the maximum value of perceived brightness. In other words when the intensity goes from 0 to 255, the perceived brightness goes from 0 to 100 as s hown in Figure 1. This range is chosen for convenience. The role of this function is to emphasize intensities in the mid range and deemphasize very high and very low intens ities, which approximates the brightness

PAGE 18

9 perception by HVS. The biggest problem with brightness perception models is how to test them and tell whether they produce reason able results or not. In this case “Lena” image corrupted by additive white Gaussian no ise shown in Figure 2 is used as a test. Then the first residual image is computed as original image minus corrupted image, which is shown in Figure 3. The second residua l image shown in Figure 4 is computed as the difference between the same images, but this time both images were modified by the function given by equation (3). In the firs t residual image we see uniform amount of noise across the image. However, in the second residual image there is less noise in the areas corresponding to the very da rk or very bright areas in the origin al image and there is more noise in the areas where intensity is in the mid range. This is approximately what we see in the noisy image: noise is less visible in very bright or very dark areas than in other areas of the image. A ccording to the models given by equations (1) and (2) the noise would be most visible in the very dark areas of the image, which is clearly not the case. That is why the model given by equation (3) is used instead of the models given by equations (1) and (2). It is not possible to give general fo rmula that rela tes perceived brightness and intensity, because this depends on the display device, so the model of brightness perception must be determined sepa rately for each display device.

PAGE 19

10 Figure 1: Nonlinear function modeling brightness perception Figure 2: "Lena" image with additive noise

PAGE 20

11 Figure 3: Error image Figure 4: Error image computed after original and noisy images were modified by the function (3)

PAGE 21

12 2.3. Contrast sensitivity function Contrast sensitivity function (CSF) de scribes the frequency response of HVS. HVS is not equally sensitive to all spatial freque ncies, which can be shown by the following experiment. First we design a 2-D filter, wh ich will approximately have the following magnitude frequency response: otherwise 0 for 1 ) (max 2 2 2 1 min 2 1f f f f f f H (4) In the last formula f1 and f2 are normalized frequencies, taking values between -0.5 and 0.5. Therefore, the values for fmin and fmax must be between 0 and 0.707. This filter cannot be realized exactly but it can be numerically appr oximated by creating a finite impulse response (FIR) filter using frequenc y sampling method. FIR filter obtained by this method will have the same magnitude freq uency response as the ideal filter at given points in the frequency plane. Magnitude frequency response of the ideal filter and the obtained FIR filter are shown in Figures 5-6 for fmin=0.35 and fmax=0.4. Then we generate white Gaussian noise and filter it by the FIR filter obtained using the described procedure. This filtered noise is added to the original Lena image for various values of fmin and fmax. Each time the filtered noise is scaled so that its standard deviation is 10. We get the sequence of images shown in Figu res 7-14. All images in the sequence are corrupted by the noise with the same variance, but the noise is not e qually visible in each image. The noise is quite noti ceable at low frequencies and then it becomes even more noticeable with increase in frequency up to some point. After that we see gradual decrease in noise visibility with increase in frequency. At very high frequencies we can hardly see any noise at all ev en though these images are co rrupted by the noise with the

PAGE 22

13 same variance as the rest of the sequence. This example shows that HVS is a frequency selective system. If HVS were not a frequenc y selective system, we would see the same amount of noise in each image in the sequen ce. Since we see less noise at very low and very high frequencies than in the mid fre quency range, HVS must be a band-pass system. This is modeled by a contrast sensitivity f unction (CSF), which repr esents the frequency response of HVS. CSF models HVS as a 2-D filte r and it is given in the normalized units. It is equal to one at the freque ncy where the sensitivity is ma ximal and everywhere else is less than one. Also frequency is expresse d in cycles/degree. This may look strange because it seems natural to express it in cycles/cm. The reason for this is the following. Images can be observed form the various distances and the same spatial frequency expressed in cycles/cm will be perceived differe ntly for different viewing distances. If the viewing distance is greater signal of the certain spatial frequency will appear to have greater frequency than that same signal view ed from the smaller distance. That means that HVS perceives spatial frequency in cycles per unit visual angle instead of frequency in cycles per unit length. Various functions have been suggested to model this effect. Most of them have a band-pass character but they have maximum at different frequencies. For example CSF shown in [4] has maximum at 3 cycles/degree, while CSF used in [9] has maximum at 8 cycles/degree. It is also noted in [10] th at various CSF suggested by different authors have maximum at frequencies ranging from 3 to 10 cycles/degree. Since there is no reliable way to tell which one of these functi ons represents the best model of HVS, none of them will be used here. Instead, CSF given by the following formula will be used:

PAGE 23

14 Figure 5: Ideal magnitude frequency response Figure 6: Magnitude frequency response of FIR filter

PAGE 24

15 Figure 7: "Lena" image with filtered noise (fmin=0, fmax=0.025) Figure 8: "Lena" image with filtered noise (fmin=0.05, fmax=0.1)

PAGE 25

16 Figure 9: "Lena" image with filtered noise (fmin=0.1, fmax=0.15) Figure 10: "Lena" image with filtered noise (fmin=0.15, fmax=0.2)

PAGE 26

17 Figure 11: "Lena" image with filtered noise (fmin=0.3, fmax=0.35) Figure 12: "Lena" image with filtered noise (fmin=0.35, fmax=0.4)

PAGE 27

18 Figure 13: "Lena" image with filtered noise (fmin=0.45, fmax=0.5) Figure 14: "Lena" image with filtered noise (fmin=0.5, fmax=0.55)

PAGE 28

19 ) ) 2622 0 ( exp( ) 695 0 0509 0 ( ) ( where ree cycles/deg 5 3 for S(f) ree cycles/deg 5 3 for ), ( ) (1 1f f f S f f f S f H (5) In the last formula, f denotes frequency in cycles/degree and represents the parameter which controls the rate at which C SF decreases after it reaches its maximum at 3.5 cycles/degree. This func tion is shown in Figure 15 for =0.75. Figure 15: Contrast sensitivity function Although the previous experiment with the sequence of “Lena” images shows that HVS is frequency selective, we must be cauti ous with this interpretation. This example shows that noise at high frequencies is le ss visible than noise at low frequencies. However, when high frequency components represent information contained in an image

PAGE 29

20 (for example some details or small objects) instead of noise, those components become very important for a human observer. Because of this effect, HVS is not modeled here as a filter with fixed parameters. Instead, a parameter which controls attenuation at frequencies higher than 3.5 cycles/degree, is introduced. This parameter is user-defined and depends on the reference image. If the re ference image contains one large object (for example “Lena” image) than we use higher values for than in the case when the reference image contains smaller objects or lo t of fine details. This requires some user intervention but it yields better results. This property of HVS can be taken in to account by filtering input image by CSF. The first step in this process is to compute vert ical and horizontal viewi ng angles. The vertical visual angle can be computed from Figure 16, where the height of the image is denoted by h, viewing distance is denot ed by d and eye of a human ob server is at the point O y h O d Figure 16: Computation of vertical visual angle Then the vertical visual angle can be found from the following formula: d hy2 2 tan (6) If we express this angle in degrees we get:

PAGE 30

21 d hy2 tan 2 1801 0 (7) If we denote width of the image by w, th e horizontal viewing a ngle can be found as: d wx2 tan 2 1801 0 (8) For h=15 cm, w=15 cm and d=60 cm the values for the horizontal and vertical viewing angles are: x=14.25 and y=14.25 degrees. If this image is 512x512 pixels, the sampling frequencies in horizonta l and vertical directions in samples/degree are found as: gree samples/de 93 35 512 gree samples/de 93 35 512 f fy ys x xs (9) Then we compute the DFT of the input imag e denoted by f(m,n), whose size is M by N pixels: )) ( ( ) ( n m f DFT l k F (10) If we denote CSF by H(f), the DFT of the output image denoted by X(k,l) is: ) ( )) ( ( ) ( l k F l k f H l k X (11) where 2 2) ( ) ( ) ( k f l f l k fy x (12) 1 2 / for 2 / 0 for ) ( N l N f N l N N l f N l l fxs xs x (13)

PAGE 31

22 1 2 / for 2 / 0 for ) ( M k M f M k M M k f M k k fys ys y (14) Finally, the output signal de noted by x(m,n) is found as inverse DFT of X(k,l): )) ( ( ) ( l k X IDFT n m x (15) f(m,n) x(m,n) Figure 17: Filtering of an image by CSF The described procedure can be represented as a simple 2-D filtering as shown in Figure 17, where it is assumed that 2-D filter has a radial symmetry: ) ( ) (2 2 2 1 2 1f f H f f H (16) 2.3. Simple HVS model Based on the discussion in this chapte r we can create a simple model of HVS shown in Figure 18, which models brightness per ception and frequency response of HVS. The output signal in this case depends on the view ing distance, the width and the height and the number of pixels of the input image. Th is is reasonable because image perception by HVS also depends on these parameters f(m,n), p(m,n) x(m,n), y(m,n) Figure 18: Simple model of HVS H(f1,f2) Nonlinear function 2-D filter

PAGE 32

23 3. IMAGE QUALITY MEASURE 3.1. MSE as image quality measure The objective of this research is to develop an image quality measure, which will produce results that are in agreement with su bjective assessment. But before this, it is reasonable to ask why simple measures such as MSE or SNR are not working well. These simple measures work when we have images with the same type of degradation. For example, if an image is distorted by additive white Gaussian noise the image with the smallest MSE or SNR will be perceived to be the closest to the original image. This will also be the case for all others types of distor tion such as blurring or coding. If we have a sequence of images obtained by distorting an image by various amounts of the same type of degradation, the image with the smalle st MSE or SNR will be perceived by a human observer to be the closest to the original im age. However, when we compare images with different types of degradation, the image w ith the smallest MSE or SNR will not always be perceived to be the closest to the original image. This can be easily seen by looking at the sequence of “Lena” images distorted by va rious types of degrada tion: impulsive saltpepper noise, additive white noise, multiplicativ e speckle noise, mean shift, contrast stretching, blurring (which is a result of low-pass filtering), JPEG and JPEG2000 coding. The original image and distorte d images are shown in Figures 19-27. All distorted images have the same MSE (except for the JPEG code d image) but their visual quality is very different. Since images with the same MSE have very different visual quality, it can

PAGE 33

24 be concluded that MSE is not the criteri on used by a human observer in judging how close the distorted image is to the original image. If MSE were the criterion then the images with the same MSE would have the sa me visual quality. Anot her example for this is the sequence of “Lena” images shown in Figures 7-14. All images in the sequence have the same MSE (because the variance of added noise is always the same), but the visual quality of those images is very different. This is a well-known fact and this is why many other algorithms for image quality assessment have been developed. These algo rithms are using various models of HVS, which are used to transform input intensitie s from the original and distorted images. Quality measure is then computed using tran sformed intensities as MSE or some kind of weighted MSE. The basic idea in these algorith ms is to take into account the properties of HVS by modeling the process of visual pe rception. However our knowledge of this process is very limited and these algor ithms frequently use very questionable assumptions. In some cases these algorithms are very complicated and they do not perform very well. Main problem with MSE or MSE-based image quality measures is the fact that they treat an image as a set of uncorrelated num bers. When MSE is computed correlation between the neighboring pixels is completely ignored. This information is very important because human observer is more concerned with correlation betw een the neighboring pixels, which determines what type of structure they form in an image, than with the actual intensity at a given location. In other words the intensity at a given point is not so important, as the structure which that pixel crea tes with surrounding pixels.

PAGE 34

25 Figure 19: Original "Lena" image Figure 20: "Lena" image with impulsive salt-pepper noise (MSE=225, Q=0.8543)

PAGE 35

26 Figure 21: "Lena" image with additive white noise (MSE=225, Q=0.7115) Figure 22: "Lena" image with multiplicative speckle noise (MSE=225, Q=0.7224)

PAGE 36

27 Figure 23: "Lena" image with mean shift (MSE=225, Q=0.9954) Figure 24:"Lena" image with contrast stretching (MSE=225, Q=0.9839)

PAGE 37

28 Figure 25: "Lena" image with blurring (MSE=224, Q=0.2611) Figure 26: JPEG coded "Lena" image (MSE=215, Q=0.2216)

PAGE 38

29 Figure 27: JPEG2000 coded "Lena" image (MSE=225, Q=0.1837) 3.2. Definition of image quality measure Original or reference image is denoted by f(m,n) and distorted image is denoted by p(m,n). Both images have MxN pixels. In th e first step, both images will be transformed by the simple model of HVS, which models the brightness per ception and frequency selectivity (CSF). This model is presented in Figure 18 in the previous Chapter. The model is based on the properties of HVS discussed in Chapter two. The nonlinear function models brightness perception and it is given by equation (3). The 2-D filter is given by equation (16), where H(f) is given by equation (5). The result of this filtering depends on the viewing distance, the width and height and the number of pixels in input images, as it was described in Ch apter 2, so these

PAGE 39

30 parameters must also be given. The processi ng of the input images f(m,n) and p(m,n) by this model will produce images x(m,n) and y(m, n) respectively, which will be used in the computation of the quality measure. Processed images x(m, n) and y(m,n) can be used to compute weighted MSE (WMSE) or weighted SNR (WSNR). This is better than regular MSE or SNR, because it takes into account some properties of HVS, but it still does not produce satis factory results. For example these measures do not explain why we perceive contrast stretched image as very similar to the original image, despite very large error. The possible answer to this question is that HVS must use some cr iterion other than MSE when determining similarity of the distorted image to the refe rence image. By looking at the sequence of the distorted “Lena” images we see that the images with mean shift and contrast stretching are much closer to the original image than rest of the distorte d images. The main difference between these two images and the rest of them is that in these two cases there is a high degree of linear relationship between these two images and the original image, while in all other cases it is much lower. This leads to an important conclusion: HVS perceives two images to be similar when th ere is a linear relations hip between them. The degree of linear relationship between the two vectors can be me asured by correlation coefficient. If we ha ve two vectors X = [x1 x2 … xN] and Y = [y1 y2 … yN], then the correlation coefficient between them is defined as:

PAGE 40

31 N i i Y N i i X N i Y i Y N i X i X N i Y i X i XY Y X XY XYy N m x N m m y N m x N m y m x N1 1 1 2 1 2 1 1 1 ) ( 1 ) ( 1 ) )( ( 1 where (17) Correlation coefficient defined by equa tion (17) takes values between -1 and 1. It takes the value of 1 only when there is a li near relationship betwee n X and Y and in all other cases is smaller than one. The closer it is to zero, the lower degree of linear relationship exists between X and Y. Correlation coefficient will be used to create the image quality measure. Images x(m,n) and y(m,n) obtained after processing th e reference image f(m,n) and the distorted image are partitioned into 8x8 pixel blocks. For each of these blocks, the correlation coefficient between the corresponding sample s from images x(m,n) and y(m,n) is computed. If we denote the value of the correlation coefficient between x(m,n) and y(m,n) in the i-th block of data by xy(i) then we can find the average value of the correlation coefficient between the x(m,n) and y(m,n) as: L i xy avg xyi L ) ( 1_ (18) where L is the total number of data blocks Correlation coefficient between two images can be computed in one step, by applying eq uation (17) on all the pixels of the two

PAGE 41

32 images at once. This method would result in relatively high values for correlation coefficient regardless of the distortion, becau se correlation coefficient computed this way would overlook difference in details between the two images. By computing correlation coefficient locally we force it to concentrat e more on the difference in details, which are essential in image quality assessment. This is also closely related to the way in which human observer assesses image quality by scanning an image piece by piece and determining the difference between corres ponding blocks of pi xels. That is why correlation coefficient is computed as the av erage value of locally computed correlation coefficients instead of computing it in one step. Average correlation coefficient gi ven by equation (18) is a good indicator of similarity between the reference image and th e distorted image, but it has one downside: it does not differentiate between random and si gnal dependent distortion. It is important for any image quality measure to differentia te between these two types of distortion, because they have very different visual e ffect on a human observer. Random distortion has much smaller effect on image quality than the same amount of signal dependent distortion. This can be easily seen by compar ing the first three distorted “Lena” images (images with impulsive noise, additive noise and multiplicative noise) with the last three “Lena” images (images with blurring, JP EG and JPEG2000 coding). The first three images have random distortion and higher visual quality compared to last three images, which have signal dependant distor tion and lower visual quality. Since average correlation coefficien t from equation (18) does not differentiate between random and signal dependant distortion, it will treat any loss of correlation equally. Therefore, it will overestimate image degradation in the case of random noise

PAGE 42

33 and underestimate it in the case of signal dependent noise. This problem can be eliminated by computing average correlation coefficient between the image x(m,n) and the error image e(m,n), which is given by: ) ( } { ) ( ) (_n m y sign n m x n m eavg xy (19) Then we compute local correlati on coefficients block by block between images x(m,n) and e(m,n). If the local correlation coefficient for the i-th block of data is denoted by xe(i), the average correlation coefficient between images x(m,n) and e(m,n) is: L i xe avg xei L ) ( 1_ (20) This coefficient will have very small positive or negative value close to zero, when there is little or no correlation between th e images x(m,n) and e(m,n) and this will indicate random noise. On the other hand if there is a significant correlation between x(m,n) and e(m,n) the average correlation coefficient given by equation (20) will have higher absolute value, indicating signal de pendent noise. Finally, image quality measure denoted by Q is defined as: ) 15 0 3 0 | | exp( ) 15 0 3 0 | | exp( ) 15 0 3 0 | | exp( ) 15 0 3 0 | | exp( 5 0 2 1 ) ( where | | } {_ _ _ ) ( __ avg xe avg xe avg xe avg xe avg xe f avg xy avg xyf sign Qavg xe (21)

PAGE 43

34 Equation (21) needs some explanation. Quality measure (Q) will have the same sign as average correlation coefficient given by equation (18), but its magnitude will be modified value of the magnitude of the aver age correlation coefficient. The way in which this value is modified depends on the aver age correlation coefficient between the images x(m,n) and e(m,n) given by equation (20). Th is modification is obtained using function f( xe_avg) which appears in equation (21) and is shown in Figure 28. If the magnitude of the average correlation coefficient given by equati on (20) is small (close to zero), then it indicates random noise. If the va lue of this coefficient is close to one, then it indicates signal dependant noise. When the value of this coefficient is somewhere in between that indicates transition between random and signal de pendant noise. If an image is corrupted by random noise the magnitude of the quality measure will be higher than the magnitude of the average correlation coefficient given by equation (18) and lower if the image is corrupted by signal dependent noise, with tr ansition period in between. This is in accordance with the previous discussion on eff ects of random and signal dependent noise. The values of the various constants that app ear in equation (21) are determined by trial and error experiments using various images with random and signal dependant distortion.

PAGE 44

35 Figure 28: Function modeling the different eff ect of random and signal dependent noise Finally, the described algorithm can be summarized in the following four steps: 1. Given the reference image f(m,n), distor ted image p(m,n), width, height and number of pixels of the input images and viewing distance, compute images x(m,n) and y(m,n) using th e described model of HVS. 2. Compute the average correlation coefficient given by equation ( 18) as the average value of locally computed correlation co efficients between images x(m,n) and y(m,n). 3. Compute the average correlation coefficient given by equation ( 20) as the average value of the locally computed correlation coefficients between images x(m,n) and e(m,n), which is given by equation (19). 4. Find the image quality meas ure using equation (21).

PAGE 45

36 4. RESULTS 4.1. “Lena” image with vari ous types of distortion Performance of the algorithm will be il lustrated by several examples. In all examples in this chapter the following parameter values will be used: image height and width of 15 cm, viewing distance of 60 cm and image size of 512x512 pixels. Parameter from the equation (6) will depend on the original image. First example will be the sequence of “Lena” images with various types of distorti on shown in Figures 1927. For this image, we set =0.75. All images in the sequence have the same MSE value of 225 (except for JPEG coded image, which has MSE of 215), but th eir visual quality is very different. The results are shown in Table 1. Table 1: “Lena” images with various types of distortion Type of distortion MSE Quality measure (Q) Impulsive salt-pepper noise 225 0.8543 Additive white Gaussian noise 225 0.7115 Multiplicative speckle noise 225 0.7224 Mean shift 225 0.9954 Contrast stretching 225 0.9839 Blurring 224 0.2611 JPEG coding 215 0.2216 JPEG2000 coding 225 0.1837 These results are in agreement with visual quality of the corresponding images. Although we cannot define some clear criteri on for subjective quality assessment, human observer can intuitively feel wh en distorted image is more or less close to the reference

PAGE 46

37 image. Most human observers would agree with the rankings given by the proposed quality measure and in that sense these results are very reasonable. 4.2. “Goldhill” image with various types of distortion In this example, the proposed algor ithm is applied to the sequence of distorted “Goldhill” images. The types of distortion are the same as in the previous example with distorted “Lena” images, but in this case MSE values are smaller and differences in subjective visual quality betw een distorted images are smaller. In this case, we set =0.25 because this image contains smaller object s compared to “Lena” image. Again the distorted images have same or very close MSE values. These images are shown in the Figures 29-37 and results are shown in Tabl e 2. The results are in agreement with subjective visual quality of the distorted images. Table 2: “Goldhill” images with various types of distortion Type of distortion MSE Quality measure (Q) Impulsive salt-pepper noise 120 0.9353 Additive white Gaussian noise 121 0.8258 Multiplicative speckle noise 121 0.8469 Mean shift 121 0.9975 Contrast stretching 121 0.9889 Blurring 122 0.5206 JPEG coding 117 0.4076 JPEG2000 coding 122 0.3880

PAGE 47

38 Figure 29: Original "Goldhill" image Figure 30: "Goldhill" image with impulsi ve salt-pepper noise (MSE=120, Q=0.9353)

PAGE 48

39 Figure 31: "Goldhill" image with add itive white noise (MSE=121, Q=0.8258) Figure 32: "Goldhill" image with multiplic ative speckle noise (MSE=121, Q=0.8469)

PAGE 49

40 Figure 33: "Goldhill" image with mean shift (MSE =121, Q=0.9975) Figure 34: "Goldhill" image with cont rast stretching (MSE=121, Q=0.9889)

PAGE 50

41 Figure 35: "Goldhill" image with blurring (MSE=122, Q=0.5206) Figure 36: JPEG coded "Goldh ill" image (MSE=117, Q=0.4076)

PAGE 51

42 Figure 37: JPEG2000 coded "Gol dhill" image (MSE=122, Q=0.3880) 4.3. “Couple” image with va rious types of distortion In this example, the algorithm will be applied to the sequence of distorted “Couple” images. All distorted images have same or very close MSE values but their visual qualities are different. However, the differences in visual qualit ies are smaller than in the previous two examples. Fo r this image, parameter is set to 0.15 because the image has a lot of small objects which attract viewer’s attention, which means that information contained in high frequency compon ents is important. If we used =0.75 (as in the case of “Lena” image) for this image, then it w ould eliminate this information and distortion would be underestimated. On the other hand, if we used =0.15 for “Lena” image, then it would overestimate high frequenc y distortion in that case. That is why this parameter is

PAGE 52

43 not fixed. Instead its value is set depending on the content of each image. Sequence of “Couple” images is shown in Figures 3846 and results are given in Table 3. Table 3: “Couple” images with various types of distortion Type of distortion MSE Quality measure (Q) Impulsive salt-pepper noise 81 0.9553 Additive white Gaussian noise 81 0.8393 Multiplicative speckle noise 81 0.8436 Mean shift 81 0.9990 Contrast stretching 81 0.9924 Blurring 81 0.7832 JPEG coding 82 0.6046 JPEG2000 coding 82 0.5535 Figure 38: Original "Couple" image

PAGE 53

44 Figure 39: "Couple" image with impulsive salt-pepper noise (MSE=81, Q=0.9553) Figure 40: "Couple" image with ad ditive white noise (MSE=81, Q=0.8393)

PAGE 54

45 Figure 41: "Couple" image with multipli cative speckle noise (MSE=81, Q=0.8436) Figure 42: "Couple" image with mean shift (MSE=81, Q=0.9990)

PAGE 55

46 Figure 43: "Couple" image with contrast stretching (MSE=81, Q=0.9924) Figure 44: "Couple" image with blurring (MSE=82, Q=0.7832)

PAGE 56

47 Figure 45: JPEG coded "Couple" image (MSE=82, Q=0.6046) Figure 46: JPEG2000 coded "Couple" image (MSE=82, Q=0.5535)

PAGE 57

48 4.4. “Tiffany”, “Lake” and “Mandrill” images with JPEG coding In the next example, the algorithm is applied on three different JPEG coded images: “Tiffany”, “Lake” and “Mandrill”. All three im ages have similar MSE values but very different visual quality. Values for parameter are: 0.75 for “Tiffany” and “Mandrill” images and 0.25 for “Lake” image. Images are shown in Figures 47-52 and results are presented in Table 4. Table 4: “Tiffany”, “Lake” and “Mandrill” images with JPEG coding Image MSE Quality measure (Q) Tiffany 165 0.2331 Lake 168 0.4323 Mandrill 163 0.9356 Figure 47: Original "Tiffany" image

PAGE 58

49 Figure 48: JPEG coded "Tiffany" image (MSE=165, Q=0.2331) Figure 49: Original "Lake" image

PAGE 59

50 Figure 50: JPEG coded "Lake" image (MSE=168, Q=0.4323) Figure 51: Original "Mandrill" image

PAGE 60

51 Figure 52: JPEG c oded "Mandrill" image (MSE=163, Q=0.9356) 4.5. “Woman”, “Man” and “Bar bara” images with blurring Next example shows performance of the algorithm on three different blurred images: “Woman”, “Man” and “Barbara”. Again, all th ree images have iden tical MSE values, but very different visual quality. Values of parameter are: 0.75 for “Woman” image and 0.5 for “Man” and “Barbara” images. These images are shown in Figures 53-58 and results are presented in Table 5. Table 5: “Woman”, “Man” and “B arbara” images with blurring Image MSE Quality measure (Q) Woman 200 0.1456 Man 200 0.4074 Barbara 200 0.7855

PAGE 61

52 Figure 53: Original "Woman" image Figure 54: "Woman" image with blurring (MSE=200, Q=0.1456)

PAGE 62

53 Figure 55: Origin al "Man" image Figure 56: "Man" image with blurring (MSE=200, Q=0.4074)

PAGE 63

54 Figure 57: Original "Barbara" image Figure 58: "Barbara" image with blurring (MSE=200, Q=0.7855)

PAGE 64

55 In all examples presented here the proposed quality measure ranks images according to their visual quality. This is in contrast to MSE values which can be identical even when the difference in the extent of distortion is very large. 4.6. “Lena” image with inverted gray scale The proposed quality measure has anot her interesting property. As can be seen from equation (21), this quality measure can take valu es that range from -1 to 1. In other words it can take positive and negative values. It takes negative values when the average correlation coefficient given by equation (18) is negative. This happens when one of the input images is inverted, which means the br ight area becomes dark and vice versa. In this example the reference image is the orig inal “Lena” image and distorted images are inverted original “Lena” image and inverted “Lena” images with impulsive and additive noise. If the original image is f(m,n), th en the corresponding inverted image can be computed as 255-f(m,n). These images are shown in Figures 59-61 and the results are given in Table 6. Table 6: “Lena” images with inverted grayscale Image MSE Quality measure (Q) Inverted “Lena” image 9258 -1 Inverted “Lena” image with impulsive noise 9369 -0.8543 Inverted “Lena” image with additive noise 9485 -0.7115

PAGE 65

56 Figure 59: Inverted "Lena" image (MSE=9258, Q=-1) Figure 60: Inverted "Lena" image with impulsive salt-pepper noise (MSE=9369, Q=-0.8543)

PAGE 66

57 Figure 61: Inverted "Lena" image with additive white noise (MSE=9485, Q=-0.7115) Quality measure values for inverted images with impulsive and additive noise have the same magnitude as the values obtained for the corresponding non-inverted images, but they have negative sign. This is in a sense related to what we see: images with the same structure but inverted grayscale. MS E produces only very large values, which is another illustration that MSE is not related to the way we perceive image similarity. 4.7. “Lena” image with blurring and additive noise Next example illustrates the perfor mance of the proposed quality measure on the sequence of “Lena” images distorted by two so urces of degradation: blurring caused by low-pass filtering and additive white Gaussian noise (zero-mean and standard deviation of ). Images contain some amount of noise and some blurring in different proportions.

PAGE 67

58 First image contains only blurring, second imag e contains less blurring and some noise, third image contains less blurring and more noise than the second etc. Last image contains only noise. All images in the sequence have the same MSE. These images are shown in Figures 62-67 and result s are shown in Table 7. Table 7: “Lena” images with blurring and additive noise Image MSE Quality measure (Q) blurring only 225 0.2031 blurring and additive noise ( = 6.8) 225 0.2625 blurring and additive noise ( = 9.4) 225 0.3549 blurring and additive noise ( = 11.5) 225 0.5411 blurring and additive noise ( = 13.3) 225 0.6637 additive noise ( = 15) 225 0.7108 Figure 62: "Lena" image with blurring (MSE=225, Q=0.2031)

PAGE 68

59 Figure 63: "Lena" image with blurring and additive noise ( =6.8), (MSE=225, Q=0.2625) Figure 64: "Lena" image with blurring and additive noise ( =9.4), (MSE=225, Q=0.3547)

PAGE 69

60 Figure 65: "Lena" image with blurring and additive noise ( =11.5), (MSE=225, Q=0.5411) Figure 66: "Lena" image with blurring and additive noise ( =13.3), (MSE=225, Q=0.6637)

PAGE 70

61 Figure 67: "Lena" imag e with additive noise ( =15), (MSE=225, Q=0.7108) 4.8. Comparison of JPEG and JPEG2000 coding using “Lena” image In the next example, the proposed quality measure will be used to compare two algorithms for image coding: JPEG and JPEG 2000. In both of these algorithms image intensities are transformed into some coeffi cients and those coefficients are coded and transmitted. JPEG uses discrete cosine tran sform and JPEG2000 uses wavelet transform, so they produce different types of degradat ion. The performance of these two algorithms is compared at several coding rates and quality measure and MSE are computed for each case. These images are shown in Figures 68-81 and re sults are shown in Table 8. MSE values for JPEG2000 coded im ages are always lower than the corresponding values for JPEG coded images at the same bit rate. However, quality measure values for

PAGE 71

62 bit rates greater than 0.5 bits/pixel are sli ghtly higher for JPEG coded images than those for JPEG2000 images. By careful inspection of these images it can be seen that JPEG images at these bit rates ar e slightly sharper compared to JPEG2000 coded image which contain some blurring. When th e bit rate is lower than 0.5 bits/pixel, quality measure decreases rapidly for JPEG coding, but very slowly for JPEG2000 coding. This is in agreement with MSE values and visual quality which deteriorates more quickly for JPEG coding than for JPEG2000 coding. Table 8: Comparison of JPEG and JPEG2000 coding JPEG JPEG2000 Rate[bits/pixel] MSE Q MSE Q 1.1561 9.11 0.9628 6.45 0.9255 0.5501 19.97 0.8339 13.64 0.8104 0.3647 32.85 0.6667 20.75 0.7289 0.2460 59.17 0.4806 31.27 0.6319 0.1747 120.11 0.3283 44.85 0.5868 0.1428 218.60 0.2290 54.51 0.5153 0.1129 1030.40 0.0441 67.56 0.4873

PAGE 72

63 Figure 68: JPEG coding at 1.1561 bits/pixel (MSE=9.11, Q=0.9628) Figure 69: JPEG2000 coding at 1.1561 bits/pixel (MSE=6.45, Q=0.9255)

PAGE 73

64 Figure 70: JPEG coding at 0.5501 bits/pixel (MSE=19.97, Q=0.8339) Figure 71: JPEG2000 coding at 0.5501 bits/pixel (MSE=13.64, Q=0.8104)

PAGE 74

65 Figure 72: JPEG coding at 0.3647 bits/pixel (MSE=32.85, Q=0.6667) Figure 73: JPEG2000 coding at 0.3647 bits/pixel (MSE=20.75, Q=0.7289)

PAGE 75

66 Figure 74: JPEG coding at 0.246 bits/pixel (MSE=59.17, Q=0.4806) Figure 75: JPEG2000 coding at 0.246 bits/pixel (MSE=31.27, Q=0.6319)

PAGE 76

67 Figure 76: JPEG coding at 0.1747 bits/pixel (MSE=120.11, Q=0.3283) Figure 77: JPEG coding at 0.1747 bits/pixel (MSE=44.85, Q=0.5868)

PAGE 77

68 Figure 78: JPEG coding at 0.1428 bits/pixel (MSE=218.6, Q=0.2290) Figure 79: JPEG2000 coding at 0.1428 bits/pixel (MSE=54.51, Q=0.5153)

PAGE 78

69 Figure 80: JPEG coding at 0.1129 bits/pixel (MSE=1030.4, Q=0.0441) Figure 81: JPEG coding at 0.1129 bits/pixel (MSE=67.56, Q=0.4873)

PAGE 79

70 4.9. Performance of quality m easure on two different images In the last example in this chapter, the algorithm is applied on two totally different images, where one of them is the reference im age and the other is the distorted image. In this case there is no similarity at all be tween the two images and the quality measure takes values very close to zero. These values are much smaller than any of the values obtained when the distorted image is a degr aded version of the reference image. The results are given in Table 9. Table 9: Performance of quality measure on two different images Reference image Distorted image Quality measure (Q) “Lena” “Man” -0.00002 “Woman” “Barbara” -0.00007 “Goldhill” “Couple” -0.0009 “Tiffany” “Lena” 0.0005 “Lake” “Mandrill” 0.0004

PAGE 80

71 5. CONCLUSION An algorithm for image quality assessment has been described. The algorithm uses a simple HVS model, which is used to process input images. CSF used in this model is not fixed; it has one user-defined parameter, wh ich controls attenuation at high frequencies. This way it is possible to get better resu lts than in the case when CSF with fixed parameters is used. This is due to the fact that HVS treats very di fferently high frequency components present in the original image than those of noise. Two processed images are used to compute average correlation coeffici ent, which measures the degree of linear relationship between two images. This way we take into account structural similarity between two images, which is ignored by MS E-based measures. Finally, image quality measure is computed as the average correla tion coefficient between two input images modified by the average correlation coefficien t between original image and error image. This way we differentiate between random a nd signal dependant distortion, which have different impact on a human observer. Image quality measure described here depends on viewing distance and width, height and number of pixels of input images. This is reasonable because image perception by HVS also depends on these same parameters. Details of an image viewed from a close distance may not be visible when the same image is viewed from a greater viewing distance. This is modeled by CSF. As the viewing distance increases sampling

PAGE 81

72 frequencies in samples/degree given by equa tion (8) also increase which moves image spectrum into higher frequencies. Since CSF has greater attenuation at higher frequencies this will suppress spectral components at high frequencies whic h are responsible for image details. The results presented here are obtained for the following parameters: viewing distance 60 cm, width of images 15 cm height of images 15 cm and all images are 512x512 pixels. Images in this document are not shown in their normal size. Their size was reduced so that two images can fit into one page, whic h significantly reduced the number of pages. Therefore, their subjective visual quality cannot be judged based on images shown here, it can only be judged when images are view ed in their normal size and when display device can display 256 different gray levels. If the display device cannot display 256 gray levels some information from the input images will be discarded and some quantization noise will be introduced. This is particularly significant if the difference between the input images is small, because then this difference can be masked by the quantization noise introduced by the display device. Proposed image quality measure performs reasonably well. The examples presented here demonstrate that this measure ranks images according to their visual quality in cases when MSE based measures fail to do that. Howe ver, subjective evaluatio n is still the best way for image quality assessment. No algorit hm, no matter how sophisticated, can model something as complex as HVS. Future research in this area should focus on finding better models for brightness perception, which will include characteri stics of display device. Another possible improvement is creating a CSF, wh ich models HVS more accurately.

PAGE 82

73 REFERENCES 1. S. Daly, “The visible difference predicto r: an algorithm for assessment of image fidelity,” in Digital images and human vision, A. B. Watson (ed.), MIT Press, Cambridge, MA, pp. 179-206, 1993. 2. R. J. Safranek and J. D. Johnston, “A perceptually tuned subba nd image coder with image dependant quantization and post-qua ntization data comp ression,” in Proc. International Conference on Acoustics, Sp eech, and Signal Processing (ICASSP),Vol. 3, pp. 1945-1948, 1989. 3. N. Damera-Venkata, T. D. Kite, W. Geisler, B. L. Evans and A. C. Bovik, "Image quality assessment based on a degradati on model," IEEE Transaction on Image Processing, Vol. 9, No. 4, pp. 630-650, April 2000. 4. T. N. Pappas and R. J. Safranek, "Perceptua l criteria for image quality evaluation," in Handbook of image and video processing, Academic Press, May 2000. 5. T. N. Pappas, T. A. Michel and R. O. Hinds, “Supra-threshold perceptual image coding,” in Proc. Internati onal Conf. on Image Processing (ICIP), Vol. I, pp. 237-240, 1996. 6. Z. Wang, A. C. Bovik and L. Lu, "Why is image quality assessment so difficult?," in Proc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vol. 4, pp. 3313-3316, 2002. 7. Z. Wang and A. C. Bovik, "Universal image quality i ndex," IEEE Signal Processing Letters, Vol. 9, No. 3, pp. 81-84, March 2002. 8. J. S. Lim: "Two-dimensional signal and image processing," Prenti ce-Hall, Engelwood Cliffs, New Jersey, 1990. 9. J. L. Mannos and D. J. Sakrison, “The e ffects of a visual fide lity criterion on the encoding of images,” IEEE Transactions on Information Theory, Vol. IT-20, No. 4, pp. 525-536, July 1974. 10. T. N. Pappas and D. L. Neuhoff, “Leas t-squares model based halftoning,” IEEE Transactions on Image Processing, Vol. 8, No. 8, pp. 1102-1116, August 1999.


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001416911
003 fts
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 031010s2003 flua sbm s000|0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0000049
035
(OCoLC)52786340
9
AJJ4763
b SE
SFE0000049
040
FHM
c FHM
049
FHME
090
TK145
1 100
Ivkovic, Goran.
3 245
An algorithm for image quality assessment
h [electronic resource] /
by Goran Ivkovic.
260
[Tampa, Fla.] :
University of South Florida,
2003.
502
Thesis (M.S.E.E.)--University of South Florida, 2003.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 82 pages.
520
ABSTRACT: Image quality measures are used to optimize image processing algorithms and evaluate their performances. The only reliable way to assess image quality is subjective evaluation by human observers, where the mean value of their scores is used as the quality measure. This is known as mean opinion score (MOS). In addition to this measure there are various objective (quantitative) measures. Most widely used quantitative measures are: mean squared error (MSE), peak signal to noise ratio (PSNR) and signal to noise ratio (SNR). Since these simple measures do not always produce results that are in agreement with subjective evaluation, many other quality measures have been proposed. They are mostly various modifications of MSE, which try to take into account some properties of human visual system (HVS) such as nonlinear character of brightness perception, contrast sensitivity function (CSF) and texture masking. In these approaches quality measure is computed as MSE of input image intensities or frequency domain coefficients obtained after some transform (DFT, DCT etc.), weighted by some coefficients which account for the mentioned properties of HVS. These measures have some advantages over MSE, but their ability to predict image quality is still limited. A different approach is presented here. Quality measure proposed here uses simple model of HVS, which has one user-defined parameter, whose value depends on the reference image. This quality measure is based on the average value of locally computed correlation coefficients. This takes into account structural similarity between original and distorted images, which cannot be measured by MSE or any kind of weighted MSE. The proposed measure also differentiates between random and signal dependant distortion, because these two have different effect on human observer. This is achieved by computing the average correlation coefficient between reference image and error image. Performance of the proposed quality measure is illustrated by examples involving images with different types of degradation.
590
Adviser: SANKAR, RAVI
653
image coding.
image quality measure.
human visual system.
contrast sensitivity function.
brightness perception.
0 690
Dissertations, Academic
z USF
x ELECTRICAL ENGINEERING
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.49