International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

OBTAINING SUPER-RESOLUTION IMAGES BY COMBINING LOW-RESOLUTION IMAGES WITH HIGH-FREQUENCY INFORMATION DERIVEDFROM TRAINING IMAGES
Emil Bilgazyev , Nikolaos Tsekos , and Ernst Leiss
1,2,3
1

Department of Computer Science, University of Houston, TX, USA

2

eismailov2@uh.edu,

ntsekos@cs.uh.edu,

eleiss@uh.edu

ABSTRACT
In this paper, we propose a new algorithm to estimate a super-resolution image from a given low-resolution image, by adding high-frequency information that is extracted from natural high-resolution images in the training dataset. The selection of the high-frequency information from the training dataset is accomplished in two steps, a nearest-neighbor search algorithm is used to select the closest images from the training dataset, which can be implemented in the GPU, and a sparse-representation algorithm is used to estimate a weight parameter to combine the high-frequency information of selected images. This simple but very powerful super-resolution algorithm can produce state-of-the-art results. Qualitatively and quantitatively, we demonstrate that the proposed algorithm outperforms existing state-of-the-art super-resolution algorithms.

KEYWORDS
Super-resolution, face recognition, sparse representation.

1. INTRODUCTION
Recent advances in electronics, sensors, and optics have led to a widespread availability of video-based surveillance and monitoring systems. Some of the imaging devices, such as cameras, camcorders, and surveillance cameras, have limited achievable resolution due to factors such as quality of lenses, limited number of sensors in the camera, etc. Increasing the quality of lenses or the number of sensors in the camera will also increase the cost of the device; in some cases the desired resolution may be still not achievable with the current technology. However, many applications, ranging from security to broadcasting, are driving the need for higher-resolution images or videos for better visualization [1]. The idea behind super-resolution is to enhance the low-resolution input image, such that the spatial-resolution (total number of independent pixels within the image) as well as pixel-resolution (total number of pixels) are improved.

DOI : 10.5121/ijcsit.2013.5202

19

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

In this paper, we propose a new approach to estimate super-resolution by combining a given low-resolution image with high-frequency information obtained from training images (Fig. 1). The nearest-neighbor-search algorithm is used to select closest images from the training dataset, and a sparse representation algorithm is used to estimate a weight parameter to combine the high-frequencyinformation of selected images. The main motivation of our approach is that the high-frequency information helps to obtain sharp edges on the reconstructed images (see Fig. 1).

(a) (b) (c) Figure 1: Depiction of (a) the super-resolution image obtained by combining (b) a given low-resolution image and (c) high-frequency information estimated from a natural high-resolution training dataset.

The rest of the paper is organized as follow: Previous work is presented in Section 2, a description of our proposed method is presented in Section 3, the implementation details are presented in Section 4, and experimental results of the proposed algorithm as well as other algorithms are presented in Section 5. Finally, Section 6 summarizes our findings and concludes the paper.

2. PREVIOUS WORK
In this section, we briefly review existing techniques for super-resolution of low-resolution images for general and domain-specific purposes. In recent years, several methods have been proposed that address the issue of image resolution. Existing super-resolution (SR) algorithms can be classified into two classes: multi-frame-based and example-based algorithms [8]. Multi-frame-based methods compute an high-resolution (HR) image from a set of low-resolution (LR) images from any domain [6]. The key assumption of multi-frame-based super-resolution methods is that the set of input LR images overlap and each LR image contains additional information than other LR images. Then multi-frame-based SR methods combine these sets of LR images into one image so that all information is contained in a single output SR image. Additionally, these methods perform super-resolution with the general goal of improving the quality of the image so that the resulting higher-resolution image is also visually pleasing. The example-based methods compute an HR counterpart of a single LR image from a known domain [2, 13, 18, 10, 14]. These methods learn observed information targeted to a specific domain and thus, can exploit prior knowledge to obtain superior results specific to that domain. Our approach belongs to this category, where we use a training database to improve reconstruction output. Moreover, the domain-specific SR methods targeting the same domain differ considerably from each other in the way they model and apply a priori knowledge about natural images. Yang et al. [22] introduced a method to reconstruct SR images using a sparse representation of the input LR images. However, the performance of these example-based SR methods degrades rapidly if the magnification factor is more than 2. In addition, the performance of these SR methods is highly dependent on the size of the training database.
20

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

Freeman et al. [8] proposed an example-based learning strategy that applies to generic images where the LR to HR relationship is learned using a Markov Random Field (MRF). Sun et al. [17] extended this approach by using the Primal Sketch priors to reconstruct edge regions and corners by deblurring them. The main drawback of these methods is that they require a large database of LR and HR image pairs to train the MRF. Chang et al. [3] used the Locally Linear Embedding (LLE)

Figure 2: Depiction of the pipeline for the proposed super-resolution algorithm.

manifold learning approach to map the the local geometry of HR images to LR images with the assumption that the manifolds between LR and HR images are similar. In addition, they reconstructed a SR image using K neighbors. However, the manifold between the synthetic LR that is generated from HR images is not similar to the manifold of real scenario LR images, which are captured under different environments and camera settings. Also, using a fixed number of neighbors to reconstruct an SR image usually results in blurring effects such as artifacts in the edges, due to over- or under-fitting. Another approach is derived from a multi-frame based approach to reconstruct an SR image from a single LR image [7, 12, 19]. These approaches learn the co-occurrence of a patch within the image where the correspondence between LR and HR is predicted. These approaches cannot be used to reconstruct a SR image from a single LR facial image, due to the limited number of similar patches within a facial image. SR reconstruction based on wavelet analysis has been shown to be well suited for reconstruction, denoising and deblurring, and has been used in a variety of application domains including biomedical [11], biometrics [5], and astronomy [20]. In addition, it provides an accurate and sparse representation of images that consist of smooth regions with isolated abrupt changes [16]. In our method, we propose to take advantage of the wavelet decomposition-based approach in conjunction with compressed sensing techniques to improve the quality of the super-resolution output.

21

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

3. METHODOLOGY
Table 1. Notations used in this paper

Symbols X

Description Collection of training images

Xi
xi , j

ith training image ( R m n )

j th patch of ith image ( R k l )

threshold value sparse representation of ith patch

i
.
p

l p -norm
Dictionary forward and inverse wavelet transforms high- and low-frequencies of image Concatenation or vectors or matrices

D W , W 1 ,

[]

k l Let X i R m n be the ith image of the training dataset X = {X i : i = 0...N} , and x i , j R

be the j

th

patch of an image X i = { xi , j : j = 0...M } . The wavelet transform of an image patch

x will return low- and high-frequency information:

W ( x ) = [ ( x ), ( x )] ,
(1)

where W is the forward wavelet transform, ( x ) is the low-frequency information, and ( x) is the high-frequency information of an image patch x . Taking the inverse wavelet transform of high- and low-frequency information of original image (without any processing on them) will result in the original image:

x = W 1 ([ ( x), ( x)]) ,
where W
1

(2)

is the inverse wavelet transform. If we use Haar wavelet transform with its

2 (nonquadratic-mirror-filter), then the low-frequency coefficients being 0.5 instead of information of an image x will actually be a low-resolution version of an image x , where four neighboring pixels are averaged; in other words, it is similar to down-sampling an image x by a factor of 2 with nearest-neighbor interpolation, and the high-frequency information ( x) of an image x will be similar to the horizontal, vertical and diagonal gradients of x .
Assume that, for a given low-resolution image patch yi which is the ith patch of an image y , we can find a similar patch x j = { j = 0 NM } from the natural image patches, then by combining

yi with the high-frequency information ( x j ) of a high-resolution patch x j , and taking the

22

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

inverse wavelet transform, we will get the super-resolution y (see Fig. 1):

y i* = W 1 ([ y i , ( x j )]) ,

yi (x j ) 2 0 ,

(3)

where 0 is a small nonnegative value. It is not guaranteed that we will always find an x j such that

yi ( x j ) 0 , thus, we 2

introduce an approach to estimate a few closest low-resolution patches ( ( x j ) ) from the training dataset and then estimate a weight for each patch ( x j ) which will be used to combine high-frequency information of the training patches ( x j ) . To find closest matches to the low-resolution input patch yi , we use a nearest-neighbor search algorithm:

c = {ci : ci = yi ( xc ) 1 , xc X } ,
i 2 i

(4)

where c is a vector containing the indexes ( ci ) of training patches of the closest matches to input patch yi , and

1 is the radius threshold of a nearest-neighbor search. After selecting the closest

matches to yi , we build two dictionaries from the selected patches x j ; the first dictionary will be the joint of low-frequency information of training patches ( x j ) where it will be used to estimate a weight parameter, and the second dictionary will be the joint of high-frequency information of training patches ( x j ) :

Di = { ( x j ) : j c} , Di = { ( x j ) : j c} .

(5)

We use a sparse representation algorithm [21] to estimate the weight parameter. The sparse-representation i of an input image patch yi with respect to the dictionary D i , is used as a weight for fusion of the high-frequency information of training patches ( Di ):

i = argmin yi Di i 2 + i 1.
i

(6)

The sparse representation algorithm (Eq. 6) tries to estimate yi by fusing a few atoms (columns) of the dictionary D i , by assigning non-zero weights to these atoms. The result will be the sparse-representation i , which has only a few non-zero elements. In other words, the input image patch yi can be represented by combining a few atoms of D i ( yi Di i ) with a weight parameter i ; similarly, the high-frequency information of training patches Di can also be
23

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

combined with the same weight parameter i , to estimate the unknown high-frequency information of an input image patch yi :

y i* = W

([y i , D i i ]) ,

(7)

where y i* is the output (super-resolution) image patch, and W 1 is the inverse wavelet transform. Figure 2 depicts the pipeline for the proposed algorithm. For the training step, from high-resolution training images we extract patches, then we compute low-frequency (will become low-resolution training image patches) and high-frequency information for each patch in the training dataset. For the reconstruction step, given an input low-resolution image y , we extract a patch yi , find nearest neighbors c within the given radius

1 (this can be speeded-up usinga GPU), then from selected

neighbors c , we construct low-frequency D i and high-frequency dictionaries D i , where the low-frequency dictionary is used to estimate the sparse representation i of input low-resolution patch yi with respect to the selected neighbors, and the high-frequency dictionary Di will be used to fuse its atoms (columns) with a weight parameter, where the sparse representation i will be used as a weight parameter. Finally, by taking the inverse wavelet transform (W ) of a given low-resolution image patch yi with fused high-frequency information, we will get the super-resolution patch y . Iteratively repeating the reconstruction step (red-dotted block in Fig. 2) for each patch in the low-resolution image y , we will obtain the super-resolution image y .
* *
1

4. IMPLEMENTATION DETAILS
In this section we will explain the implementation details. As we have pointed out in Sec. 3, we k l extract patches for each training image X i = { xi , j : j = 0...M } with xi , j R . The number

M depends on the window function, which determines how we would like to select the patches. There are two ways to select the patches from the image; one is by selecting distinct patches from an image, where two consecutive patches dont overlap, and another is by selecting overlapped patches (sliding window), where two consecutive patches are overlapped. Since the l2 -norm in nearest-neighbor search is sensitive to the shift, we slide the window by one pixel in horizontal or vertical direction, where the two consecutive patches will overlap each other by ( k 1) l or

k (l 1) , where xi , j Rkl . To store these patches we will require an enormous amount of storage space N ( m k ) ( n l ) k l , where N is the number of training images and
has a resolution of 10001000 pixels, to store the patches of 40 40 , we will require 1.34TB of storage space, which would be inefficient and computationally expensive.To reduce the number of patches, we removed patches which dont contain any gradients, or contain very few gradients,

X i R m n . For example, if we have 1000 images natural images in the training dataset, and each

xi , j

2 2

2 where is the sum of gradients along the vertical and horizontal directions (

24

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

xi , j =

xi , j x

xi , j y

), and

2 is the threshold value to filter out the patches with less gradient

variation. Similarly, we calculate the gradients on input low-resolution patches ( yi ), and if they are below the threshold 2 , we upsample them using bicubic interpolation, where no super-resolution reconstruction will be performed on that patch. To improve the computation speed, the nearest-neighbor search can be calculated in the GPU, and since all given low-resolution patches are calculated independently of each other, multi-threaded processing can be used for each super-resolution patch reconstruction. In the wavelet transform, for low-pass filter and high-pass filter we used [ 0.5 , 0.5 ] and [ 0.5, 0.5 ], where 2D filters for wavelet transform are created from them. These filters are not quadratic-mirror-filters (nonorthogonal), thus, during the inverse wavelet transform we need to multiply the output by 4 . The reason for choosing these values for the filters is, the low-frequency information (analysis part) of the forward wavelet transform will be the same as down-sampling the signal by a factor of 2 with nearest neighbor interpolation, which is used in the nearest-neighbor search.During the experiments, all color images are converted to YCbCr , where only the luminance component ( Y ) is used.For the display, the blue- and red-difference chroma components ( Cb and Cr ) of an input low-resolution image are up-sampled and combined with the super-resolution image to obtain the color image y (Fig. 2). Note that we can reduce the storage space for the patches to zero, by extracting the patches of the training images during reconstruction. This can be accomplished by changing the neighbor-search algorithm, and can be implemented in GPU. During the neighbor-searching, each GPU thread will be assigned to extract low-frequency ( xl , j ) and high-frequency information ( x l , j ) at an assigned position j of a training image X l ; we compute the distance to the input low-resolution image patch yi , and if the distance is less than the threshold used to construct D i :
*

2 , then the GPU thread will return

the high-frequency information ( xl , j ) , where the returned high-frequency information will be

2 2

Di = { (ci ) : ci = y i (x c )
i

1, x c X } .
i

(8)

As a threshold (radius) value for nearest-neighbor search algorithm we used 0.5 for natural images, and 0.3 for facial images. Both low-frequency information of training image and input image patches are normalized before calculating euclidean distance. We selected these values experimentally, where at these values we get highest SNR and lowest MSE. As we know that that the euclidean distance ( in nearest-neighbor search) is sensitive to noise, but in our approach, our main goal is to reduce the number of training patches which are close to input patch. Thus, we take a higher the threshold value for nearest-neighbor search, where we select closest matches, then the sparse representation is performed on them. Note that, sparse representation estimation (Eq. 6) tends to estimate input patch from training patches, where noise is taken care of [14].Reducing the storage space will slightly increase the super-resolution reconstruction time, since the wavelet transform will be computed during the reconstruction.
25

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

5. EXPERIMENT RESULTS
We performed experiments on a variety of images to test the performance of our approach (HSR) as well as other super-resolution algorithms: BCI [1], SSR [22] and MSR [8]. We conducted two types of experiments. For the first one, we performed the experiment on the Berkeley Segmentation Dataset 500 [15]. It contains natural images, where the natural images are divided into two groups; the first group of images (Fig. 3(a)) are used to train super-resolution algorithms (except BCI), and the second group images (Fig. 3(b)) are used to test the performance of the super-resolution algorithms. To measure the performance of the algorithms, we use mean-square-error (MSE) and signal-to-noise ratio (SNR) as a distance metric. These algorithms measure the difference between the ground truth and the reconstructed images. The second type of experiment is performed on facial images (Fig. 4), where face recognition system is used as a distance metric to demonstrate the performance of the super-resolution algorithms.

(a) (b) Figure 3: Depiction of Berkeley Segmentation Dataset 500 images used for (a) training and (b) testing the super-resolution algorithms.

(a) (b) Figure 4: Depiction of facial images used for (a) training and (b) testing the super-resolution algorithms.

26

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

5.1 Results on Natural Images

In Figure 5, we show the output of the proposed super-resolution algorithm, BCI, SSR, and MSR. The red rectangle is zoomed-in and displayed in Figure 6. In this figure we focus on the effect ofsuper-resolution algorithms on low-level patterns (fur of the bear). Most of the super-resolution algorithms tend to improve the sharpness of the edges along the border of the objects, which looks good to human eyes, and the low-level patterns are ignored. One can see that the output of BCI is smooth (Fig. 5(b)), and from the zoomed-in region (Fig. 6(b)) it can be noticed that the edges along the border of the object are smoothed, and similarly, the pattern inside the regions is also smooth. This is because BCI interpolates the neighboring pixel values in the lower-resolution to introduce a new pixel value in the higher-resolution. This is the same as taking the inverse wavelet transform of a given low-resolution image with its high-frequency information being zero, thus the reconstructed image will not contain any sharp edges. The result of MSR has sharp edges, however, it contains block artifacts (Fig. 5(c)). One can see that the edges around the border of an object are sharp, but the patterns inside the region are smoothed, and block artifact are introduced (Fig. 6(c)). On the other hand, the result of SSR doesnt contain sharp edges along the border of the object, but it contains sharper patterns compared to BCI and MSR (Fig. 5(d)). The result of the proposed super-resolution algorithm has sharp edges, sharp patterns, as well as fewer artifacts compared to other methods (Fig. 5(e) and Fig. 6(e)), and visually it looks more similar to the ground truth image (Fig. 5(f) and Fig. 6(f)). Figure 7 shows the performance of the super-resolution algorithms on a different image with fewer patterns. One can see that the output of the BCI is still smooth along the borders, and inside the region it is clearer. The output of MSR looks better for the images with fewer patterns, where it tends to reconstruct the edges along the borders.

(a)

(b)

(c)

(d) (e ) (f) Figure 5: Depiction of low-resolution, super-resolution and original high-resolution images. (a) Low-resolution image, (b) output of BCI, (c) output of SSR, (d) output of MSR, (e) output of proposed algorithm, and (f) original high-resolution image. The solid rectangle boxes in red color represents the regions that is magnified and displayed in Figure 5 for better visualization. One can see that the output of the proposed algorithm has sharper patterns compared to other SR algorithms. 27

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6: Depiction of a region (red rectangle in Figure 3) for (a) low-resolution image, output of (b) BCI, (c) SSR, (d) MSR, (e) proposed algorithm, and (f) original high-resolution image. Notice that the the proposed algorithm has sharper patterns compared to other SR algorithms.

(a)

(b)

(c)

(d)

(e )

(f)

Figure 7: Depiction of low-resolution, super-resolution and original high-resolution images. (a) Low-resolution image, (b) output of BCI, (c) output of SSR, (d) output of MSR, (e) output of proposed algorithm, and (f) original high-resolution image. The solid rectangle boxes in yellow and red colors represent the regions that were magnified and displayed on the right side of each image for better visualization. One can see that the output of the proposed algorithm has better visual quality compared to other SR algorithms. 28

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

In the output of SSR, one can see that the edges on the borders are smooth, and inside the regions it has ringing artifacts. The SSR algorithm builds dictionaries from high-resolution and low-resolution image patches by reducing the number of atoms (columns) of the dictionaries under a constraint that these dictionaries can represent the image patches in the training dataset with minimal difference. This is similar to compressing or dimension reduction, where we try to preserve the structure of the signal, not the details of the signal, and sometimes we get artifacts during the reconstruction1. We also computed the average SNR and MSE to quantitatively measure the performance of the super-resolution algorithms. Table 5.1 depicts the average SNR and MSE values for BCI, MSR, SSR, and HSR. Notice that the proposed algorithm has the highest signal-to-noise ratio and the lowest difference mean-square-error.
Table 5.1: Experimental Results

Dist. Metric \ SR Algorithms SNR ( dB ) MSE

BCI 23.08 5.45

SSR 24.76 5.81

MSR 18.46 12.01

HSR 25.34 3.95

5.2. Results on Facial Images

We conducted experiments on surveillance camera facial images (SCFace)[9]. This database contains 4,160 static images from 130 subjects. The images were acquired in an uncontrolled indoor environment using five surveillance cameras of various qualities and ranges. For each of these cameras, one image from each subject at three distances, 4.2 m , 2.6 m , and 1.0 m was acquired. Another set of images was acquired by a mug shot camera. Nine images per subject provide nine discrete images ranging from left to right profile in equal steps of 22.5 degrees, including a frontal mug-shot image at 0 degrees. The database contains images in visible and infrared spectrum. Images from different quality cameras mimic real-world conditions. The high-resolution images are used as a gallery (Fig. 4(a)), while the images captured by a camera with visible light spectrum from a 4.2 m distance are used as a probe (Fig. 4(b)). Since the SCFace dataset consists of two types of images, high-resolution images and surveillance images, we used the high-resolution images to train SR methods and the surveillance images as a probe. We used Sparse Representation based Face Recognition proposed by Wright et . al [21], to test the performance of the super-resolution algorithms. It has been proven that the performance of the face recognition systems relies on low-level information (high-frequency information) of the facial images [4]. The high-level information, which is the structure of the face, affects less the performance of face recognition systems compared to low-level information, unless we compare human faces with other objects such as monkey, lion, car, etc., where the structures between them
1

The lower-frequencies of the signal affect more the difference between original and reconstructed signals, compared to the higher-frequencies. For example, if we remove the DC component (0 Hz) from one of the signals, original or reconstructed, the difference between them will be very large. Thus keeping the lower-frequencies of the signal helps to preserve the structure and have minimal difference between the original and reconstructed signals.

29

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013

are very different. Most of the human faces have similar structures: two eyes, one nose, two eyebrows, etc., and in the low-resolution facial images, the edges (high-frequency information) around the eyes, eyebrows, nose, mouth, etc., are lost which decreases the performance of face recognition systems [21]. Even for humans it is very difficult to recognize a person from a low-resolution image (see Fig. 4). Figure 8 depicts the given low-resolution images, and the output of super-resolution images. The rank-1 face recognition accuracy for LR, BCI, SSR, MSR, and our proposed algorithms are: 2%, 18%, 13%, 16%, and 21%. Overall, the face recognition accuracy is low, but compared to the face recognition performance on the low-resolution images, we can summarize that the super-resolution algorithms can improve the recognition accuracy.

6. CONCLUSION
We have proposed a novel approach to reconstruct super-resolution images for better visual quality as well as for better face recognition purposes, which also can be applied to other fields. We presented a sparse representation-based SR method to recover the high-frequency components of an

(a)

(b)

(c)

(d)

Figure 8: Depiction of LR and output of SR images. (a) Low-resolution image, output of (b) BCI, (c) SSR, and (d) proposed method. For this experiment we used a patch size of 10 10 pixels; thus when we increase the patch size we introduce ringing artifacts, which can be seen in the reconstructed image (d). Quantitatively, in terms of face recognition our proposed super-resolution algorithm outperforms other super-resolution algorithms.

SR image. We demonstrated the superiority of our methods over existing state-of-the-art super-resolution methods for the task of face recognition in low-resolution images obtained from real world surveillance data, as well as better performance in terms of MSE and SNR. We conclude that by having more than one training image for the subject we can significantly improve the visual quality of the proposed super-resolution output, as well as the recognition accuracy.

REFERENCES
[1] [2] [3] M. Ao, D. Yi, Z. Lei, and S. Z. Li. Handbook of remote biometrics, chapter Face Recognition at a Distance:System Issues, pages 155167. Springer London, 2009. S. Baker and T. Kanade. Hallucinating faces. In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pages 8388, Grenoble, France, March 28-30, 2002. H. Chang, D. Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding. In Proc. IEEEInternational Conference on Computer Vision and Pattern Recognition, pages 275282,Washington DC., 27 June-2 July 2004. G. Chen and W. Xie. Pattern recognition using dual-tree complex wavelet features and svm. In Proc. Canadian Conference on Electrical and Computer Engineering, pages 2053 2056, 2008. 30

[4]

International Journal of Computer Science & Information Technology (IJCSIT) Vol 5, No 2, April 2013 [5] [6] [7] [8] [9] [10] [11] A. Elayan, H. Ozkaramanli, and H. Demirel. Complex wavelet transform-based face recognition. EURASIP Journal on Advances in Signal Processing, 10(1):113, Jan 2008. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar. Fast and robust multiframe super resolution. IEEE Transactions on Image Processing, 13(10):13271344, 2004. W. T. Freeman, T. R. Jones, and E. C. Pasztor. Learning low-level vision. IEEE International Journal of Computer Vision, 40(1):2547, 2000. W. T. Freeman and C. Liu. Markov random fields for super-resolution and texture synthesis, chapter 10, pages 130. MIT Press, 2011. M. Grgic, K. Delac, and S. Grgic. SCface - surveillance cameras face database. Multimedia Tools and Applications Journal, 51(3):863879, 2011. P. H. Hennings-Yeomans. Simultaneous super-resolution and recognition. PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA, 2008. J. T. Hsu, C. C. Yen, C. C. Li, M. Sun, B. Tian, and M. Kaygusuz. Application of wavelet-based POCS superresolution for cardiovascular MRI image enhancement. In Proc. International Conference on Image and Graphics, pages 572 575, Hong Kong, China, Dec. 18-20, 2004. K. I. Kim and Y. Kwon. Single-image super-resolution using sparse regression and natural image prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(6):1127 1133, 2010. C. Liu, H. Y. Shum, and C. S. Zhang. A two-step approach to hallucinating faces: global parametric model and local nonparametric model. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 192198, San Diego, CA, USA, Jun. 20-26, 2005. J. Mairal, M. Elad, and G. Sapiro. Sparse representation for color image restoration. IEEE Transactions on Image Processing, 17(1):5369, Jan. 2008. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th International Conference on Computer Vision, volume 2, pages 416 423, July 2001. G. Pajares and J. M. Cruz. A wavelet based image fusion tutorial. Pattern Recognition, 37(9):18551872, Sep. 2004. J. Sun, N. N. Zheng, H. Tao, and H. Shum. Image hallucination with primal sketch priors. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages II 72936 vol.2, Madison, WI, Jun. 18-20, 2003. J. Wang, S. Zhu, and Y. Gong. Resolution enhancement based on learning the sparse association of image patches. Pattern Recognition Letters, 31(1):110, Jan. 2010. Q. Wang, X. Tang, and H. Shum. Patch based blind image super-resolution. In Proc. IEEE International Conference on Computer Vision, Beijing, China, Oct. 17-20, 2005. R. Willet, I. Jermyn, R. Nowak, and J. Zerubia. Wavelet based super resolution in astronomy. In Proc. Astronomical Data Analysis Software and Systems, volume 314, pages 107116, Strasbourg, France, 2003. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2):210227, February 2009. J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution as sparse representation of raw image patches. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, AK, Jun. 23-28, 2008.

[12] [13]

[14] [15]

[16] [17]

[18] [19] [20]

[21]

[22]

31