IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO.

3, MARCH 2010

561

A Fast Nonparametric Noncausal MRF-Based Texture Synthesis Scheme Using a Novel FKDE Algorithm
Arnab Sinha and Sumana Gupta
AbstractIn this paper, a new algorithm is proposed for fast kernel density estimation (FKDE), based on principal direction divisive partitioning (PDDP) of the data space. A new framework is also developed to apply FKDE algorithms (both proposed and existing), within nonparametric noncausal Markov random eld (NNMRF) based texture synthesis algorithm. The goal of the proposed FKDE algorithm is to use the nite support property of kernels for fast estimation of density. It has been shown that hyperplane boundaries for partitioning the data space and principal component vectors of the data space are two requirements for efcient FKDE. The proposed algorithm is compared with the earlier algorithms, with a number of high-dimensional data sets. The error and time complexity analysis, proves the efciency of the proposed FKDE algorithm compared to the earlier algorithms. Due to the local simulated annealing, direct incorporation of the FKDE algorithms within the NNMRF-based texture synthesis algorithm, is not possible. This work proposes a new methodology to incorporate the effect of local simulated annealing within the FKDE framework. Afterward, the developed texture synthesis algorithms have been tested with a number of different natural textures, taken from a standard database. The comparison in terms of visual similarity and time complexity, between the proposed FKDE based texture synthesis algorithm with the earlier algorithms, show the efciency. Index TermsKernel density estimation (KDE), Markov random eld (MRF), nonparametric density estimation, principal component analysis, texture synthesis, vector quantization.

I. INTRODUCTION

ATURAL texture synthesis is an important problem in image processing and computer vision. Texture synthesis models can be broadly classied within two domains, spatial and transformed domains, respectively. Within spatial-domain models, there are two basic categories; Linear, [1][3] and nonlinear models [4], [5]. All the models within image-domain, share a common property, i.e., the pixel random variable is modeled as a function of neighborhood pixel random variables. In the transformed-domain, the synthesis algorithms can be classied into two categories, the rst one [6] is based upon the

Manuscript received May 01, 2009; revised September 01, 2009. First published November 20, 2009; current version published February 18, 2010. This work was supported by the BSNL Telecom Centre of Excellence (Project Number EE/BSNL/20080033). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Arun Abraham Ross. The authors are with the Department of Electrical Engineering, Indian Institute of Technology, Kanpur, UP 208016, India (e-mail: arnabiitk@gmail.com; sumana@iitk.ac.in). Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TIP.2009.2036685

modeling of the transform domain with respect to a nonlinear model, and the second one [7][9] is based upon modeling the transformed domain with respect to some nonlinear constraints. There are also other models as described in [10][12], which use both spatial and transformed domain informations. There is also another sub-class, named as, intuitive models. Within this sub-class one can nd spatial-domain [13][17], mixed-domain [18] and transformed-domain [19] algorithms. All the spatial-domain intuitive models have been derived from the earlier work of [5]. We have considered the texture synthesis algorithm of [5] as our basis. The advantage of these intuitive algorithms is faster speed, but at the cost of probable convergence to a local optimum. Nonparametric, noncausal, Markov random eld (NNMRF) model [5] provides a theoretical background for synthesizing textures from a broad range of natural textures. The sampling process for synthesizing texture depends upon repetitive estimation of Markovian conditional density. This process has a large computational complexity [20], which does not favor real-time application. Recently, a number of algorithms have been proposed for fast kernel density estimation (FKDE) in multivariate data analysis eld, [21][23]. Direct application of FKDE algorithms within NNMRF framework for texture synthesis is not possible, due to the evolving nature of conditional density. This evolving nature is required for modeling local simulated annealing, which in turn provides faithful texture synthesis result [5]. In this paper, we have proposed a new FKDE algorithm and compared its performance with existing algorithms, in terms of the computational complexity, time complexity, and error precision. It is concluded from the discussion and analysis that, the proposed algorithm is more efcient than the earlier algorithms for the same error bound. Furthermore, we have developed a new texture synthesis algorithm to apply the FKDE algorithms within the NNMRF framework for fast texture synthesis. Results are shown for a broad range of textures from stochastic to near-regular, and are compared in terms of visual similarity and time complexity. The results provide the efciency of the proposed FKDE algorithm and its application to model evolving nature of conditional density for fast texture synthesis. An abridged version of this work has been published in [24]. In this paper, a detailed discussion of the proposed algorithms and a comparative analysis of the proposed algorithms with the earlier works is provided. In Section II, we give the basic description of NNMRF model for texture synthesis. The computational complexity analysis of texture synthesis through NNMRF model is provided in Section II-B. Section III introduces the problem denition of FKDE and describes the existing approaches along with their

1057-7149/$26.00 2010 IEEE

Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

562

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

limitations. In Section IV, we describe PDDP tree structure. The proposed FKDE algorithm is described in Section V. Section VI provides the results of comparison of different FKDE algorithms with respect to each other and with KDE. Finally, in Section VII, the texture synthesis algorithms that are developed with the FKDE algorithms, are discussed. Here, we have considered two earlier FKDE algorithms and the proposed FKDE algorithm for texture synthesis. Section VIII discusses the results of the proposed algorithm for natural texture synthesis. The paper is concluded in Section IX. II. NONPARAMETRIC MRF AND TEXTURE SYNTHESIS The description of MRF is taken from the view point of lattice models. The lattice, , is a collection of random variables , where, (r.v. henceforth) at the sites, . The random variables are described as , i.e., they belong to the same state space. The MRF assumption implies (1) which describes the fact that given a neighbor set, , the r.v. at and this condis is independent of all other sites, tional probability is termed as local conditional pdf (LCPDF). The neighborhood system is dened with the help of following two axioms: , ; , . Let us now consider the nonparametric MRF model as described in [5] in the context of texture synthesis. Assume, and signify the input and output texture lattices, respectively, and for simplicity, we also assume that denote the neighborhood set of the pixel r.v. , where represents the number of neighbors. For each pixel in the output lattice, , we estimate the LCPDF , for (grey values) from the data where, . This is generated from the input texture through Parzen window estimator using the product kernels as

reect the condence of synthesis of the random variables at site . The range corresponding random variable of the condence random variables varies from 0 to 1, where 1 represents complete condence, and 0 none at all. The rule for is dened as updation of the condence at site (3) where is a random variable generated from uniform distribution. Let us dene a neighborhood vector of condence , . The esrandom variables as timation of LCPDF according to the condence eld is dened as

(4) , and correspond to the neighborhood vectors of where output and input textures, respectively. The kernel for neighborhood vector can be written as

(5) where , is the cardinality, represents trans, and are the random variable of pose, , , and , respectively. the random vectors The initial values of the condence r.vs are zero. As time progresses, these values approach 1. Therefore, from (3) and (5), it is clear that as we progress in time the LCPDF (4) will approach . The the actual form starting from the marginal density updation of the condence eld is done as a new pixel is synthesized. Since we are approaching the true LCPDF through local simulated annealing, we can sample LCPDF according to independent conditional mode (ICM) algorithm as described in [25]. In , , which has the maximum probICM, we look for ability according to the given neighborhood , i.e., maximum LCPDF. The advantage of ICM is its speed, whereas other sampling algorithms (such as, Gibbs sampler, Metropolis sampler) has a very slow convergence rate with almost sure convergence to the global optimum [5]. In this paper, we have worked with ICM algorithm. B. Computational Complexity of Texture Synthesis Algorithm Based on NNMRF The computational complexity for the calculation of LCPDF is of the order of (4), where is the is the number of pixels within input texture sample and dimensionality of the set . In [20], we have tried to reduce this computational complexity by reducing the dimension of the neighborhood set . In this work, the computational complexity is reduced by reducing , which is equal to 128 128 in the current work.

(2) is the Gaussian kernel function, where is the bandHere, width parameter and can be estimated as , where is the average marginal variance [5]. In the following, a brief description of the sampling process from the nonparametric conditional density, as described in [5], is given. A. Local Simulated Annealing: Texture Synthesis Algorithm For synthesizing textures from a random initialization, we need local simulated annealing [5]. In this method the LCPDF (2) evolves with time. For local simulated annealing we require another random eld dened on the output lattice structure. We dene this random eld as . In this paper, we use the term condence eld, to describe the behavior and the requirement of the eld in the process of texture synthesis. The
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

SINHA AND GUPTA: FAST NONPARAMETRIC NONCAUSAL MRF-BASED TEXTURE SYNTHESIS SCHEME

563

III. INTRODUCTION TO KDE PROBLEM In this section, we formally state the problem of kernel density estimation (KDE), and dene two error criteria to analyse be a the effect of fast algorithms upon the KDE. Let be a target data point. In set of source data points, and nonparametric kernel density estimation, the probability of the target data point is calculated as follows: (6) where, in the case of Gaussian kernel, . Here, is , the number of source data points, , is the Gaussian kernel, and , [26]. The computational complexity of for each target data point. In case of parametric (6) is density estimation, after the parameters are learnt or estimated, the computational complexity of density estimation becomes , i.e., proportional to the dimension. Obviously the is very large computational complexity of KDE, i.e., compared to any parametric density estimator. One has to (number of source data points) within reduce the effect of the KDE with a balanced error threshold. In general, kernels (multivariate or univariate) have numerically nite support, e.g., in the case of Gaussian kernel, if the , in case absolute value of the kernel argument ( of Gaussian kernel) is 100, the value of the kernel term becomes ; when the kernel argument is quite negligible, i.e., 700, it becomes impossible to calculate the exponential term according to the IEEE standard. One can use this knowledge in kernel density estimation to minimize the required number of source data points. This is done by considering the source data points within a radius from the target data point, as shown in (7) (7) is a subspace corresponding to a constant radius Here, (say ) around the data point , Fig. 1. Therefore, the precision error is

Fig. 1. Explaining the nite support of kernel in KDE.

tational complexity and unsupervisedness for attaining a given and . A. Existing FKDE Algorithms and Their Limitations In the following Sections III-A1, III-A2, and III-A3, we briey describe three earlier FKDE algorithms along with their limitations. 1) Reconstructionhistogram: In [21], an algorithm named as reconstructionhistogram, was proposed for reducing the computational complexity of the KDE. The source data set is rst clustered within a number of clusters (user dened), . Here describes the mean vector cluster. Dene as the number of source data of the cluster. Thereafter the probability of target vectors within data vector is calculated as follows: (10) Equation (10) can be thought of as KDE of given the source . It data points at cluster centroids with a weight factor can be also thought of as a modied Gaussian Mixture Model [27]. Therefore, it does not have the exibility of original KDE. Moreover, the error bound is not specied in [21]. We have not considered this algorithm as FKDE. 2) Improved Fast Gaussian Transform: In [22], the source and data points are rst clustered. If the distance, between any cluster centroid, is more than a user-dened threshold (say ), the source data points within the corresponding cluster are not considered for the calculation of KDE. Let us dene the clusters as described in Section III-A1. The following equation describes the FKDE according to IFGT:

(8) In (8), we have described a least upper error bound (LUEB) for the FKDE algorithms. This error may not provide a good estimate of the underlying density, because if the actual density is lower than this precision error it will give a zero value. Therefore, we dene another measure along with this absolute precision measure. We term this error as relative error and it is dened as (9) In the rest of this paper, we will analyze the proposed FKDE algorithm with respect to the earlier algorithms in terms of compuAuthorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

(11) is the approximation that arises from In (11), the truncated multivariate Taylor expansion (only possible for Gaussian kernel [22]). To implement the original KDE (for any kernel function) the modied equation will be (12)

564

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

TABLE I WHY DO WE NEED ANOTHER ALGORITHM FOR FKDE?

Fig. 3. FKDE based upon kd-tree. Fig. 2. R varies with target data vector t and cluster shape and Computational complexity may increase in IFGT.

We term this FKDE as cluster based FKDE or C-FKDE. In is different from [required for error above equation, bound described in (8)]. The variation of is dependent under consideration. on the target data and the cluster This can be best understood from Fig. 2. One can x the to a maximum value as (13) where is the maximum distance between its centroid and its source data points, within cluster. In this case, the computational complexity of the algorithm may increase, as explained . in Fig. 2, due to the increase in 3) Kd Tree Based FKDE (KD-FKDE): In [23], another algorithm have been proposed for FKDE using kd-tree data structure. In kd-tree the partition takes place with respect to mean (or median) along the dimension which has maximum variation. Therefore, it denes a partition boundary perpendicular to the maximum variance dimension, Fig. 3. One can build a tree structure by recursively partitioning each sub-partition. For FKDE, one particular node (or sub-space) can be excluded (or included) from (within) the nal KDE if the and the minimum distance between the target data vector bounded (either hyper-rectangle or hyper-sphere) sub-space, Fig. 3, is more (less) than . We term this FKDE as KD-FKDE, since it is based upon the KD-tree structure. In Table I, we have summerized the advantages and disadvantages of C-FKDE and KD-FKDE. From this table, it is clear that in KD-FKDE one can optimally choose the , which is not possible in case of C-FKDE. On the other hand, KD-FKDE does not have a compact representation of the clustered space as
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

compared to C-FKDE. From this discussion, it follows that, one needs a partition with explicit boundary denition and compact representation. IV. PRINCIPAL DIRECTIVE DIVISIVE PARTITIONING (PDDP) In this section, the basic clustering algorithm PDDP [28], is described. In this algorithm, the information extracted from the Principal Component Analysis (PCA) of a set of source data points, is used to dene the cluster boundary. The partitioning involves two basic steps as described below: project each source data point within the present space onto the rst principal direction (eigen vector corresponding to the largest eigen value); partition the present space into two sub-spaces with respect to the mean (or median) of the projected values. One can build up a tree structure by recursively partitioning each sub-space. From this algorithm we construct source data index , where vectors for each leaf node, and is the depth of the tree structure. In the above mentioned clustering algorithm, all the principal vectors (eigen vectors) excluding the rst one, dene the boundary for the partition. Therefore, at each node (except the leaf nodes) of the tree, one such boundary is created for partitioning the space constructed by the source data points of the present node. An arbitrary example of PDDP clustering and tree structured data space is shown in Fig. 4. V. PROPOSED FKDE ALGORITHM In the previous Section IV, the tree structured data space is described. In this section, we use this tree structure for fast kernel density estimation. The fundamental difference between the proposed algorithm and KD-tree based algorithm [23] is

SINHA AND GUPTA: FAST NONPARAMETRIC NONCAUSAL MRF-BASED TEXTURE SYNTHESIS SCHEME

565

B. Need to Proceed Further? Upon selecting a node for processing, we must know whether its children are to be included, or we exclude the present node and all its children from further processing. This sub-module answers this question and the pseudo-code is given in algorithm 2. Project the target data vector upon the rst principal eigen , where vector of the data space of the present node, be the rst eigen vector, and be the projected target data point. Here we dene the concept of outside/inside for the target data point . If is within the boundaries of the proand , we say that is within jected source data vectors, the projected space [i.e., inside, Fig. 5(b)], otherwise it is outside [Fig. 5(a)]. If is outside, it is of interest to know how far is it , from the boundary, dened as , or its minimum distance from the projected space. If we do not go further and return a value of zero to its parent node. This is shown in Fig. 5(a) and (b). Algorithm 2 Process a node: Part-2-> Whether to process the present node 1: Input: 2: Input: 3: Input: vectors 4: Input: 5: Input: 6: if 7: 8: 9: 10: 11: A. Leaf Node? If the present node is a leaf node, then we calculate the kernel with each source data point within the present terms for node. The sum of these kernels is sent to its parent node. Thus, at the root node, one can accumulate all the required kernel eval, and later with a proper scaling function uations one can calculate the KDE of . The pseudo-code is described in algorithm 1. If the present node is not a leaf node, proceed to the second module. Algorithm 1 Process a leaf node 1: Input: 2: Input: 3: Return
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

Fig. 4. Arbitrary example of PDDP partitioning; Latin numerics dene the nodes (not leaf nodes).

that, we will use the maximum variance principal dimension instead of maximum variance dimension [23]

(14) To begin with, one nds the neighborhood leaf nodes corresponding to a target data point for a xed distance threshold . The source data points within these leaf nodes are then used for nal calculation of the kernel density of the target data point, as shown in (14). We have sub-divided the proposed FKDE algorithm into three parts, to answer the following three questions: 1) Is the present node a leaf node? If yes, then evaluate KDE and return the value to its parent node, otherwise go to step 2. 2) Is there any need to proceed further for the children of the present node? If yes, then go to step 3, otherwise return a zero value to its parent node. 3) Which child node (left or right or both) of the present node needs to be processed further? We term this proposed FKDE algorithm as PDDP-FKDE algorithm. The discussions related to the above three questions or sub-modules of the proposed algorithm, are described in the following sub-sections.

: target data vector : projected target data vector : boundaries of projected source data : the reconstruction error matrix : The radius threshold

is outside then Calculate if then return zero (0) end if

12: 13: 14: 15: 16:

if return zero (0) else

then

Which child to process further; algorithm 3 end if

17: end if On the other hand, if or is inside, evaluate the . Let the minimum reconstruction reconstruction error error for the source data points that has been calculated a priori, be . The reconstruction error is formally dened as , where is the reconstructed from the , we projected vector . Therefore, if do not proceed further, i.e., return zero value to its parent node; otherwise if , go for the third module.

: data index vector : target data vector

566

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

Fig. 5. FKDE algorithm: (a) Target point is outside; (b) target point is within one partition.

D. Computational Complexity Algorithm 3 Process a node: Part-1->Which child to process 1: Input: 2: Input: 3: Input: 4: Input: 5: 6: 7: if 8: then consider both the children : The rst principal eigen vector of the current node : Target data vector : The projected mean vector of node : The radius threshold , (6). The The computational complexity for KDE is , ideal computational complexity for FKDE is (7), where denotes cardinality of the set. The problem is to locate the source data points such that . As described in involve a path through Section V, the search for leaf nodes the tree structure. At each node (steps described in Section V), one has to decide whether to search in the left child or in the right child or within the children. This decision making process has a computational complexity of the order of , due to the projection of to . If we look at the algorithm 2, the computational complexity of the reconstruction error calculation, seems to be of order due to the operation , where is a full matrix. But, the reconstruction error can be calculated as shown in the following:

9: else 10: 11: 12: 13: 14: if then consider the left child else consider the right child. end if (15) . Equation (15) shows where, is a scalar and that the computational complexity of the reconstruction error evaluation in this particular case, is of the order of , instead of . We now consider the worst, best and average computational complexities of the FKDE algorithms as discussed below. In the worst case, one has to consider all the data points, i.e., all the leaf nodes. In this case, the computational complexity be, since . This is comes very unlikely in practice. In the best case, only one leaf node has to be considered, and, therefore, the minimum computational , complexity becomes since . In most cases, will lie close to one or more boundaries, and in those cases, one has to consider more than one leaf nodes, say this number is . Since, the complexity for searching is additive and number of nodes will have approximately number of source data points, on an average the computational complexity would be approximately .

15: end if C. Which Child Node to Be Included for Processing? Let the mean vector of the present node be and the projected target data vector be . Therefore, the projected mean vector is . Dene the difference between and , as , Fig. 5(b). Now, if , it implies that according to the maximum principal variance direction, can reside within either child of the present node. Therefore, we have to further process both children. Otherwise, if and is negative, it implies , i.e., resides within left child of the present node. If none of the above conditions are true, then we have to process the right child. The pseudo-code is described in algorithm 3.
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

SINHA AND GUPTA: FAST NONPARAMETRIC NONCAUSAL MRF-BASED TEXTURE SYNTHESIS SCHEME

567

TABLE II COMPARATIVE ANALYSIS OF FKDE ALGORITHMS

TABLE III TIME COMPLEXITY COMPARISON BETWEEN FKDE ALGORITHMS

TABLE IV TIME TAKEN IN TEXTURE SYNTHESIS

Fig. 6. Data set creation for FKDE analysis. (a) Image considered for creating the data set; (b) creation of the data space.

Fig. 7. Comparison of FKDE algorithms. (a) Time complexity analysis of KDE, C-FKDE, KD-FKDE, and PDDP-FKDE; (b) relative error analysis for KD-FKDE and PDDP-FKDE.

VI. RESULTS FOR FKDE In this section, we compare the FKDE algorithms with respect to KDE, and with respect to each other. We have considered a color image shown in Fig. 6(a) for the construction of the data space. After extracting the source data points from the image, we have estimated KDE for 1,00,000 uniformly distributed target locations. We have considered {3,6,9,12} dimensions for the dataspace, and the creation of these different dimensional dataspaces from the image is shown in Fig. 6(b). be the pixel locations for and Let, , where the image size is . Now, if locations, then we have number we consider, only of source data points with dimension 3, due to the color values R, G, and B. If we consider another pixel in the neighborhood with the , i.e., {1,2} as shown in Fig. 6(b), we have 6-D data space, and so on and so forth. Here, the depth of the tree structures (for both KD-FKDE and PDDP-FKDE) is 7 and the . upper bound of relative error precision is kept at
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

In Table III, the time needed by different algorithms are shown. From this table it becomes clear that PDDP-FKDE is best compared to all other algorithms in terms of time requirement. In Table II, the error analysis with respect to KDE algorithm is shown. From Table II, the following points become clear. In terms of the relative error analysis and radius required (to get such error bound), C-FKDE algorithm is worst. Moreover, from Table III, it can be seen that C-FKDE takes more time than other FKDE algorithms. The maximum error of proposed PDDP-FKDE is less than or equal to other earlier algorithms, i.e., KD-FKDE and C-FKDE. VII. COMPUTATIONALLY EFFICIENT TEXTURE SYNTHESIS ALGORITHM WITH FKDE ALGORITHMS The computational complexity of earlier texture synthesis algorithm [5], is mainly due to the evaluation of (4) at each output pixel . Incorporation of proposed FKDE algorithm within the LCPDF estimation (4), can reduce this high com. This incorporation reputational complexity quires an estimate of the radius threshold, i.e., in case of C-FKDE, and in cases of KD-FKDE and PDDP-FKDE. Moreover, is dependent upon , (13). Therefore, we will describe how to estimate this radius threshold , in case of LCPDF estimation through local simulated annealing. A. Effect of Local Simulated Annealing be the required error precision, be the number Let of source data vectors, be the target vector (where ),

568

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

be one of the source vector (where ). One can and from the KDE of , if (16) neglect the source data vector is satised

(16) From this equation, one notes that the decision of excluding a source data vector from the calculation of KDE can be taken by and with . Therefore, comparing the distance between one can use the distance between the and the cluster centroid (in case of C-FKDE) or use partitions (in cases of KD and PDDP based tree structure) for the FKDE. In case of local simulated annealing, (16) takes the form of (17)

node (in where is the maximum variance dimension of the case of KD-FKDE). For PDDP-FKDE, these two scaler values and , where and designates the node are number within the tree structure. In order to incorporate the effect of local simulated annealing within this decision making process, one has to consider the effect of annealing upon the radius threshold . Exact mathematical understanding is impossible; therefore, we have tried to build up an approximate analysis as follows: (19) Let us consider, for all , i.e., the condence values of neighborhood pixels are all same and equal to . Now, (16) can be expressed in an equivalent form of the . One can (19). This equation describes a new threshold easily observe that when all the condence values within the neighborhood are approximately same, the value of the radius decreases as increases (i.e., annealing prothreshold gresses). Therefore, the approximately starts from a very large value and approaches its true value as annealing process progresses. , for It is not possible to have equal values in all . Therefore, exact incorporation is not possible. However, from the above discussion, we can make a crude approximation that, as annealing process progresses the value of approaches starting from an initial large value. This approximation can be made by considering (the condence (which is difcult to x given value at pixel site ) rather than ) within the estimation of new radius threshold to incorporate the effect of local simulated annealing in the tree structures. This new rule is dened in (20) (20) In the next section, we will discuss and analyse the results of each texture synthesis algorithm along with their comparisons to the original one [5]. VIII. RESULTS AND DISCUSSIONS: TEXTURE SYNTHESIS ALGORITHMS Natural textures range from near-regular to stochastic. We have tested the computationally efcient texture synthesis algorithms (Section VII), for texture samples that are taken from standard database [29]. The samples considered can be broadly categorizeded into three groups, 1) near-regular (NR), 2) stochastic (ST), and 3) near-regular with stochasticity (NR+ST). The order (required to dene the neighborhood system of NNMRF) is kept xed at 20 for all cases. The texture synthesis is carried out at single resolution. Here we have considered and depth of the tree structures as equal to 8. The initialization of the output texture is a uniformly distributed random noise and is same for all texture synthesis algorithms; the updation of the condence eld is also same for all algorithms. The number of iterations for texture synthesis has been xed at 50 for all algorithms. Consequently the results obtained for time complexity and visual similarity depend solely upon the algorithms used and do not depend on the type of texture used.

(17) From (16), we get the corresponding equivalence form in terms , of radius threshold . In (17) we introduce a new term for whose off-diagonal values are zero, and . In FKDE algorithms, we do not have direct access to the source data vectors. At rst we need to nd source clusters . The source vectors residing within the neighborhood of within these clusters are considered in the KDE of . The searching algorithms for the clusters, are different in C-FKDE, KD-FKDE and PDDP-FKDE. B. Incorporation of R Within C-FKDE Based on Local Simulated Annealing and cluster In C-FKDE, one use the distance between , to decide whether to include centroids or exlude the corresponding cluster within the nal KDE of , as described in (12). One can replace with in (16), to decide the inclusion or exclusion of the corresponding cluster within the local simulated annealing environment, as shown in (18). Therefore, C-FKDE can be incorporated within the texture synthesis algorithm directly through (18) (18) C. Incorporation of R Within KD-FKDE and PDDP-FKDE Based on Local Simulated Annealing The major requirement of within the tree based FKDE algorithms (KD-FKDE and PDDP-FKDE), can be found in step number 7 of the algorithm 3, for deciding which child to process. with . This distance Here one has to compare a distance value is calculated from two scaler values, and ,
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

SINHA AND GUPTA: FAST NONPARAMETRIC NONCAUSAL MRF-BASED TEXTURE SYNTHESIS SCHEME

569

Fig. 8. Near-regular texture synthesis result. (a) 102; (b) NNMRF; (c) C-FKDE; (d) KD-FKDE; (e) PDDP-FKDE (proposed); (f) 49; (g) NNMRF; (h) C-FKDE; (i) KD-FKDE; (j) PDDP-FKDE (proposed); (k) 20; (l) NNMRF; (m) C-FKDE; (n) KD-FKDE; (o) PDDP-FKDE (proposed); (p) 22; (q) NNMRF; (r) C-FKDE; (s) KD-FKDE; (t) PDDP-FKDE (proposed); (u) 53; (v) NNMRF; (w) C-FKDE; (x) KD-FKDE; (y) PDDP-FKDE (proposed).

The results for near-regular and stochastic texture classes are shown in Figs. 8 and 9, respectively. From Fig. 8, it can be observed that the computationally efcient texture synthesis algorithm (with FKDE) produces better result than the earlier algorithm in cases of near-regular textures. Whereas, in case of stochastic texture class (Fig. 9) the results match closely. Fig. 10 shows the results for the third class of natural texture is shown. The results establish the effectiveness of incorporation of the
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

proposed FKDE (PDDP-FKDE) within texture synthesis algorithm. In Table IV, the time required by each algorithm is given. It can be easily observed that, KD-FKDE based texture synthesis algorithm has poor performance compared to all the other algorithms, in terms of time requirement. This is expected, since a high-dimensional KD-tree can not perform better. From Table IV, it can be seen that, although C-FKDE based algorithm takes the minimum time, from the view-point of visual similarity of

570

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

Fig. 9. Stochastic texture synthesis result. (a) 110; (b) NNMRF; (c) C-FKDE; (d) KD-FKDE; (e) PDDP-FKDE (proposed); (f) 60; (g) NNMRF; (h) C-FKDE; (i) KD-FKDE; (j) PDDP-FKDE (proposed); (k) 9; (l) NNMRF; (m) C-FKDE; (n) KD-FKDE; (o) PDDP-FKDE (proposed); (p) 93; (q) NNMRF; (r) C-FKDE; (s) KD-FKDE; (t) PDDP-FKDE (proposed); (u) 97; (v) NNMRF; (w) C-FKDE; (x) KD-FKDE; (y) PDDP-FKDE (proposed).

D D D

D D

the synthesized textures, it is clear that, C-FKDE does not perform as good. We conclude that only PDDP-FKDE based algorithm is an optimum choice for fast texture synthesis from both perspectives of visual similarity and time complexity. IX. CONCLUSION The main contribution of this work is to analyse the nonparametric FKDE from the viewpoint of a strict upper error bound
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

(8) and to develop a new algorithm based upon this error bound. It has been shown that in FKDE, the boundaries (hyperplanes) instead of the cluster centroids for partitioning the space produce lesser computational complexity (Section III-A). The clustering algorithm PDDP (introduced in [28]) has been shown to have the desired boundary-based partitioning property (Section IV). The computational complexity of PCA can be reduced further by using smart algorithms like [30], [31].

SINHA AND GUPTA: FAST NONPARAMETRIC NONCAUSAL MRF-BASED TEXTURE SYNTHESIS SCHEME

571

Fig. 10. Near-regular + stochastic texture synthesis result. (a) 104; (b) NNMRF; (c) C-FKDE; (d) KD-FKDE; (e) PDDP-FKDE (proposed); (f) 4; (g) NNMRF; (h) C-FKDE; (i) KD-FKDE; (j) PDDP-FKDE (proposed); (k) 55; (l) NNMRF; (m) C-FKDE; (n) KD-FKDE; (o) PDDP-FKDE (proposed); (p) 82; (q) NNMRF; (r) C-FKDE; (s) KD-FKDE; (t) PDDP-FKDE (proposed).

D D

D D

In this paper, the computational complexity of the earlier texture synthesis algorithm [5] has been reduced through the incorporation of proposed FKDE. This improves the practical utility of the synthesis algorithm. Direct incorporation of the tree structure is not possible. Therefore, a new scheme is proposed for the incorporation of FKDE algorithms within local simulated annealing based LCPDF estimation. The proposed algorithm in texture synthesis has been tested for a number of textures taken from standard database [29], and compared with the original algorithm [5] as well as the existing FKDE techniques. The results, shown in Figs. 8, 10, and 9, establish the efciency of the proposed FKDE method for synthesis of natural textures.
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.

ACKNOWLEDGMENT The authors would like to thank the Indian Institute of Technology for the multiple computer facilities in the computer center. REFERENCES [1] R. Chellappa and R. L. Kashyap, Texture synthesis using 2-d noncausal autoregressive models, IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 1, pp. 194203, Jan. 1985. [2] J. M. Francos, A. Z. Meiri, and B. Porat, A unied texture model based on a 2-d wold-like decomposition, IEEE Trans. Signal Process., vol. 41, no. 8, pp. 26652678, Aug. 1993. [3] K. B. Eom, 2-d moving average model for texture synthesis and analysis, IEEE Trans. Image Process., vol. 7, no. 12, pp. 17411746, Dec. 1998.

572

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010

[4] G. Jacovitti, A. Neri, and G. Scarano, Texture synthesis-by-analysis with hard-limited gaussian processes, IEEE Trans. Image Process., vol. 7, no. 11, pp. 16151621, Jul. 1998. [5] R. Paget and I. D. Longstaff, Texture synthesis via a noncausal nonparametric multiscale markov random eld, IEEE Trans. Image Process., vol. 7, no. 6, pp. 925931, Jun. 1998. [6] G. Fan and X.-G. Xia, Wavelet-based texture analysis and synthesis using hidden markov models, IEEE Trans. Circuits Syst.I: Fundam. Theory Appl., vol. 50, no. 1, pp. 106120, Jan. 2003. [7] S. C. Zhu, Y. N. Wu, and D. Mumford, Minimax entropy principle and its application to texture modeling, Neural Comput., vol. 9, pp. 16271660, 1997. [8] S. C. Zhu, X. W. Liu, and Y. N. Wu, Exploring texture ensembles by efcient markov chain monte carlo-toward a trichromacy theory of texture, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 6, pp. 554569, Jun. 2000. [9] J. Portilla and E. P. Simoncelli, A parametric texture model based on joint statistics of complex wavelet coefcients, Int. J. Comput. Vis., vol. 40, no. 1, pp. 4971, 2000. [10] P. Campisi and G. Scarano, A multiresolution approach for texture synthesis using the circular harmonic functions, IEEE Trans. Image Process., vol. 11, no. 1, pp. 3751, Jan. 2002. [11] D. Charalampidis, Texture synthesis: Textons revisited, IEEE Trans. Image Process., vol. 15, no. 3, pp. 777787, Mar. 2006. [12] J. Zhang, D. Wang, and Q. N. Tran, A wavelet-based multiresolution statistical model for texture, IEEE Trans. Image Process., vol. 7, no. 11, pp. 16211627, Nov. 1998. [13] A. A. Efros and T. K. Leung, Texture synthesis by non-parametric sampling, presented at the IEEE Int. Conf. Computer Vision, 1999. [14] L. Wei and M. Levoy, Fast texture synthesis using tree-structured vector quantization, presented at the SIGGRAPH, 2000. [15] M. Ashikhmin, Synthesizing natural textures, presented at the ACM Symp. Interactive 3D Graphics, 2001. [16] V. Kwatra, A. Schdl, I. Essa, G. Turk, and A. Bobick, Graphcut textures: Image and video synthesis using graph cuts, presented at the SIGGRAPH, 2003. [17] Q. Wu and Y. Yu, Feature matching and deformation for texture synthesis, presented at the SIGGRAPH, 2004. [18] L. Toniettot, M. Walter, and C. R. Jung, Patch-based texture synthesis using wavelets, presented at the SIGGRAPH, 2005. [19] D. J. Heeger and J. R. Bergen, Pyramid-based texture analysis/synthesis, in Proc. SIGGRAPH, 1995, pp. 229238. [20] A. Sinha and S. Gupta, Approximation of conditional density of markov random eld and its application to texture synthesis, presented at the IEEE Int. Conf. Image Processing, 2007. [21] K. Zhang, M. Tang, and J. T. Kwok, Applying neighborhood consistency for fast clustering and kernel density estimation, presented at the Computer Society Conference on Computer Vision and Pattern Recognition, 2005. [22] C. Yang, R. Duraiswami, N. A. Gumerov, and L. Davis, Improved fast gauss transform and efcient kernel density estimation, presented at the 9th IEEE Int. Conf. Computer Vision, 2003.

[23] A. G. Gray and A. W. Moore, Nonparametric density estimation: Toward computational tractability, IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 13441348, Aug. 2003. [24] A. Sinha and S. Gupta, Fast estimation of nonparametric kernel density through pddp, and its application in texture synthesis, presented at the 1st Int. Academic Conf. Visions of Computer Science, 2008. [25] J. E. Besag, On the statistical analysis of dirty pictures, J. Roy. Statist. Soc. B, vol. 48, pp. 259302, 1986. [26] D. W. Scott, Multivariate Density EstimationTheory, Practice and Visualization. New York: Wiley, 1992. [27] G. McLachlan and D. Peel, Finite Mixture Models. New York: Wiley, 2000. [28] D. L. Boley, Principal direction divisive partitioning, Data Mining Knowl. Disc., vol. 2, no. 4, pp. 325344, 1998. [29] P. Brodatz, Textures: A Photographic Album for Artists and Designers. Toronto, ON, Canada: Dover, 1966. [30] R. J. Schilling and S. L. Harris, Applied Numerical Methods for Engineers Using MATLAB and C. New York: Brooks/Cole, 2000. [31] A. Sharma and K. K. Paliwala, Fast principal component analysis using xed-point algorithm, Pattern Recognit. Lett., vol. 28, no. 10, pp. 11511155, 2007.

Arnab Sinha was born in Kolkata, West Bengal, India, on March 24, 1979. He received the B.Tech. degree in electronics and communication engineering from Kalyani Government Engineering College, Kalyani, West Bengal, in 2001, and the M.Tech. degree in radio physics and electronics from Science College, Kolkata, in 2003. He is currently pursuing doctoral studies at the Department of Electrical Engineering, Indian Institute of Technology (IIT), Kanpur, India. Since August 2009, he has been with the SAMSUNG R&D Lab, Noida, India.

Sumana Gupta (M90) received the B.E. and M.E. degrees in electrical communication engineering from Indian Institute of Science, Bangalore, India, and the Ph.D. degree in electrical engineering from the Imperial College of Science and Technology, London, U.K. She is currently a Professor in the Department of Electrical Engineering, Indian Institute of Technology, Kanpur, India. Her current research interests are in video compression, video restoration, and image analysis.

Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:475 UTC from IE Xplore. Restricon aply.