Computer Physics Communications 158 (2004) 106112 www.elsevier.

com/locate/cpc

A recursive algorithm for nding HDMR terms for sensitivity analysis

H. Kaya a, , M. Kaplan a , H. Saygn a,b
a Istanbul Technical University, Institute of Informatics, Computational Science and Engineering Program, 34469 Maslak, Istanbul, Turkey b Istanbul Technical University, Institute of Energy, 34469 Maslak, Istanbul, Turkey

Received 17 April 2003; accepted 8 December 2003

Abstract High Dimensional Model Representation (HDMR) is an efcient technique which decomposes a multivariate function into a constant, univariate, bivariate functions and so on. These functions are forced to be mutually orthogonal by means of an orthogonality condition. The technique which is generally used for high-dimensional input-output systems can be applied to various disciplines including sensitivity analysis, differential equations, inversion of data and so on. In this article we present a computer program that computes individual components of HDMR resolution of a given multivariate function. The program also calculates the global sensitivity indices. Lastly the results of the numerical experiments for different set of functions are introduced. 2004 Elsevier B.V. All rights reserved.
PACS: 02.50.Sk; 02.30.Mv; 02.70.-c Keywords: Multivariate analysis; High dimensional model representation; Sensitivity analysis

1. HDMR fundamentals There have been recent efforts to develop efcient methods for tackling multivariate functions in such a way that the components of the approximation formula to express a multivariate function are ordered starting from a constant and gradually approaching to multivariance as we proceed along the terms like univariate, bivariate and so on. This type of method was offered by Sobol [1] and its revised and generalized
* Corresponding author.

form was proposed and applied to various problems [25]. The method is called High Dimensional Model Representation (HDMR) and its basic philosophy can be given through the following general equation for a given multivariate function f (x1 , . . . , xn ) dened over an n-dimensional unit hypercube K n :
n n

f (x) = f0 +
i=1

fi (xi ) +
i,j =1 i<j

fij (xi , xj ) + (1)

+ f12...n (x1 , . . . , xn ),
E-mail addresses: hkaya@be.itu.edu.tr (H. Kaya), mkaplan@be.itu.edu.tr (M. Kaplan), sayginh@itu.edu.tr (H. Saygn). 0010-4655/$ see front matter 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.comphy.2003.12.005

where all variables x1 , . . . , xn are assumed to vary between 0 and 1 without loss of generality. There are totally 2n terms on the right-hand side of Eq. (1).

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

107

The right-hand side additive terms of this equality are orthogonal components of the original function. The orthogonality is achieved by imposing the following vanishing condition (2):
1

The basic underlying philosophy of HDMR is to generate an approximation scheme using the rst few terms of the expansion (1) usually up to the secondorder terms: f(x) = f0 +
n n

dxil (xil )fi1 ...ik (xi1 , . . . , xik ) = 0

0

fi (xi ) +
i=1 i,j =1 i<j

fij (xi , xj ).

(9)

if il {i1 , . . . , ik },

(2)

where (xil ) is a weight function. The condition is known as Sobols condition and it has a direct consequence formalized as in Eq. (3): dx (x)fi1 ...ik (xi1 , . . . , xik )fj1 ...jl (xj1 , . . . , xjl ) = 0
Kn

if {i1 , . . . , ik } {j1 , . . . , jl }. /

(3)

Having these constraints, it is shown that all HDMR terms can be determined step by step [1]. Zeroth order HDMR term f0 which is a constant can easily be found by taking the n-dimensional integral of (1) and applying the condition (2) for the unit weight function: f0 =
Kn

f (x) dx.

(4)

In a similar fashion, to nd the rst-order HDMR terms, fi (xi ), Eq. (1) is integrated n 1 times with respect to all variables except xi and the resulting equation is simplied by means of the vanishing condition (2): fi (xi ) =
K n1

f (x) dxi f0 ,

(5)

where
n

dx =
i k=1 k=i

dxk .

(6)

It is straightforward to obtain the required equations for higher-order HDMR functions: fij (xi , xj ) =
K n2

f (x) dxij fi (xi ) fj (xj ) f0 , (7)

where
n

dxij =
k=1 k=i,j

dxk .

(8)

The function f (x) is sometimes called as model output and the variables are called independent model inputs. HDMR expansion can be also regarded as hierarchically correlated function expansion in terms of the input variables. For example fi (xi ) represents the individual contribution of the variable xi whereas fij (xi , xj ) represents the cooperative effect of the inputs xi and xj together and so on. The constant term f0 can be thought as the mean response of the system. It is observed that component functions in Eq. (1) arising in most physical systems do not exhibit high-order cooperativity among the input variables. Therefore HDMR approximation has a clear physical meaning. Almost all numerical experiments show that the third and higher-order HDMR functions have very small impact on the output, so it sufces to cut the approximation at the second order although the expansion (1) is of nite order and is always an exact representation of the original problem. Choosing different weight functions yields different kinds of HDMRs. As we did in this article the case where the weight is taken unity is called the Sobols case. This type of presentation is also called ANOVA-HDMR since it is the same as the ANOVA (analysis of variance) decomposition used in statistics [6]. If the weight is chosen as the Dirac-delta function, we can get rid of integrals and obtain a different approach which is called Cut-HDMR [3]. For CutHDMR, a reference point is needed to determine the HDMR functions. Cut-HDMR expansion gives exact result along the lines, planes, volumes etc. through and around the reference point whereas Sobol-HDMR returns a good average value everywhere. A main disadvantage of Sobol-HDMR is high-dimensional integrals for which Monte Carlo based numerical integration is advised [1]. There are two main approaches where HDMR is valuable; (1) generating an approximation for a model by using the rst few HDMR terms, (2) identifying the key variables with the most inuence on the outputs.

108

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

In the former approach, the approximation is known as Fully Equivalent Operational Model (FEOM) [2,5]. It is used to create a mathematical model if the input/output relation is not known explicitly or the existing mathematical model is not cost-effective. There are various studies on HDMR based FEOM especially for atmospheric photochemistry models [2,4]. In the second approach, sensitivities of variables are calculated by using HDMR coefcients. Then the global variance-based sensitivity indices are determined by using the following formulas D=
Kn

Let
1

i =
0

dxi gi (xi ).

(16)

The zeroth and rst-order HDMR terms of f (x) can easily be found as
n

f0 =
i=1

i , 1 j n.

(17) (18)

fj (xj ) = gj (xj ) j ,

(x) dx f02 , fi2...ik (xi1 , . . . , xik ) dx. 1

(10)

Taking the square of Eq. (1) and integrating with respect to all variables we get
n

Di1 ...ik =
Kn

(11)
Kn

f 2 (x) = f02 +
n i=1K n n

fi2 fi2 ...ik . 1

Taking square of Eq. (1), integrating with respect to all variables and using the vanishing condition (2) one may get
n n

+
k=2 i1 ,...,ik =1 K n i1 <<ik

(19)

D=
k=1 i1 ,...,ik =1 i1 <<ik

Di1 ...ik .

(12)

Putting Eqs. (15), (17) and (18) into Eq. (19) and use Eq. (16) for simplications, we get
n n

If Eq. (12) is divided by D, the sensitivity estimates are obtained

n n

1=
k=1 i1 ,...,ik =1 i1 <<ik

Si1 ...ik ,

(13)

k=2 i1 ,...,ik =1 K n i1 <<ik

fi2...ik = 0. 1

(20)

where Di1 ...ik (14) . D Of course to nd the sensitivity indices, the squares of all functions considered so far are assumed to be integrable over K n . It can be easily seen that sensitivity indices must be nonnegative and less than 1. Sensitivity indices are used to capture the input/output relation of the model. For a certain class of functions the relation can be determined in advance. Si1 ...ik = Remark 1. If an n-dimensional function is dened as a sum of 1-dimensional functions then all the sensitivity indices Si1 ...ik for k 2 are zero. Proof. Consider the function f (x) = g1 (x1 ) + + gn (xn ). (15)

Eq. (20) is a sum of nonnegative terms whose sum is equal to zero therefore each individual term must be zero and it follows that fi1 ...ik = 0 for k 2 and Di1 ...ik = 0 for k 2 hence Si1 ...ik = 0 for k 2. HDMR representation may not always give proper results for some types of functions. Eq. (35) with p = 0 is an example for such cases. Remark 2. If an n-dimensional function is dened as a product of 1-dimensional functions whose integrals are zero then all HDMR terms except the last one are zero. Proof. It is evident from the denition of HDMR terms. We have mentioned that HDMR expansion can be used as a replacement for existing mathematical models. For this purpose, starting from the constant

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

109

f0 , the rst few terms of the representation (1) should be evaluated. In order to know the goodness of the approximation the following scalars can be used
n 2 1 = i=1 n 2 2 = i=1

Si ,
n

(21)

Si +
i,j =1 i<j

Sij ,

(22)

These scalars constitute an increasing sequence whose elements are nonnegative and the value of the 2 biggest one i.e., n is unity. It is easily seen that they are ordered according to Eq. (23). Generally the sequence is getting very close to unity at rst or second elements and hence provide a good approximation
2 1 2 2 2 3

The main problem is to nd out which integrals are needed to obtain HDMR functions. Then the required integrals are evaluated by means of symbolic integration. A distinctive feature of the algorithm is the ability to save the evaluated integrals for later calls. Whenever it needs an integral it rst tries to nd out whether it is evaluated before or not. For this reason, a primitive method must be used to transform the sets into numbers. The transformation must produce a unique number preferably an integer for a set. In this article we choose base arithmetic whose general formulation is as follows
k

{i1 , . . . , ik }
s=1

is (n + 1)ks ,

(25)

2 n = 1.

(23)

where n is the number of variables in the system. Similarly the HDMR functions are represented by sets. For example the term fij is shown as H{i,j } . General denition is as follow H{i1 ,...,ik } fi1 ...ik . (26)

Although we can manually determine HDMR resolution and the global sensitivity indices for some set of functions, a computer program is needed for general functions with large number of variables. For this purpose we developed a computer program, details of which is explained in the coming section.

The general formula that gives the required integrals for HDMR terms can be stated in Eq. (27)
m |M mi |

HM =
i=0

(1)

i k=1

IN\M mi (k) ,

(27)

2. The algorithm This section is devoted to the details of the program and the underlying algorithm. The program is written in a language capable of symbolic integration [7]. It simply takes the multivariate function and the maximum number of variables then returns the components of the HDMR resolution of that function. It can also keep the intermediate calculations for later calls. Underlying algorithm efciently uses sets and recursive functions. The sets play an important role in the developed algorithm. The integrals required to determine HDMR terms are represented by sets. For example double integration with respect to the variables xi and xj is represented by the set {i, j }. For the sake of simplicity, the integrals will be denoted by the symbol I
1 1

where M = {i1 , . . . , im } is an ordered subset of N = {1, . . . , n} and M l represents the ordered subsets of M whose number of elements are equal to l. Number of sets in M l is denoted by |M l |. For illustrational purposes let us determine f12 by using Eq. (27) for N = {1, 2, 3, 4}. If M = {1, 2} and m = 2, subsets of M are found as follows: M 2 = {1, 2}, M 1 = {1}, {2}, M = ,
0

M 2 = 1, M 1 = 2, M
0

(28) (29) (30)

= 1.

Putting the above equations into Eq. (27) we get

1 2 1

HM =
k=1

IN\M 2 (k)
k=1

IN\M 1 (k) +
k=1

IN\M 0 (k) (31) (32)

= I{3,4} I{2,3,4} I{1,3,4} + I{1,2,3,4} . f (x) dxik , (24)

I{i1 ,...,ik }
0

dxi1
0

The rest of the problem is to calculate those integrals listed in Eq. (32).

110

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

The main part of the algorithm which we call kernel is designed to evaluate those integrals. The kernel takes a set as input and returns the corresponding integral value as output. (See Fig. 1.) The kernel function is designed in such a way that only the required integrals are taken. The required integrals are determined by using the formula in Eq. (27). Moreover evaluated integrals are stored for later calls. The ow chart of the kernel function is displayed in Fig. 2. Recursive denition of the kernel function makes the algorithm compact and clear compared to the older algorithm [8]. Another advantage of the new algorithm is the ability of generating all HDMR functions from the zeroth order to the nth order no matter how big n is. However the program can only be used for models for which the explicit form is known and symbolic integration can be carried out without difculty. Moreover, the success of the algorithm depends on sym-

bolic integration capabilities of the software. Therefore, if the real model does not satisfy those requirements then one must deal with either Cut-HDMR or ANOVA-HDMR with numerical integration schemes. There are studies on the sampling design for numerical integration to decrease the sampling effort [9]. The authors are currently studying for this kind of problem and are using Monte Carlo based numerical integration scheme [10].

3. Numerical experiments In this section, HDMR functions and hence global sensitivity indices of some test functions are presented. 3.1. Experiment 1 In this experiment we choose 4 different functions with 4 variables and compute the sensitivity estimates

Fig. 1. The kernel function.

1: Input set M and n-dimensional function F 2: if M is empty then 3: Return F 4: else 5: if M = {i} then 6: Convert M into the number a 7: if hdmr a is dened then 8: Return hdmr a 9: else 10: Evaluate k = F dxi 11: Store hdmra = k 12: Return k 13: end if 14: else 15: M = {i1 , . . . , im } m > 1 16: Convert M into the number a 17: if hdmr a is dened then 18: Return hdmra 19: else 20: N = {i1 , . . . , im1 } 21: k = Kernel(N,F) dxm 22: Store hdmra = k 23: Return k 24: end if 25: end if 26: end if

F = x1 + x2 + x3 + x4 , G = x1 x2 + x3 + x4 , H = x1 x2 x3 + x4 , I = x1 x2 x3 x4 . (33)

We examine the impact of different sets of variables upon the output of the function. There are 24 1 = 15 possible variations: individual effects of variables, pairwise cooperative effects, and so on. These effects can be observed by using our algorithm based on HDMR method. Table 1 shows that for the function F , all variables have equal contributions. In the function G, x3 and x4 have larger contributions than x1 and x2 because the product x1 x2 produces small numbers compared to the sum x3 + x4 on the interval [0, 1]. The table also reects the cooperative actions of variables. For the function H , the column 14 is empty since there is no relation between x1 and x4 . 3.2. Experiment 2 In this experiment, we choose two different functions including a parameter p [11]. This time we try to measure how much the HDMR approximation is close to the real function. For this purpose, the terms

Fig. 2. Algorithm of the kernel function.

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

111

Table 1 Relative effects of individual and cooperative effects of variables for the functions F , G, H and I in (33) 1 F G H I 25.0 9.67 4.97 15.4 2 25.0 9.67 4.97 15.4 3 25.0 38.0 4.97 15.4 4 25.0 38.0 79.6 15.4 12 3.22 1.66 5.14 13 1.66 5.14 14 5.14 23 1.66 5.14 24 5.14 34 5.14 123 0.55 1.71 124 1.71 134 1.71 234 1.71 1234 0.57

Sum of columns for each row should sum up to 100.0%.

Table 2 Contributions of HDMR terms to the approximation for different values of p for the function (34) n=5 p=1 p=2 p=3 p=4 p=5 p = 10
2 1 2 2 2 3 2 4 2 5

for every kind of functions. Moreover one may need to evaluate higher-order HDMR terms to achieve better approximations.

0.672 0.497 0.416 0.371 0.343 0.284

0.941 0.850 0.790 0.751 0.723 0.659

0.994 0.976 0.959 0.945 0.935 0.907

0.999 0.998 0.996 0.995 0.993 0.989

1.0 1.0 1.0 1.0 1.0 1.0

4. Conclusion We introduced a computer program which is written in a symbolic algebra language capable of taking symbolic integration for calculating HDMR functions and global sensitivity indices of a given function by means of dening recursive procedures. The program stores the intermediate calculations for later calls to save computation time. Although some parts of the program need improvements, for instance, the transformation from sets to the integers, it can be used as an efcient tool for HDMR researches.

Table 3 Contributions of HDMR terms to the approximation for different values of p for the function (35) n=5 p=0 p=1 p=2 p=3 p=4 p=5 p = 10 p = 100
2 1 2 2 2 3 2 4 2 5

0.0 0.098 0.518 0.744 0.846 0.899 0.974 0.999

0.0 0.359 0.864 0.965 0.988 0.995 0.999 0.999

0.0 0.706 0.979 0.997 0.999 0.999 0.999 1.0

0.0 0.938 0.999 0.999 0.999 0.999 0.999 1.0

1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

References
[1] I.M. Sobol, Sensitivity estimates for nonlinear mathematical models, Math. Model. Comput. Exper. 1 (1993) 407414. [2] J.A. Shorter, P.C. Ip, H. Rabitz, An efcient chemical kinetics solver using high dimensional model representation, J. Phys. Chem. A 103 (1999) 71927198. [3] . Al, H. Rabitz, Efcient implementation of high dimens sional model representations, J. Math. Chem. 29 (2001) 127 142. [4] I. Banejee, M.G. Ierapetritou, Design optimization under parameter uncertainty for general black-box models, Ind. Eng. Chem. Res. 41 (2002) 66876697. [5] G. Li, S.W. Wang, H. Rabitz, S. Wang, P. Jaffe, Global uncertainty assessments by high dimensional model representation (HDMR), Chem. Eng. Sci. 57 (2002) 44454460. [6] H. Rabitz, .F. Al, J. Shorter, K. Shim, Efcient inputoutput s model representations, Comput. Phys. Commun. 117 (1999) 1120. [7] MuPAD, The open computer algebra system, http://www. mupad.de, Last visited June 03, 2003.

in Eq. (23) are used,

n

f (x) =
i=1 n

(1 + 3pxi2 ) , p+1 (4xi 2) + p . p+1

(34)

f (x) =
i=1

(35)

Results are shown in Tables 2 and 3. Especially Table 3 displays very interesting results. The rst row of the table claims that the HDMR approximation fails until the very last term in which all the contribution is accumulated. The program successfully catches the effect of the parameter p on the functions. It proves that the HDMR resolution does not give fruitful results

112

H. Kaya et al. / Computer Physics Communications 158 (2004) 106112

[8] M. Demiralp, A.A. Kanmaz, High dimensional model representation: a symbolic computer program for nding orthogonal components, 12. National Mechanics Congress, Konya, Turkey, 2001, in Turkish. [9] G. Li, S.W. Wang, H. Rabitz, Practical approaches to construct RS-HDMR component functions, J. Phys. Chem. A 106 (2002) 87218733.

[10] M. Kaplan, H. Kaya, H. Saygn, Parametric approximation for reactor criticality calculation using high dimensional model representation based methods via Monte Carlo simulation and interpolation techniques, in preparation. [11] I.M. Sobol, Theorems and examples on high dimensional model representation, Reliability Engineering System Safety 79 (2003) 187193.