Applied Mathematical Sciences, Vol. 3, 2009, no.

11, 509 - 517

Bayesian Estimation for the Pareto Parameters

Using Quasi-Likelihood Function

M. S. Youssef

The Higher Technological Institute

yousefselim@yahoo.com

Abstract

In this paper, Bayesian estimation for the Pareto distribution, this thesis aims
to investigate the possibility of using the quasi-likelihood function in the Bayesian
approach. We introduce a new method which we called "Quasi-Likelihood
Estimation". This method reduces to the usual Bayesian estimation if the distribution
is a member of the exponential family. We use the maximum quasi-likelihood
estimating the unknown parameters of the Pareto distribution.

Keywords: Linear Model; Bayesian Method of Estimation; Quasi-Likelihood

1- Introduction
The quasi-likelihood function was introduced by Wedderburn (1974), to be
used for estimating the unknown parameters in generalized linear models. The idea of
quasi-likelihood weakens the assumption that we know exactly the distribution of the
random component in the model, and replace it by an assumption about how the
variance changes with the mean.
The quasi-likelihood function could be used for estimation in the same way as
the usual likelihood function. Wedderburn (19974) and McCullagh (1983) showed
that the maximum quasi-likelihood estimates have many properties similar to
particular, the maximum quasi-likelihood estimate of the vector β (the vector of
parameters in regression models) is a asymptotically normal with mean β , and
asymptotic covariance may be derived in the usual fashion from the second derivative
matrix of the quasi-likelihood function. Also, if the underlying distribution comes
from a natural exponential family the maximum quasi-likelihood estimate maximizes
the likelihood function and so it has full asymptotic efficiency; under more general
distributions there is some loss of efficiency, which had been investigated by Firth
(1987) and Hill & Tsai (1988).
510 M. S. Youssef

We introduce the maximum quasi-likelihood estimates of the unknown

parameters of the Pareto distribution and our new quasi-Bayesian method of
estimation.

2- Quasi-Bayesian Estimation
In the present section, we present a new Bayesian estimation technique. We
can apply it without specifying the likelihood function of the sample observations, if
the relationship between the mean and the variance is know. That is, to construct a
posterior distribution the likelihood function could be replaced with the natural
exponential of the quasi-likelihood function. This method reduces to the usual
Bayesian estimation if the quasi-likelihood and the log-likelihood function are
identical.
Let x1 , x 2 , L , x n be an independent random sample, with mean μ = μ (θ ) ,
where θ is a vector of parameters, and the variance var ( x ) = φ V (μ ) , where V (.) is
some know variance function, and φ is a dispersion parameter which could be known
or unknown. The quasi-likelihood Q ( x; μ , φ ) can be derived as defined by the
relation;
∂ x − μi
Q ( xi , μ i ) = i , (A)
∂ μi V ( μi )
and the natural exponential of Q ( x; μ , φ ) will be used as a likelihood function. Using
⎛ φ⎞
a suitable prior density g (θ , φ ) the posterior distribution f * * ⎜θ , ⎟ can be
⎝ x⎠
constructed as

f * * (θ , φ x ) ∝ ⎛⎜ ∏ { exp [Q ( x; μ , φ )] }⎞⎟ g (θ , φ )
n
(2.1)
⎝ i =1 ⎠
where μ = μ (θ ) , θ = (θ 1 , θ 2 , L , θ n ) , θ i ∈ Θ , Θ is the parameter space,
i = 1, 2 , L , n , and φ > 0 . The joint mode of (2.1) can be used to obtain a Bayes
estimate for θ , and marginal densities may be obtained from (2.1) by integrating out
one or more of the unknown parameters, another set of Bayes estimates for the
unknown parameters can be obtained with respect to some loss functions, using these
marginal densities.
Now, let us apply the above method for a simple variance function. Suppose
that x1 , x 2 , L , x n is an independent random sample from an unknown distribution and
suppose that the variance and the mean are known to be proportional and the constant
of proportionality is φ = 1 , so that
Ε (x μ ) = μ , and Var (x μ ) = μ V (μ ) . (2.2)
Where V (μ ) denotes the variance function from (A), we have
Q (x; μ ) = log μ − μ and
Bayesian estimation for the Pareto parameters 511

e x p [ Q ( x; μ )] = μ x θ − μ . (2.3)
In order to construct a posterior distribution, a natural conjugate prior density of μ
is
g (μ ) = (μ υ )υ e − μ υ ,υ > 0 , μ > 0
1
(2.4)
μ Γυ
Using (2.3) and (2.4), the posterior probability distribution function (p. d. f) of μ is
⎛∏ n x −μ ⎞ υ −1 −μ υ
⎜ μ ie ⎟ μ e
⎝ i =1 ⎠
f * * (μ / x1 , x 2 , L , x n ) = ∞
⎛ n x −μ ⎞ μ −1 −μ υ
∫ ⎜∏ μ ie ⎟ μ e ∂μ
0 ⎝ i =1 ⎠
λ
= (λ μ )θo −1 e −λ μ , θ o , λ, μ > 0 (2.5)
Γ(θ o )
o
Where θ o = ∑ x i + υ and λ = n + υ .this, corresponding to a quadratic, loss function,
i =1
the quasi-Bayesian estimator is the posterior mean for (2.5) viz

μ ** = Ε(μ / x ) =
∞ (λμ )o e −λμ ∂μ = θ o
∫ (2.6)
0 Γ (θ o ) λ
the Bays risk of μ ** is
∞
(
var(μ / x ) = ∫ μ − μ ** )2
f ** (μ / x )∂μ =
θo
(2.7)
0 λ3
another Bays estimator is the mode of the posterior p. d. f (2.5), viz
θ −1
μ= o (2.8)
λ
It may be noted that the variance function V (μ ) = μ is associated with the Poisson
distribution which is a member of the exponential family and so the natural
exponential of the quasi-likelihood and the likelihood function are equivalent and this
make the quasi-Bayesian results identical to the usual Bayesian results.

3: Quasi-Likelihood Estimation for the Pareto Distribution

In this section, we derive the maximum quasi-likelihood estimates, and the quasi-
Bayesian estimates for the unknown parameter of the Pareto distribution. This
distribution is useful here because its quasi-likelihood and its log likelihood function
are different, and this property enables us to make a comparison between the quasi-
Bayesian method and the likelihood Bayesian estimation.
3.2.1: Quasi-Likelihood estimation:
Consider the p.d.f of the Pareto distribution given by
f ( y ) = αk α y − (α + 1) , α > 0, y ≥ k > 0 (3.1)
and let Ε( y ) = μ , then
512 M. S. Youssef

αk
μ= (3.2)
(α − 1 )
and the variance is
α −1 α −1
var ( y ) = μ2 = V (μ ) (3.3)
α (α − 2 ) α (α − 2 )
where V (μ ) = μ 2 is the variance function. Thus for a sample of size n , the
quasi-likelihood function is given by
n
∑ yi − n μ
∂Q i =1
= (3.4)
∂μ μ2
which gives
n
− ∑ yi − n μ
Q ( y; μ ) = i =1
(3.5)
μ − n log μ
Substituting for μ from (3.2) in (3.5), the quasi-likelihood function (3.5) as a
function of α , k , becomes

) = − α − 1 ∑ y i − n log ⎜⎜ α − 1 ⎟⎟
⎛ ⎞
Q (y ; k , α
n
(3.6)
αk ⎝ αk ⎠
i =1

The maximum quasi-likelihood estimates of α , k , are derived as follows

n
∑ yi
∂Q ∂Q ∂μ −k nk
= = i =1 2 +
∂α ∂ μ ∂α μ (α − 1 ) 2
μ (α − 1 )
n
∑ yi
n
= − i =1 (3.7)
α (α − 1 ) α 2 k
Equation the above derivative to zero, we obtain the quasi-likelihood estimates
of α , denoted by α~ , as
n
∑ yi
i =1
α~ = − n
(3.8)
∑ yi − n k
i =1
Next, since k is a lower bound on the random variable y , Q ( y ; k , α ) must be
maximized subject to the constraint
~
k ≤ min y i (3.9)
~
By inspection the value of k which maximizes (3.6) subject to (3.9) is
~
k = min y i (3.10)
Thus (3.8) and (3.10) give the maximum quasi likelihood estimates of α and
k.
Bayesian estimation for the Pareto parameters 513

~ nα
We note that k is the maximum likelihood estimate of k , but α~ is not, , the
(n − 2 )
same estimates (3.8), and (3.10), could be derived from the extended quasi-likelihood
function associated with the variance function (3.3).
3.2.2 Quasi-Bayesian estimation for the Pareto distribution:
In this sub-section, the method of quasi-Bayesian estimation is applied to the
Pareto distribution. From (3.6), the natural exponential of the quasi-likelihood
function for a sample of size n observation is from the Pareto distribution is given by
⎛ α −1 ⎞
− D o ⎜⎜ ⎟⎟
exp (Q ( y1 ; α , k )) = α −n
(α − 1) n
k −n
e ⎝αk ⎠
α > 1 , 0 < k ≤ D1 (3.11)
where Do = ∑ y i , and D1 = min ( y i )
o

i =1
now, we derive three posterior distributions of α and k ; when α is known
and k is unknown, when k is known and α is unknown, and when both α and k
are unknown.
Case (1): α is known and k is unknown
Since α is known and k is unknown, a prior P.d.f of k is provided by (3.2.1), as
f 1 (k ) = a o bo o k ao − 1
−a
0 < k < bo (3.12)
where a o , bo are positive. From (3.11) and (3.12) the posterior P.d.f of k is
k D2 − D3
f 1* ( k α y ) = e k
0 < k ≤ D4 (3.13)
A1
α −1
where D 2 = a o n − 1 , D3 = Do , D 4 = min (bo − D1 ) and A1 is the
α
normalizing constant, given by
D4 − D3 k
A1 = ∫ k D2
e ∂k (3.14)
0
i.e. A1 is an incomplete gamma function, and could be calculated numerically.
The mode of (3.13), which is a Bays estimate of k and is
n
∑ yi
⎛ α − 1 ⎞ i =1
k* =⎜ ⎟ (3.15)
⎝ α ⎠ n − ao + 1
Also, the med ian of (3.13) is another Bays estimate of k and is given by
k **
k 2 − D3 k
D
∫ e ∂ k = 0.5 (3.16)
0 A1
Where k ** is the median of (3.13).
Still another Bays estimate of k is the mean of the posterior p.d.f (3.13), viz
1 D 2 +1 − D3 k
Ε (k α , y ) = k e ∂ k = 0.5 (3.17)
A1
The corresponding Bays risk is the variance of the posterior p.d.f (3.13), viz
514 M. S. Youssef

D3
2 D2 −
var (k α , y ) =
1 k
∫ [k − Ε (k α , y )] k e k
∂ k (3.18)
A1 0
The median, the mean and the variance given by (3.16), (3.13) and (3.18)
respectively.

Case (2): α is unknown and k is known

Since k is known and α is unknown, we may use f (α ) = αα c0 −1 e − d 0 α , α > 0 as a
prior P.d.f of α , so that
f 2 (α ) = α α c0 −1 e − d 0 α α >0 (3.19)
Where c o , d o are positive. Thus, the posterior P.d.f of α is
(α − 1)n α D5 ⎧ ⎛ α − 1 ⎞⎫
−
(α k y ) =
f 2* exp ⎨− d 0 α − D6 ⎜ ⎟⎬ α > 1 (3.20)
A2 ⎩ ⎝ α ⎠⎭
n y
Where D5 = c o − n − 1 , D6 = ∑ i , and A2 is the normalizing constant, given by
i =1 k
∞ ⎧ ⎛ α − 1 ⎞⎫ −
A2 = ∫ (α − 1) α D5 exp ⎨− d 0 α − D6 ⎜ ⎟⎬∂α
n
(3.21)
α >1 ⎩ ⎝ α ⎠⎭
The mode of (3.20), is given by
d0 α * ( )
3
( )
− (n + D5 + d 0 ) α * + (D5 + D6 )α * − D6 = 0
2
(3.22)
Also, the median of (3.20) is given by
ε **
(α − 1)n α D5 ⎧ ⎛ α − 1 ⎞⎫
∫ exp ⎨− d 0 α − D5 ⎜ ⎟⎬∂α = 0.5 − (3.23)
α >1 A2 ⎩ ⎝ α ⎠⎭
Still another Bayes estimate of k is the of the posterior mean and variance of
(3.20) are
∞
Ε ( α k y ) = ∫ α f 2* (α k y ) ∂α − (3.24)
α >1
and
∞
var ( α k y ) = ∫ [α − Ε ( α k y )] f 2* (α k y ) ∂α −
2
(3.25)
α>1

The mode, the median, the mean and the variance given by (3.28), (3.23) and (3.24)
and (3.25) respectively.
We calculate the quasi-Bayesian estimates and the coefficient of variation for k when
α is known which is given by (3.17) and (3.18). also the corresponding results of
Bayesian estimation are obtained and theses results are listed in table (3.1).
Table.(3.2) summarizes the results of Bayesian and quasi-Bayesian estimation for α
when k is known, which is given by (3.24) and (3.25). tables, (3.3) and (3.4) gives
the efficiency of the quasi-Bayesian estimation of k when α is known, and of α
when k is known, respectively.
Bayesian estimation for the Pareto parameters 515

Table (3.1): Bayesian and Quasi-Likelihood Estimation for Pareto Data for k when
α is known (α = 2)

Sample size n = 10 n = 20 n = 30

Prior C.V. of C.V. of C.V. of C.V. of C.V. of C.V. of

Parameters Bays Quasi- Bays Quasi- Bays Quasi-
a0 b0 estimate Bays estimate Bays estimates Bays
s of k estimates s of k estimates of k estimates
of k of k of k
1 1 0.0465 0.1640 0.0326 0.1630 0.0161 0.0912
2 2 0.0443 0.1554 0.0239 0.1549 0.0156 0.0910
0.5 3 0.0471 0.1703 0.0246 0.1640 0.0190 0.0980
0.42 3.1 0.0476 0.1709 0.0246 0.1656 0.0163 0.0981
0.6 3.1 0.0472 0.1690 0.0244 0.1652 0.0160 0.0931
1 3.2 0.0453 0.1625 0.0240 0.1632 0.0157 0.0914
2 3 0.0406 0.1516 0.0236 0.1548 0.0153 0.0872
3 3.5 0.0432 0.1394 0.0231 0.1459 0.0151 0.0834

Table (3.2): Bayesian and Quasi-Likelihood Estimation for Pareto Data for α when
k is known (k = 2)

Sample size n = 10 n = 20 n = 30

Prior C.V. of C.V. of C.V. of C.V. of C.V. of C.V. of

Parameters Bays Quasi- Bays Quasi- Bays Quasi-
c0 b0 estimate Bays estimate Bays estimates Bays
s of k estimates s of k estimates of k estimates
of k of k of k
0.5 0.5 0.0455 0.1601 0.0317 0.1634 0.0160 0.0911
1 0.75 0.0433 0.1552 0.0228 0.1548 0.0153 0.0910
1 1 0.0462 0.1701 0.0239 0.1644 0.0189 0.0980
1.5 1.5 0.0466 0.1690 0.0238 0.1680 0.0161 0.0983
2.1 1 0.0463 0.1622 0.0234 0.1650 0.0160 0.0931
2 1.1 0.0462 0.1582 0.0240 0.1622 0.0159 0.0913
2.1 1.5 0.0404 0.1512 0.0232 0.1548 0.0156 0.0870
2 2 0.0431 0.1393 0.0230 0.1456 0.0152 0.0834
516 M. S. Youssef

Table (33): Efficiency of Quasi- Bayesian Estimates of the Parameter α when k is

known (k = 3) .

Prior Parameters n = 10 n = 20 n = 20
c0 b0
0.5 0.5 0.6136 0.6084 0.2280
1 0.75 0.6342 0.3900 0.2199
1 1 0.6990 0.2512 0.2161
1.5 1.5 0.8069 0.3047 0.2190
2.1 1 0.6152 0.1923 0.2101
2 1.1 0.6452 0.2100 0.2109
2.1 1.5 0.7123 0.2469 0.2010
2 2 0.8086 0.3590 0.2231

Table (3.4): Efficiency of Quasi- Bayesian Estimation of the Parameter k when α is

known (α = 2 ) .

Prior Parameters n = 10 n = 20 n = 20
a0 b0
1 1 0.1158 0.0620 0.0381
2 2 0.1173 0.0363 0.0371
0.5 3 0.1126 0.0365 0.0425
0.42 3.1 0.1147 0.0367 0.0312
0.6 3.1 0.1147 0.0363 0.0372
1 3.2 0.1150 0.0360 0.0380
2 3 0.1172 0.0368 0.0399
3 3.5 0.1270 0.0375 0.0415

Tables. (3.1) and (3.2) shows that the difference between the coefficients of variation
of the Bayesian estimates and the quasi-Bayesian estimate is small, further the
difference is decreases as the sample size and the values of the prior parameters
increases. also, the coefficient of variation of the quasi-Bayesian estimates of α is
smaller then the coefficient of variation of the Bayesian estimate of α .
Table. (3.3) shows that the quasi-Bayesian estimate of the parameter k has a low
efficiency relative to the Bayesian estimate, also the efficiency decreases as the
sample size increases.
Table. (3.4) shows the quasi-Bayesian estimate of the parameter α has a high
efficiency even for small sample size and large values of prior parameters. But, the
efficiency decreases as the sample size increases.
Bayesian estimation for the Pareto parameters 517

References

1- Arnold, B.C. and Press, S. J. (1983). Bayesian inference for Pareto population.
Journal of Econometrices. 21, 287-306.
2- Arnold, B.C. and Press, S. J. (1986). Bayesian analysis of censored or grouped date
from Pareto population in Bayesian inference and decision techniques. Eds. P.
Goel and A. Zellner, Amsterdam: North Holland, pp. 157 – 173.
3- Arnold, B.C. and Press, S. J. (1989). Bayesian estimation and prediction for Pareto
data. JJASA, vol. 4. No. 408, 1079 – 1084.
4- Hill, J. R. and Tsai, C.L. (1988). Calculating the efficiency of maximum quasi-
likelihood estimation. Apple. Statist. 37, 2, 219 – 230.
5- McCullagh, P. and Nelder, J.A. (1983). Generalized linear models. London:
Chapman and Hall.
6- Wedderburn, R.W. M. (1974). Quasi-likelihood functions, generalized linear
models, and the Gauss-Newton method. Biometrika 61, 439-447.

Received: November, 2008