526

IEEE TRANSACTIONS ON RELIABILITY, VOL. 46, NO. 4, 1997 DECEMBER

Semi-Markov Models with an Application to Power-Plant Reliability Analysis

Mihael Perman University of Ljubljana, Ljubljana Andrej SenegaEnik University of Ljubljana, Ljubljana Matija Tuma University of Ljubljana, Ljubljana
Key Words - Markov process, Semi-Markov process, Transition probability, Weibull holding time, Power-plant operation, Availability. Summary & Conclusions - Systems with, 1) a finite number of states, and 2) random holding times in each state, are often modeled using semi-Markov processes. For general holding-time distributions, closed formulas for transition probabilities and average availability are usually not available. Recursion procedures are derived to approximate these quantities for arbitrarily distributed holding-times;these recursion procedures are then used to fit the semi-Markov model with Weibull distributed holding-times to actual power-plant operating data. The results are compared to the more familiar Markov models; the semi-Markov model using Weibull holding-times fits the data remarkably well. In particular comparing the transition es shows that the probability of the system being in the state of refitting converges more quickly to its limiting value as compared to convergence in the Markov model. This could be because the distribution of the holding-times in this state is rather unlike the exponential distribution. The more flexible semi-Markov model with Weibull holding-times describes more accurately the operating characteristics of power-plants, and produces a better fit to the actual operating data. processes have been extensively studied; see [3] for a review. Section 2 considers the problem of computing the transition probabilities for semi-Markov processes. Analytic formulas in [4] are in terms of infinite series of convolutions of the semi-Markov kernel; Laplace transforms are used to give closed form expressions. Unfortunately, these Laplace transforms can rarely be inverted analytically. Numerical computation of inverse Laplace transforms for a related problem concerning the distribution of the number of u p states in a given time interval is in [5]. To avoid Laplace transforms, we derive a recursion formula for the Markov renewal kernel when the holding-times have a discrete lattice distribution, and then use this kernel to compute the transition probabilities. We obtain a recursion formula for the average system availability. Section 3 applies the recurrence formulas to approximate the transition probabilities and related quantities when the holding-times have a Weibull distribution with parameters depending only on the present state. The continuous Weibull distribution is first replaced by a lattice distribution closely resembling it. This lattice distribution is a slight modification of the discrete Weibull distribution [6]. The parameters in the semi-Markov model are estimated with Weibull holding-times from recorded holdingtimes in states for the Slovenian coal-fired PP Sostanj (294 MW unit) and examine the fit of the model. (The Markov model for this data-set hm been considered in [7].) The parameters of the Weibull holding-times and the transition probabilities of the embedded Markov chain are estimated and used in the recursion. The fit is then compared to the fit of the Markov model.

1. 1NTR.ODUCTION Acronym' PP power-plant.

Continuous-time Markov chains are widely used in the state-space modeling of reliability and system-availability for electric power systems [l, 21. The state-space approach applies also to modeling PP operation and to analyzing its reliability & availability. In Markov models the holding-times are exponentially distributed which is often too restrictive and might not fit the actual operating data well. Considering more general holding-times leads to semi-Markov processes which are less amenable to analytic treatment but provide more flexible models. Semi-Markov
lThe singular & plural of an acronym are always spelled the same.

Notation -

A E,{.} F,(t) G

average unavailability

E{.IYo

i}

G' i,j

Cdf{holding-time in state i} set of system states set of system u p states system states

0018-9529/97/$10.00 01997 IEEE

PERMAN ET A L SEMI-MPLRKOVMODELS WITH AN APPLICATION TO POWER-PLANT RELIABILITY ANALYSIS

527

lattice step Sf{holding-time in state j } indicator function: Z(True) = 1, Z(Fa1se) = 0 number of transitions transition matrix for the embedded Markov chain with entries p z , j conditional probability Pr{.IYo = i} Pr,{Y, = j } : transition probability at time t [defined in (2)] limtAw (t)]: transition probability from i to j for {X,} semi-Markov kernel with entries time-dependlent Q with entries, Q a , j ( t ) Markov renewal matrix with entries Ra,3 time-instant of transition n time, a r.v. time Weibull distributed r.v. embedded discrete Markov chain r.v. with values in G trajectory of the embedded Markov chain limiting availability limt-tw[At~, for j E G [scale, shape] parameter of Weibull distribution (EO, t]) = Ra,j(t):Markov renewal measure Z(i = j) equilibrium measures cardinality o F set A. Other, standard notation is given in Information for R,eaders & Authors at the rear of each issue. Figures 2 & 3 were produced with MATLAB [13].

that the semi-Markov kernel is of the form:

Qz,j(t)

= Pa,j

. Fz(t);

(3)

thus the distribution of the holding-time depends only on the present state. For Weibull holding-times, used in section 3, the semi-Markov kernel is:
Qa,j

( t ) = Pa,j * weif(t/~z; Pz).

(4)

3. The continuous-time process {yt : t 2 0) which describes the system state that is evolving according to the semi-Markov model is:

Y,

= X , on the interval T,

5 t < T,+l.

(5)

4. The holding-times are as defined in (2). 4 We compute Pr{Y, = jlY0 = i } , the average availability of such a system in a finite time interval [O,t], and the asymptotic behavior of the system as time 3 CO. To compute the availability or average availability of a system that evolves as a semi-Markov process, the transition probabilities need to be computed. The formulas for these transition probabilities have been studied extensively [4]. A brief derivation is given here. Definition 1. The R(t)over G is:
Ri,j(t)=
CO

E , { x Z ( X , =j).Z(T, E [O,t])};i,j E G ; t 2 0.
n=O

(6)

R.oughly, (7) comes from adding the probabilities of events of the form: {process Y visits j for occasion n in ( s , s ds), then stays in j until time t}. Given that Pr{X,+1 = j, T,+l - T, I t1X0,. . . ,X,; TO,. . . , T,} a transition to j occurs at time s, what happens after s = Pr{X,+1 = j , Tn+l- T, I tlX,}. (1) is conditionally s-independent of what happens before s. Then factor out the first term on the left in (7), viz, probAssumptions ability that no transition occurs in (t - s,t]; sum over n to 1. The process is homogeneous in the sense that there obtain Ri,j(t); and integrate over s. is a family of generalized Q i , j for i, j E G (a semi-Markov The average system-availability is: kernel) such that:

2. TRANSITION PR.OBABILITIES Following Cinlar [4], iz semi-Markov process is defined as a sequence of 2-dimens ional r.v., { (X,, T,) : n E 1 , 2 , . .. } with the properties: 0 X , is a discrete-time Markov chain taking values in a countable set of possible states G of the system and represents its state after transition n. 0 The holding-timles, T , - T,-1, between two transitions are r.v. whose distribution depends only on the present state X , and the state after the next transition:

The transition probabilities are given by theorem 1.

Theorem 1 [4]:

Pr{yt = jlY0 = i } =

The proof is in appendix A.1.

Qi,j(t)

Pr{X,+1 = j , %+I - T,

I tlx,

= i},

(2)

for any two states i , j : 6 G. 2. The holding-times, T,+1 -Tn, are positive with probability 1. While in (2) the holding-times can depend on the present & following states, this paper mostly assumes

To simplify the notation, use:

H j ( s ) = Prj{Tl > s } = 1 kEG

Qj,k(s).

(9)

528

IEEE TRANSACTIONS ON RELIABILITY, VOL. 46, NO. 4, 1997 DECEMBER

Corollary 1 comes from theorem 1. Corollary 1. For YO = i,

R,ewrite (12) for discrete holding-times on the lattice: R(m.h) =

jEG'

I + C [ Q ( l. h ) - Q ( ( I
1=1

- 1) . h)]x

R ( ( m- I ) h ) ,

(13)

The proof is in appendix A.2. Compute the limits of the average unavailability as t increases. Appendix A.3 treats the special case where the holding-time distribution depends only on the present state, The formula can be derived from [4];a short, simple derivation is given here. Derivation Eq (7) for transition probabilities is used for Weibull holding-times. Since the Weibull distribution does not have a simple Laplace transform, the kernel R ( t ) cannot be computed analytically. For computation, the Weibull distribution is replaced by an appropriate discrete distribution (see section 3). For discrete holding-times that take only values in the lattice { k . h : IC > 0}, (7) becomes: prz{yk ,h = j } =
k

R(0) = I; I E identity matrix

= c&,~).

This recursion is used in section 3 to approximate transition probabilities when the holding-times have the Weibull distribution. The continuous Weibull distribution is replaced by a discrete lattice distribution which closely 4 resembles it. 3. WEIBULL HOLDING-TIMES2 This section presents a model for PP operation with 6 states and Weibull holding-times. The transition probabilities are computed with ( l l ) , and the parameters for the Weibull distribution and the transition probabilities for the embedded Markov chain have been estimated from real operating data. 3.1 Model Assumptions for This Example

[R2,J(Z. h) - Rz,J((Z - 1 ) .h ) ]. H ( ( k - 1 + 1 ) .h ) . (11)

z=o

Because R,,J is a jump function for any i , j , the integral becomes a sum. Let Rz,J(-h)= 0. R(t) can be computed recursively:
00

R,,j(t) = E , { Z
n=O
00

qxn = j )

Z(Tn E [O,tl)

= E , { E i { C Z ( X , = j ) .x(Tn E [O,t])ITi,Xi}}
n=O
00

Figure 1. PP Model with Possible Transitions

A. The system states and the possible transitions between them are in figure l. B. The 6 system states are [9, IO]: #I. Operating state (up). #a. Stoppage due to low power-demand (up). #3. Boiler failure (down). #4, Turbine failure (down), #5. Stoppage due to states other than #2 - #4 (down). #6. Refitting (down). C. The only transitions from down states, with the exThe last line of the equations follows from the precedception of the transition from state #3 to #2, are into ing one by the semi-Markov property of the sequence state 1. (This simplification is chosen on the basis of the { ( X n , T n ) : n 2 0). For an alternative approach see [8] operating data, ie, other transitions rarely occur.) Howwhere integral equations for the transition probabilities ever, the formulas from section 2 do apply to arbitrary can be obtained using the case where the set of up states contains only one state, and point availabilities rather than transit ions. interval availabilities are considered. However, a, different 2 T h e number of significant figures is not intended to imply any acset of integral equations would have to be solved for each curacy in the estimates, but t o illustrate t h e arithmetic. T h e Pascal
=&,J

+Ez{CZ(Xn =j).Z(Tn E [O,t])lTl,X1}

n=l

PZJ (t).

code used for numerical computations is available from the authors.

PERMAN ET A L SEMI-MARKOV MODELS WITH AN APPLICATION TO POWER-PLANT RELIABILITY ANALYSIS

529

D. The holding-times are Weibull distributed with parameters depending only on the current state. The Cdf of the holding-time between states i & j is therefore:
weif(u/qi; P i ) , i E G. (14)

closely resembles it. The Weibull r.v., W, is replaced by a discrete r.v., W, taking values in h, 2h,. . . such that: Pr{W = k . h} = Pr{(k - 1 ) .h

< W 5 k . h},

fork=1,2,

....

(17)

The mean of a r.v. with this Cdf is pi =-vi . r(l 1/&). E. The Slovenian coal-powered PP SoStanj 294 MW is the demonstrative example. Operating data for 1979 1996 are available, but the analysis is only for 1979 - 1991 1101. 3.2 Model Evaluation The transition probabilities of the embedded Markov chain are estimated by (15) and are shown in table 1.
@i,j

This approach is often used for numerical computation. The distribution (17) is suggested in [12] where further references are given. The distribution of W is also related to the discrete Weibull distribution as defined in [6]; ((Wlh) - h ) has the discrete Weibull distribution with suitable parameters. It is plausible to anticipate that, for small h, the transition probabilities obtained with this approximation closely resemble the transition probabilities for the original Weibull distributions. Indeed, one can easily prove that:
h-0

= ni,j/ni*

(15)

lim [Pri{y,= j } ] = Pri{x = j } , for all i , j .

Notation FT Fisher information matrix n,,3 number of transitions from state i to state j in the period in assumption E 72% number of all transitions from state i into
any other state.
Table 1. State-Transition Probabilities of the Embedded Markov Chain [PP Sostanj 294 MW]
lj1,2

With this discrete approximation, R(t) is computed via the recursion formula (13), and the transition probabilities are computed from (11). Figures 2 & 3 show the computed transition probabilities along with the analogous quantities for the Markov model with exponential holding-times. See [7] for the details of the exponential case.
Probability P(l ,l)

lji,6

= 0.43 lj1,~ = 0.36 0.01 $3,1 0.91

lj1,4 = 0.06 lj3~ = 0.08

51,s = 0.11

The parameters of th,e Weibull holding-times were estimated by maximum likelihood assuming the observations of the holding-times in at given state are i.i.d. The standard deviations were estimated from the asymptotic variances given by the inverse of FI,which for the Weibull distribution is (the prime implies the derivative):

Table 2 gives the point estimates and their estimated standard deviations [ll:i.
Table 2. Estimated Pairameters and Standard Deviations of the Weibull Distribution [PP Sostanj 294 MW]
I 2000

0.751

lo00

[the estimated standard deviation is in ( )] .,8 4 state #1 0.78 (0.031) 235.6 (16.2) #2 0.94 (0.057) 59.5 ( 5.1) #3 0.88 (0.057) 31.2 ( 3.1) #4 0.41 (0.065) 8.7 ( 4.3) #5 0.59 (0.068) 3.2 ( 0.82) #6 4.68 (1.381) 1174.0 (99.6)

l[hl

[for P(1,1)1 Weibull (solid), Exponential (dashed) Figure 2. Transition Probability The convergence of Pr,{Yt = j } to the equilibrium values is faster than in the exponential model. At t = 2000 hours there is a remarkable closeness between the theoretical limits and the transition probabilities in [4]. Table 3 gives numerical results. The Weibull model fits the actual data slightly better than the exponential model. For example, the availability of the PP in question over the 12 years

For the Weibull holding-times, closed formulas for R ( t ) are not available. As an approximation, the Weibull distribution has been replalced by a discrete distribution that

530

IEEE TRANSACXIONS ON RELIABILITY, VOL. 46, NO. 4, 1997 DECEMBER

was 86.6%. The semi-Markov model predicts the average availability computed from (10) as 89.7% whereas the Markov model predicts 90.5%. Formulas for average unavailability in the semi-Markov case are in appendix A.3.
Table 3. Transition Probabilities and Theoretical Limits [PP SoEtanj 294 MW]

E,{z(T,
n_>O

10, t ] ). z(xn =j )
> t - TnIX Tn}}.
(A-1)

.Prz{Tn+l - T n

state #1 #2 #3 #4

#5
#6

Prl{Yzooo = j } 0.81 0.08 0.03 0.00 0.002 0.06

Prl{Y, 0.81 0.08

=j }

However, given X, the holding-times are, by assumption, conditionally s-independent; hence the conditional probability in the last line in (A-1) becomes:

0.04 0.005 0.002

0.06

1LEG

Q x , ~ ,I~ Tn). (~

(A-2)

Use (A-2) in (A-1): Prz{yt = j } =

n>O

E,{Z(Tn E [0,t ] ) . Z(Xn = j )

Probabilities P(1,i) for 1=2,3,6

kEG

Q~,,,k(t - Tn))}

=
n20

E,{Z(Tn E [O, tl) * Z ( X n = 3 )

(A-3)
pL,,J is a measure on [ O , o o ) such that for every measur-

able A
[for P(1,a), P ( 1 , 3 ) ,P(1,6)1 Weibull (solid), Exponential (dashed) Figure 3. Transition Probability

C [0,CO):
nT0

P ~ , ~ ( A= )
=
-

L
n>O

E,{z(x,= j

> l ~ =.S } . Pr,{T, E d s )

E;{Z(X, = j ) . Z(Tn E A ) }

ACKNOWLEDGMENT We are pleased to thank the referees whose remarks considerably improved an earlier version of the paper. In particular we thank Dr. A. Csenki for his remarks and for pointing out to us a wealth of references in the field.

n2O
= E I { E Z ( X n = j ) .Z(Tn E A ) } .

The R'2,J(t) = ~ i , ~ ( [ O is , t the ] ) Cdf of pz,j. A.2 Proof of (10)

Q.E.D.

At

PERMAN ET A L SEMI-MARKOV MODELS WITH AN APPLICATION TO POWER-PLANT RELIABILITY ANALYSIS

531

All the integration changes are justified because the funcOn the other hand, the occupation measures in (A-8) tions integrated are positive. Q. E.D. & (A-9) are proportional to the equilibrium measure T for all starting states k : A.3 Limiting Average Unavailability The asymptotic availability of a system modeled by a semi-Markov process is important. It is sufficient to find the limit: lim
t+03

[:

. l Z ( X 8= i ) d s , for all i E G.

Individual holding-times depend on the state, but given that the states are conditionally s-independent, then

(A-5)

~ ( ~ 1 = 1E{
=
3

(
m<r1

mTm-1))
711)

The average availabilities computed in appendix A.3.2 equal the limiting transition probabilities, if these exist. The asymptotic availabilities in table 3 are computed from (A-9).

E {#[visits to j before

. E{holding-time in j
= C C k
3

*TJ .vj;

(A-9a)
71))

A.3.1 Theorem 2 Let: 0 { (X,, T,) : n 2 0) be a semi-Markov process; 0 the embedded Markov chain {X,} be irreducible, recurrent, and aperiod,ic with equilibrium measure T ; 0 depend ) only on the the semi-Markov kernel Q i , 3 ( ~ initial state i; v, = So s . F,(ds). 4 Then, with probability 1,

E{&}

= E{#{visits to 3 before

.E{holding-time in j }
ck

V.J.

(A-9b)

Substitute (A-9) into (A-7). R.EFER.ENCES

Q.E.D.

s-independently of X O for every a.

A.3.2 Proof of theorem 2

Notation

> T,-.I : X m = k } : stopping times Tr9, - Trfl-l. for n 2 0: sum of all the holding-times during k-block n Sn Cr1L-1<m<71, z(Xm =j ) . - Tm), for n 2 0: time in state j during k-block n It } . N ( t ) max{n : (Cr=oDz) Let X O = k . By assumption, {X,} is recurrent. The ( 7 , ) are well defined. Name the portion of the embedded Markov chain between ~ , - 1 and r, a k-block. The k-blocks are s-independent by strong Markov property; hence the sequence of random pairs: {(Dn,Sn) : R. 2 01, has s-independent terims. The reward renewal theorem [14] states that, with probability 1,
r,

70

0 inf{m

D,

[l] J. Endrenyi, Relzabzlzty Modelzng zn Electrzc Power Systems, 1978; John Wiley & Sons. [2] R. Billinton, R. Ringlee, A. Wood, Power System Relzabzlzty Calculatzons, 1973; MIT Press. [3] A. Csenki, Dependabzlzty f o r Systems wzth a Partztzoned State Space - Markov and Semz-Markov Theory and Computatzonal Implementatzon, 1994; Springer-Verlag. [4] E. Cinlar, Introductzon to Stochastzc Processes, 1975; Prentice-Hall. [5] A. Csenki, Occupation frequencies for irreducible finite semi-Markov processes with reliability applications, Computers & Operations Research, vol 20, num 3, 1993, pp 249-259. [6] T. Nakagawa, S. Osaki, The discrete Weibull distribution, I E E E Trans. Relzabilzty, vol R-24, 1975 Dec, pp
300-301. [7] A. Senegdnik, M. Tuma, Operating reliability of a conventional power plant, Relzabzlzty Engzneenng & System Safety, vol 37, num 3, 1992, pp 211-215. [8] A. Csenki, An integral equation approach to the interval reliability of systems modeled by finite semi-Markov processes, Relzabzlzty Engzneerzng & System Safety, vol 47, num 1, 1995, pp 37-45. [9] J. Oman, D. Dejanovit, M. Tuma, Thermal power testing of unit 4 in Thermal Power Plant Sobtanj, Proc. 4th Intl Expert Meetzng: Power Engzneenng, 1996; Maribor,

(A-7)
provided that: D, > 0 for all n, E{D1} < 03, E{Sn} < c a , which is true for models with continuous holding-times with finite s-expectations.

Slovenia. [lo] Power Plant Sofitanj 1979-1991 Review and Operating Analysis (Slovene text), Power Plant Soitanj Annual Bulletzn, 1979 - 1991; vSoStanj, Slovenia. [ll] M. Abramowitz, A. Stegun, Handbook of Mathematzcal Functzons, 1970; Dover Pub.

532

IEEE TRANSACTIONS ON RELIABILITY, VOL. 46. NO. 4, 1997 DECEMBER

[12] A. Csenki, Cumulative operational time analysis of finite semi-Markov reliability models , Relaabilaty Enganeenng t Y System Safety, vol 44, num 1, 1994, pp 17-25.
[13] M A T L A B 4 . 2 ~ Users Guide, 1994 Dec; Mathworks

[14] S. Asmussen, Applied Probability and Queues, 1987; John Wiley & Sons. AUTHORS Dr. Mihael Perman; Faculty of Mechanical Engg; Univ. of Ljubljana; ABkerEeva 6, 1000 Ljubljana, SLOVENIA. Internet (e-mail): mihael,perman@fmf.uni-lj.si Mihael Perman (born 1961) received his PhD (1990) from the Statistics Department at the University of California at Berkeley. Since then he has held positions at the University of Ljubljana, University of Southern California, and Cambridge University. He now teaches at the Faculty of Mechanical Engineering, University of Ljubljana. His research interests include stochastic processes, Brownian motion, and mathematical statistics. Dr. Andrej Senegainik; Faculty of Mechanical Engg; Univ. of Ljubljana; ASkerEeva 6, 1000 Ljubljana, SLOVENIA.
Internet (e-mail): andrej.senegacnik@fs.uni-ljsi

Andrej SenegaEnik (born 1961, in Ljubljana) received his BSc (1984), MSc (1991), and his doctoral degree (1995) at the Faculty of Mechanical Engineering Science at the University of Ljubljana. He teaches at the Faculty of Mechanical Engineering in Ljubljana. His research interests are in thermodynamics, power systems, and reliability of power plants.
Dr. Matija Tuma; Faculty of Mechanical Engg; Univ. of Ljubljana; ASkerEeva 6, 1000 Ljubljana, SLOVENIA. Matija Tuma (born 1938, in Ljubljana) received his BSc (1962) from the Faculty of Mechanical Engineering, University of Ljubljana, and his Dr.Tech.Sci (1978) from the Federal Institute of Technology (ETH) Zurich, Switzerland. He worked for several companies in Slovenia & Switzerland for 14 years before joining the Faculty of Mechanical Engineering of the University of Ljubljana as a Full Professor of Thermal Power Plant Engineering & Thermodynamics in 1982. His research interests include gas-steam cycles, cogeneration of heat & power, (in)organic Rankine cycles, and reliability of power plants. He has (co)authored over 50 technical papers.

Manuscript TR96-077 received 1996 May 28; revised 1997 February 12, 1997 July 22 Responsible editor: O.G. Qkogbaa Publisher Item Identifier S 0018-9529(97)08545-X