Escolar Documentos
Profissional Documentos
Cultura Documentos
Abstract—IEEE Standard 1366 offers a method for segmenting To address these concerns, utilities and their regulators have
reliability performance data to isolate the effects of major events developed methods for segmenting the reporting of reliability
from the underlying year-to-year trends in reliability. Recent anal- performance using a concept known as “major events.” Major
ysis by the IEEE Distribution Reliability Working Group (DRWG)
has found that reliability performance of some utilities differs from events are defined periods during the year when the electricity
the expectations that helped guide the development of the Standard delivery system is exposed to stresses that are beyond normal
1366 method. This paper proposes quantitative metrics to evaluate expectations, typically as a result of severe weather. Identifying
the performance of the Standard 1366 method in identifying major them explicitly provides a means for treating them separately
events and in reducing year-to-year variability in utility reliability. in reviewing annual performance metrics. The development of
The metrics are applied to a large sample of utility-reported reli-
ability data to assess performance of the method with alternative these methods is considered foundational in providing a cred-
specifications that have been considered by the DRWG. We find ible, historical assessment of utility reliability. This paper ex-
that none of the alternatives perform uniformly “better” than the amines the method used by industry to identify major events; it
current Standard 1366 method. That is, none of the modifications does not explore the causes of major events.
uniformly lowers the year-to-year variability in System Average Early on, the definitions for major events were based on the
Interruption Duration Index without major events. Instead, for
any given alternative, while it may lower the value of this metric local experiences of utilities, their regulators, or an oversight
for some utilities, it also increases it for other utilities (sometimes board. As a result, there was considerable variation in how each
dramatically). Thus, we illustrate some of the trade-offs that must utility defined a major event [1], [2]. In the late 1990s, the Insti-
be considered in using the Standard 1366 method and highlight tute of Electrical and Electronics Engineers (IEEE) Distribution
the usefulness of the metrics we have proposed in conducting these Reliability Working Group (DRWG) pioneered the development
evaluations.
of a statistically-based, standard method for identifying major
Index Terms—Electricity reliability, IEEE Standard 1366, major events. The method relies on identifying what are called “ma-
events, power system reliability, reliability, system average inter-
ruption duration index. jor event days” or MEDs. The method was developed based on
an expectation that it would identify, on average, 2 to 5 MEDs
I. INTRODUCTION per year, a notable limitation to using this method, as not all utili-
EVERE, yet infrequent, events (often the result of weather) ties necessarily fall within this expected range. For the most part,
S can dramatically affect the year-to-year reliability perfor-
mance of a utility. Such unforeseen events can cause widespread
however, this approach represents the vast majority of utilities.
The method is now embodied in the IEEE voluntary Standard
and prolonged interruptions of electric service on only a handful 1366 [3]–[5]. (For brevity, we shall refer to this as the Standard
of days, which can easily exceed the cumulative interruptions 1366 method in this paper.)
experienced by customers over the remainder of a year. If factors Once the development of a transparent major event screening
such as these are not taken into account, it is not only difficult method was developed, the IEEE DRWG began ongoing col-
to assess a utility’s reliability performance over time, but likely lection of detailed daily utility-reported reliability information
to lead to flawed conclusions because the number and severity from members of the working group. This work was undertaken
of these events varies greatly from year to year. so engineers could assess their reliability performance and the
impact of various engineering practices on electricity reliability
as well as establish a credible screening process for “outlier
Manuscript received November 23, 2015; revised February 29, 2016 and
April 28, 2016; accepted June 24, 2016. Date of publication July 18, 2016; date days.” Now, almost 20 years later, the DRWG has amassed a
of current version February 16, 2017. The work described in this report was much larger historical record of information on utility reliability
funded by the National Electricity Delivery Division of the U.S. Department of performance than was available to the original developers of the
Energy’s Office of Electricity Delivery and Energy Reliability under Lawrence
Berkeley National Laboratory Contract No. DE-AC02-05CH11231. Paper no. Standard 1366 method. The DRWG has conducted analysis us-
TPWRS-01665-2015. ing this larger record of information and has found that, for some
J. H. Eto, K. H. LaCommare, and M. D. Sohn are with Lawrence Berke- utilities, application of the Standard 1366 method consistently
ley National Laboratory, Berkeley, CA 94720 USA (e-mail: jheto@lbl.gov;
kshamachi@lbl.gov; mdsohn@lbl.gov). leads to the identification of significantly more MEDs per year
H. C. Caswell is with PacifiCorp, Portland, OR 97232 USA (e-mail: than was expected [6]. This has led to discussion of possible
heide.caswell@pacificorp.com). alternatives or modifications to the Standard 1366 method.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. This paper develops two metrics that quantify the perfor-
Digital Object Identifier 10.1109/TPWRS.2016.2585978 mance of such alternatives compared to the performance of the
0885-8950 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
1328 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO. 2, MARCH 2017
Standard 1366 method. The metrics formalize two fundamental A major event day is a day in which the daily SAIDI exceeds a
objectives of the Standard 1366 method; namely, identification Major Event Day threshold value [called TMED]. For the purposes of
calculating daily system SAIDI, any interruption that spans multiple
of a reasonable number of MEDs per year and, through removal
calendar days is accrued to the day on which the interruption began.
of these MEDs, reduction in the year-to-year variability of the Statistically, days having a daily system SAIDI greater than TMED
System Average Interruption Duration Index (SAIDI) and the are days on which the energy delivery system experienced stresses
System Average Interruption Frequency Index (SAIFI). We use beyond that normally expected (such as during severe weather).
the metrics to evaluate the performance of alternative formula- Activities that occur on Major Event Days should be separately
tions of the Standard 1366 method, which have been suggested analyzed and reported.
by the DRWG and others. The analysis is conducted using a The method for identifying MEDs is based on calculating a
large set of utility reliability performance information collected threshold value for daily SAIDI. Daily values that exceed the
by the DRWG. threshold are removed from the calculation of annual SAIDI.
We organize this paper as follows: Annual SAIDI calculated in this way is commonly referred to
1) In Section II, we provide background on the Standard 1366 as “SAIDI without major events.”
method. The background emphasizes that the purpose of Standard 1366 requires the previous five years of daily SAIDI
the method is to provide a reliable basis for separating the to identify the threshold value. Equation (2) represents the
highly varying influences of major events from year-to- threshold value, which is known as TM ED , and is as follows:
year assessments of reliability performance.
2) In Section III, we describe the information from the IEEE TM ED = e(α +2.5β ) (2)
DRWG Benchmark study that serves as the basis for our where for each utility:
evaluation.
3) In Section IV, we describe alternatives to the basic for- α is the log-normal average of the previous five years of daily
mulation of the Standard 1366 method that have been SAIDI;
suggested by the DRWG and others. We then compare β is the log-normal standard deviation of the previous five years
the performance of these alternatives to the performance of daily SAIDI.
of the Standard 1366 method using two metrics, which
measure different aspects of performance. B. Origin and Objective of the Standard 1366 Method
4) In Section V, we summarize our findings and offer Investigations of reliability data available in the mid-1990s
thoughts on possible next steps. led industry experts to observe that daily SAIDI values, when
plotted from smallest to largest, followed a nearly log-normal
II. IEEE STANDARD 1366 METHOD FOR IDENTIFYING distribution [7]–[9]. They observed that this pattern held fairly
MAJOR EVENTS consistent until the very largest daily SAIDI values were plotted.
That is, these values were observed to be much larger than would
This paper focuses on sustained interruptions1 and the method
be expected if the distribution were log-normal. For the purpose
articulated in Standard 1366 to partition reporting using the con-
of assessing performance without major events, these very large
cept of the MED. The metric for assessing sustained interrup-
daily SAIDI values underscored the need for segmentation and
tions is called the SAIDI. It can be thought of as the total number
supported the use of log-normal as an appropriate distribution
of minutes a customer, on average, is without electric service
to use in determining threshold values.
over a specified time period. It is defined as follows:
These observations led to the formulation of the Standard
Σ Customer Interruption Durations 1366 method. The developers expected that, on average, the
SAIDI = (1)
Σ Customers Served method would lead to the identification of 2 to 5 MEDs per
SAIDI is generally reported on an annual basis. However, year. This, of course, was no more than an expectation regarding
it can be calculated over any time period. For example, the the average performance of the method over time. It was fully
annual SAIDI represents the number of minutes customers on understood that in any given year the number of MEDs could
average are without power over an entire year, while the daily and should fluctuate higher or lower than this expected average.
SAIDI represents the number of minutes customers on average See, for example, [8].
are without power on any given day within that year. Although Still, it was expected that the log-normal distribution would
annual SAIDI are more commonly reported by utilities, daily be generally appropriate for most utilities, independent of size
values of SAIDI form the basis for identifying major events or other utility-specific characteristics [1], particularly since that
using IEEE Standard 1366. was one of the goals of the method.
It is important to emphasize that the purpose of the Stan-
A. Definition of a Major Event Day dard 1366 method is not specifically to identify MEDs. In-
stead, the purpose is to establish a reasonable, transparent, and
IEEE Standard 1366-2012 defines MEDs as follows [5]:
easily reproducible means for segmenting a utility’s reliability
performance so that routine reliability events can be assessed
1 IEEE Std. 1366-2012 defines a momentary interruption as an interruption
separately from major reliability events, which occur only a few
of five minutes or less and a sustained interruption as any interruption that
is not a momentary interruption; effectively, a sustained interruption refers to days each year, yet have large impacts on annual SAIDI. Consis-
interruptions lasting longer than five minutes. tent segmentation enables independent assessment of trends in
ETO et al.: EVALUATING THE PERFORMANCE OF THE IEEE STANDARD 1366 METHOD FOR IDENTIFYING 1329
TABLE I
SUMMARY OF IMPACTS FROM MODIFICATIONS TO THE STANDARD 1366 METHOD
Description % Change in Std Dev of SAIDI w/o MEDs % Change in Average # MEDs % Change in Slope of MEDs
++ Denotes change greater than 10% for standard deviation of SAIDI without major events and average MEDs/year, and greater than 50% for slope of MEDs/year.
+ Denotes a change between 0-10% for standard deviation of SAIDI without major events and average MEDs/year, 0-50% for slope of MEDs/year.
- Denotes a change between −10-0% for standard deviation of SAIDI without major events and average MEDs/year, −50%-0% for slope of MEDs/year.
– Denotes a change less than −10% for standard deviation of SAIDI without major events and average MEDs/year, less than 50% for slope of MEDs/year.
The table highlights—for the utilities we examined—that With respect to the objective of reducing year-to-year
none of the alternatives performs uniformly “better” than the variability in SAIDI without major events, in our opinion, the
Standard 1366 across one or more of the performance at- alternatives that vary the number of historical years used to
tributes (metrics) we considered. That is, none of the alter- calculate TM ED show the greatest promise for further explo-
natives uniformly lowers the year-to-year variability in SAIDI ration. First, in comparison to the other alternatives considered,
without major events for all utilities. Instead, for any given this alternative led to much more modest percentage changes
alternative, while it may lower the value of this metric for in this performance metric. Second, this alternative is relatively
some utilities, it also increases it for other utilities (sometimes straightforward to implement.
dramatically). It should be noted that Standard 1366 does not discuss
The table also illustrates the trade-offs that exist among per- whether applicability of the method depends on specific utility
formance metrics. The example presented involving a much characteristics. It is conceivable that one or more characteris-
larger multiplier for β shows how decreasing the number of tics of a utility, such as size or geographic location, affect how
MEDs per year directly increases the year-to-year variabil- well the Standard 1366 controls for year-to-year variability in
ity in SAIDI without major events. Thus, trade-offs among reliability performance. These are factors we hope to explore in
performance metrics must also be considered. future research.
1332 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 32, NO. 2, MARCH 2017
V. CONCLUSION
IEEE Standard 1366 prescribes a method for removing the
effects of major events from annual reliability performance
metrics by segmenting out days of extreme reliability per-
formance in support of benchmarking analysis across utili-
ties of various sizes, environments and other parameters. The
purpose is to establish a reasonable, transparent, and easily
reproducible means for segmenting a utility’s reliability per- Fig. A-1. Examples of mixture model candidates.
formance so that routine reliability events can be assessed sepa-
rately from major reliability events, which occur only a handful APPENDIX
of days each year. These major events can have a dispropor-
tionate effect on the annual reliability performance of a utility. A mixture model represents a combination of multiple distri-
Removing their effects enables better evaluations of year-to-year butions to represent a single data set. As such, mixture models
changes in utility performance in response to routine reliability are well-suited for representing data that exhibit subpopulations.
events. A mixture model is mathematically represented by a distribu-
This paper was motivated by concerns that for a large num- tion f and is a mixture of K component normal distributions as
ber of utilities, the Standard 1366 method was not performing follows:
according to the expectations that were used to develop it. For K
some utilities, application of the method leads to the identifi- f (x) = λk fk (x) (3)
cation of significantly more major events in a year than was k =1
expected and does so consistently over many years. To try and
In (3), λ represents the mixing weights, such that lambda is
understand this issue, members of the IEEE DRWG began ex-
greater than zero and the sum of the weights across K com-
perimenting with alternative formulations of the Standard 1366
ponents equals 1. Although any number of components or
method in an effort to improve its performance. Initially, the
distributions can be used, for simplicity, we considered a two-
group focused on how alternatives affected the number of major
component mixture model. The model we ran uses probabilistic
events identified.
clustering and a weighted expectation maximum algorithm to
As an extension of the efforts made by the DRWG, this paper
determine the weights that best fit each component. One advan-
provides the Working Group and those using Standard 1366 with
tage to using this formulation is that if the data for a given utility
two distinct metrics that can be used to evaluate and compare the
are, in fact, best represented with a single distribution, the mix-
performance of the Standard 1366 method with any proposed
ture model would represent this by placing all the weighting on
alternative. The first metric assesses the number of major events
the first component; that is, λ[SUB1] = 1. The Standard 1366
identified in a given year over time. The second metric assesses
method is then a one-part mixture model.
the resulting year-to-year variability in SAIDI, once the effects
Fig. A-1 below shows two sample utilities in select years
of major events have been removed.
where the five-year daily log-transformed SAIDI shows a
This paper demonstrates how the metrics can be used to il-
skewed distribution with a higher frequency of larger daily
lustrate the trade-offs that arise when considering alternative
SAIDI values than smaller values. For data that are distributed
approaches to using the current Standard 1366 method. As an
like this, a two-part mixture model is able to better represent the
example, increasing the multiplier of β in the TM ED equation
distribution with the use of multiple components. Each compo-
results in the identification of fewer major events, however this
nent, or curve, represents a proportion of the overall distribution
change increases the year-to-year variability of the resulting
using the expectation maximum as the probabilistic technique
SAIDI without major events, which defeats the purpose of the
for representing the likelihood each data point falls within either
method. Hence, increasing the multiplier is not a useful alterna-
component. The result from the fit will be estimates of λ1 , λ2 ,
tive to the current Standard 1366 method.
and the mean and variance parameters for the two individual
In exploring various sets of alternatives to the Standard 1366,
distributions.
we conclude that none perform consistently “better” than the
In order to calculate TM ED using the two-part mixture model,
current method in reducing the year-to-year variability of the
we simply use the 2.5 beta logic in Standard 1366 to identify the
SAIDI without major events. This paper demonstrates quantita-
area under the two curves that represents a confidence interval
tively the “reasonableness” of the current Standard 1366 method
of 99.45% to the total area under their combined area.
compared to the alternatives that have been considered to date.
It is not the intent of this study to suggest that changes to the
Standard 1366 method are warranted. Instead, the purpose has ACKNOWLEDGMENT
been to propose and demonstrate, using a wide cross-section The authors acknowledge the membership of the IEEE Dis-
of information on utility reliability experiences collected over tribution Reliability Working Group both for allowing us to
many years, the use of metrics that can be used to assess the analyze information they have provided to the benchmarking
performance of the Standard 1366 method compared to any study and for their expert review of early presentations of the
alternative. findings presented in this paper.
ETO et al.: EVALUATING THE PERFORMANCE OF THE IEEE STANDARD 1366 METHOD FOR IDENTIFYING 1333
REFERENCES Kristina Hamachi LaCommare received the B.S. and M.S. degrees in atmo-
spheric science from the University of California, Davis, CA, USA. She is a
[1] J. H. Eto and K. H. LaCommare, “Tracking the reliability of the U.S. Program Manager in the Electricity Markets and Policy Group. On the research
Electric Power System: An assessment of publicly available information side, she analyzes efforts to track distribution electricity reliability and improve
reported to state public utility commissions,” Lawrence Berkeley Nat. Lab., the methods used to calculate and report reliability metrics and has also helped
Berkeley, CA, USA, Rep. LBNL-1092E, Oct. 2008. [Online]. Available: DOE support the preparation of the National Transmission Grid Study. Since
http://certs.lbl.gov/certs-rtina-pubs.html 1999, she has been working at the Lawrence Berkeley National Laboratory. She
[2] J. H. Eto and K. H. LaCommare. “A quantitative assessment of utility also manages various project, group finances, subcontracts, and budgets.
reporting practices for reporting electric power distribution events,” in Proc.
2012 IEEE Power & Energy Soc. General Meeting, San Diego, CA, USA,
Jul. 22–26, 2012.
[3] IEEE Trial-Use Guide for Electric Power Distribution Reliability Indices,
IEEE Std. 1366-1998, Dec. 1998.
[4] IEEE Guide For Electric Power Distribution Reliability Indices, IEEE Std.
1366-2003, May 2004.
[5] IEEE Guide for Electric Power Distribution Reliability Indices, IEEE Std. Michael D. Sohn received the B.S. and M.S. degrees in mechanical engineering
1366-2012, May 2012. from the University of California, Los Angeles, CA, USA, and also the M.S.
[6] I. Hoogendam, “Kolmogorov–Smirnov normality test for distribution reli- degree in engineering and public policy and the Ph.D. degree in civil and
ability data analysis,” in Proc. 2014 IEEE Joint Tech. Committee Meeting, environmental engineering from the Carnegie Mellon University, Pittsburgh,
New Orleans, LA, USA, Jan. 12–16, 2014. PA, USA. He is the Deputy Leader of the Sustainable Energy Systems Group
[7] J. Bouford, IEEE DRWG General Meeting personal communication, and the former Leader of the Airflow and Pollutant Transport Group (Indoor
Aug. 1, 2014. Environment Department). Since 1998, he has been with the Lawrence Berkeley
[8] R. D. Christie. “Statistical classification of major event days in distribution National Laboratory. He is a California-licensed Professional Engineer (Civil),
system reliability,” IEEE Trans. Power Del., vol. 18, no. 4, pp. 1336–1341, and has worked at an environmental engineering firm where he conducted
Oct. 2003. environmental health risk assessments.
[9] H. Caswell, V. Forte, J. Fraser, A. Pahwa, T. Short, M. Thatcher, and V.
Werner, “Weather normalization of reliability indices: IEEE Task Force on
Weather Normalization of Reliability Indices.” IEEE Trans. Power Del.,
vol. 26, no. 2, pp. 1273–1279, Apr. 2011.