Session 1E

The Human Element

Understanding and Learning from Operational and

Safety Events
 Wayne Hardie
Hydro Manager
Chattahoochee Hydro Group
Georgia Power Company
Life Lesson Video
 Jim Miller
Consultant
Signal Hydropower Consultants, LLC
 Operations and Maintenance
The Human Element: Understanding and Learning from Operational and Safety Events
Session 1E
Wednesday, June 27, 2018

OpEx – Information Sharing in the

Hydropower Industry

NHA OPERATIONAL EXCELLENCE PROGRAM

Jim Miller
Signal Hydropower Consultants, LLC
OpEx is:

• A voluntary Event Reporting (ER) system that receives, distributes, and catalogs
hydro operating experiences, best practices, and lessons learned in a secure environment.
• The only Event Reporting program and database available to the hydro industry
• A clearinghouse of all hydro related information – private, public, and government
resources, and technical information – hydroexcellence.org
• A tool for managing aging assets in order to raise the industry’s standard of performance
• Educational resource for a workforce in transition

NHA OPERATIONAL EXCELLENCE PROGRAM

A foundational goal of OpEx is to prevent a defining event in the hydropower
industry. The secondary benefits of this goal results in improved safety, strict
environmental performance, and reduced O&M costs and unplanned outages.

Orville Dam
February 2017

Sayano-Shushenskaya Hydro Plant

August 17, 2009 Failure

Watts Bar Hydro Plant Fire

September 27, 2002

South Fork Dam Catastrophic Failure

Johnstown, PA Flood
May 31, 1889
 NHA OPERATIONAL EXCELLENCE PROGRAM

O&M
NHA OPERATIONAL EXCELLENCE PROGRAM
Legislative
Affairs
Marine Hydraulic
Energy Power
Council Dam
Committee
Safety

NHA, with over

235 members, is a OpEx
forum to unite
industry with a
Pumped common and
powerful voice Regulatory
Storage
Affairs
Council
OpEx is part of NHA’s
Hydraulic Power
Committee
Water
Small Hydro
Innovation
Council
Council

8
 OpEx provides the vehicle for the

Generator hydropower industry to share

Operators information on events in a safe
environment where the lessons
learned can distributed to
prevent reoccurrences.

Currently, 103 companies,

Equipment Service
Suppliers Providers including 85 owner/operators
are participating in OpEx
OpEx’ Scope:
• Best Practices, Event Reports and Lessons Learned in the
areas of:
• Safety – Dam, Employee, and Public
• Environmental Performance
• Maintenance
• Operations
Actual and “near miss” events are included.
OpEx is not policy, legislative, or lobbying in nature
 Event Reports By Functional Area
8, 6%
n = 142 14, 10%

74% are
O&M
29, 20%
91, 64%

Maintenance Safety Operations Environmental

These Event Reports contain over 623 lessons learned and corrective actions

Severity: Significant (73), Severe (30), Minor (39)

Human Performance was a factor in 37% of the events.

11
 Recent Event Report Postings

• Loss of Station DC System and Plant Control

• Loss of Station Power
• Bypass Reach Minimum Flow Deviation
• Scroll Case Scaffolding Failure
• Investigate the clearance violation at Beta Dam
• Recordable Injury with Handheld Grinder

All of these events were identified as a Human Performance event.

During the lift of a 20 ton head gate, the lifting eye attaching the crane hook to the
gate failed. RCA determination, “…a large manufacturing flaw… due to high levels of
sulfur and phosphorous found within the sample.” Corrective Action, “replace
and/or test all lifting eyes made prior to 1950.”

“Upon receiving the ER we inspected our work

sites and tool rooms and found a number of
suspect lifting eyes and destroyed them. Who
knows what could have happened?”

Kirk Hudson, Managing Director, Generation & Transmission

Chelan County Public Utility District

13
Feedback from participants ……………
• Utilize lessons learned and recommendations in work
scope revisions for outages and capital projects
• Maintenance Work Orders – Revision or additions to
work tasks
• Safety related event reports have been utilized in plant
safety meetings and monthly staff meetings
• Adopt OpEx template for recording events
Benefits of Participation
• Support for managing aging assets & save limited financial resources

• Training for a workforce in transition

• Learn from, and collaborate with, fellow industry colleagues on:

• Best Practices
• Lessons Learned
• Event Reports

• Avoid similar events experienced by others – employee injuries,

forced outages, and others
 Ray Chow
Operational Excellence – Regulated Renewables
Duke Energy
 Operations and Maintenance
The Human Element: Understanding and Learning from Operational and Safety Events

Ray Chow
Regulated Renewables Operational Excellence
Overview
▪ Hydroelectric Generation in Duke Energy
▪ Focus on Operational Excellence
▪ OpEx Application in Duke Energy
▪ Recent Event
Duke Energy’s Hydro Stations

Facilities MW

Pump Storage 2 2,140

Conventional 31 1,373
Duke Energy’s Hydro Stations
• 33 Hydroelectric Facilities
• 56 Dams and Dikes
• More than 3,500 MW of Installed Capacity
• More than 25,000 square miles of Territory

7%

Natural Gas/Fuel Oil

18%
39%
Coal
Nuclear
Hydro & Solar
36%

• 173 Employees
Focus on Operational Excellence
The continued drive for event-free operations through increased organizational
capabilities, risk awareness, governance and oversight, accountability and employee
engagement.

• Event-Free Operations
• Optimized Reliability
• Sustainable Efficiency
Communication of Events
Duke
Industry
Hydro
OE
Event (OpEx)
Event

Investigation
Directly to Hydro Assess Fossil/Hydro
Personnel Applicability Screening Team
Corrective
Actions

Process/Procedure Fossil/Hydro
Lessons Learned Changes Personnel

Communication
Event-Free Performance
14
Recordable Environmental Event
12 Lockout Tagout Event
Recordable Safety Event
10

0
2015 2016 2017 2018
Equivalent Force Outage Rate
1.40%

1.20%

1.00%

0.80%

0.60% 1.27% 1.27%

0.89%
0.40%
0.61%
0.20%

0.00%
2015 2016 2017 2018
Learning from Events
On Tuesday, April 17, 2018, Units 1 and 2 tripped while
swapping the neutral grounds on Bank 1 at Fishing Creek Hydro
Station. There was an issue with the programmable logic
controller (PLC) on Unit 1 and needed to shut down the unit. At
the time, Unit 1 was motoring in standby and Unit 2 was
generating. Both of these units were connected to the same U1 U2

step-up transformer via Bank 1. The ground for the Bank was
established through Unit 1.
The sequence to shutdown Unit 1 was to re-establish the neutral
ground through Unit 2. Three technicians were involved. Two of
them were senior technicians and one fairly new to the
organization. After the prejob brief, the less seasoned technician
went to Unit 2 first to close the neutral ground switch. As soon
as he closed in the ground, both Units 1 and 2 immediately
tripped.

Takeaways?
 Mike Ruszkowski
Principal Project Manager
Electric Power Research Institute
Recognizing Human Performance
Traps in Plant Procedures

Mike Ruszkowski
EPRI Principal Program Manager
Overview
▪ Research in our industry and others has led to the
discovery of human performance traps within the very
document that should provide safe, reliable, error free
equipment operation: PROCEDURES
▪ Procedure writers without formal training in these human
performance traps, unintentionally lay a minefield of
problems for the procedure end user to navigate

Identify and develop defenses to human performance challenges by

asking, “Are there ……..?”
Different Shift, Different Person, Different Decision
▪ In-field decisions without clear guidance?
▪ Force the procedure user to make a decision with little
or no guidance for making that decision
▪ Bad Ex. - Extend the backwash until the effluent cleans
up
▪ Good Ex. - WHEN effluent sample indicates XXX,
THEN STOP the backwash
Where do I start and where do I end??
▪ Multiple actions in one step?
▪ Any step that exceeds one unrelated or two related
actions is considered a multiple action step
▪ Bad Ex. - Before the vendor connects to the loading
pipe, VERIFY the following: Truck wheels are chocked,
truck is grounded, hose is properly connected, cam lock
fittings are properly locked in position, hose has been
checked for holes, signage has been posted for
unloading, and that the hose is tied down to prevent
accidental disconnect
 Avoid an Operator Scavenger Hunt
• Actions found in any procedure section other than
Prerequisites, Instructions, or Attachment?
• Humans are lazy
• Don’t like reading lots of text
• Don’t like having to “ferret” out information
• Actions placed in other locations will be missed
Guesstimation is not a process
▪ Vague steps or steps missing critical detail
▪ Using non-specific or unfamiliar terms in the expression
of the action or instruction
▪ Abstract verbs like: Impact, Analyze, Determine,
Review, Ensure, Verify
▪ Abstract adjectives like: Slow, Fast, Sufficient,
Normal, All, Periodically, Routinely, Adequate, Slightly
▪ Abstract phrases like: When Directed, As Soon As
Management …., Should
▪ Bad Ex. - LOWER tank level to approximately 20%.
▪ Good Ex. - LOWER tank level to 15% - 25%.
 Complex Math leads to Simple Problems
• Complex calculations without verification?

o What is complex? (Fatigue, mental state, distractions = all lower

math abilities)

o Requiring procedure user to perform mental math.

o Bad Ex. - LOWER tank level to 20% + 5%.

o Good Ex. - LOWER tank level to 15% - 25%.

Under Pressure……and Not In-Line
▪ Time constraints?
▪ Avoid any words that could unnecessarily cause
perceived time pressure.
▪ As soon as possible…
▪ When time permits….

▪ Inconsistent format, layout, and writing style?

▪ Differing formats, layouts, or writing styles used for same
site/utility procedures. (uncomfortable, people dislike
changing mental models)
▪ Cross your arms the “other way”
Path of Least Resistance
• Excessive physical challenges?
o When the selected components or sequence of
steps may not be the most convenient or
practical for the user.
o Climbing up to vertical platform, operating a
valve, climbing down to operate another valve,
and then immediately climbing back up platform
to operate another valve when the order did not
matter.
 Principles of Human Performance
• People make mistakes
1

• Individual behavior is influenced by organizational processes and values

2

• Mistake-likely situations are predictable and manageable

3

• Accidents and Operational Incidents can be avoided by understanding mistakes and learning from
4 them

• Leadership response to failure matters (How you react matters!)

5
 Principles of Human Performance
• Accidents and Operational
Incidents can be avoided by
understanding mistakes and
4 learning from them
The purpose of an Investigation
should be to “prevent
recurrence, not to find people to
blame”
Investigations should really
be titled…”Learning
Events!”
 Southern Company Incident Information Sharing

• Life Lesson Videos

• Internal Operational Excellence Reports
• Industry Operational Excellence Reports
• Event Learning Reports
Discussion/Questions