W.

Earl Carnes
Earl. Carnes@hq. doe. gov
How we think about error is in error:
What social science tells us about managing complex
sociotechnical organizations
Berkeley Lab
April 5, 2010
Lawrences big idea:
Scientific research
is best done through
teams of individuals
with different fields
of expertise,
working together.
Rad Lab staff 1939
(Knowledge is a collaborative construction none of us alone is sufficient)
"Organization is what you do before
you do something, so that when you do
it, its not all mixed up."
A.A. Milne (AKA Winnie the Pooh)
Most people don't recognize opportunity when
it comes, because it's usually dressed in
overalls and looks a lot like work.
Thomas Edison
If you could design the perfect
organization, what would it be like & how
would you start?
human fallibility is like gravity, weather, and
terrain, just another foreseeable hazard. Error
is pervasive
What is not pervasive are well-developed skills
to detect and contain these errors at their
early stages.
We begin with discussing error; to change
the paradigm of blame ----
Weick & Sutcliffe
Managing the Unexpected
Error, mistake, faux pas, gaffe, blunder, lapse, slip, goof,
oops, blooper - how many phrases do we have to express
the idea that things dont always happen as we expect or
as we would prefer?
Definition of Human error (Google, March 29, 2010 - 629,000 hits)
No events = no human error problems.
Training will solve human problems.
Accountability is all that is needed.
Significance determines culpability.
Experience means error-free
performance.
Errors are the cause of accidents.
Errors are bad.
Human Error Myths
Search for a new paradigm
Thomas Kuhn
Berkeley
The success of a paradigm is at the
start largely a promise of success
discoverable in selected and still
incomplete examples."
Mistakes are at the very
base of human thought,
embedded there, feeding the
structure like root nodules...
We are built to make
mistakes, coded for error...
The lower animals do not have
this splendid freedom. They
are limited, most of them, to
absolute infallibility.
Dr. Lewis Thomas The Medusa and the Snail (P.29)
There is no such thing
as a dysfunctional
organization, because
every organization is
perfectly aligned to
achieve the results it
gets.
The Practice of Adaptive Leadership: Ronald Heifetz,
Alexander Grashow and Marty Linsky
Starting with first principles:
We would be eternally miserable if our
errors worried us too much because as we
push forward we will make plenty more.
Dr. E. O. Lawrence, 1934
http://www.aip.org/history/lawrence/bigscience.htm
Human error is a consequence
not a cause. Errors are shaped
by upstream workplace and
organizational factors Only by
understanding the context of
the error can we hope to limit
its reoccurrence.
Dr. J ames Reason,
University of
Manchester
People do not operate in a vacuum,
where they can decide and act all-
powerfully. To err or not to err
is not a choice. Instead, peoples
work is subject to and constrained
by multiple factors.
Dr. Sidney Dekker
(Professor, Lund University & Pilot)
It is the expertise of the human operator
that makes it possible to adapt the
performance of the joint system, in real
time, to unexpected events and
disturbances. Every working day, across the
whole spectrum of human enterprise, a large
number of near-misses are prevented from
turning into accidents only because human
operators intervene. The system should
therefore be designed so that human
adaptation is enhanced. This will ensure that
the joint system has the requisite variety to
maintain control in an unpredictable
environment.
Dr. Erik Hollnagel, Expertise and Technology: Cognition &
Human-Computer Cooperation.1995
Sense #1 Error as the cause of failure: This
event was due to human error.
Sense #2 Error as the failure itself, i.e. the
consequences that flow from an event.
Sense #3 Error as a process, or more precisely,
a departure from the "good" process.
So what do we mean by error?
David D. Woods & Richard I. Cook, Mistaking Error, Patient Safety Handbook,
Jones and Bartlett, 2003
the New View
a shift
from person to system view of error
Person Centred View
Focus on the individual, excluding other factors
Individual responsibility and blame
- careless, at fault, bad
Solution: change behaviour / remove the
individual
System View
Focus on factors that influence errors
Human beings are fallible, errors to be expected
Solution: change system / conditions of work
Emerging Paradigm -
From error to resilience:
Traditional perspective
Things go right because
Systems are well
designed and
scrupulously maintained
Designers can foresee
any contingency
Procedures are complete
and correct
People behave as they
are expected to
Performance variability
must be eliminated
A new perspective
Things go right because PEOPLE
Learn to overcome design flaws
and functional glitches
Adjust their performance to
meet demands
Interpret and apply procedures
to match conditions
Detect and correct errors
Increasing complexity has made
modern technological systems
underspecified
PEOPLE are therefore key to
the proper functioning of
modern technology
Error may be an adaptive response, an
oversimplification of history that
primes us for complex futures and
allows us to project simple models of
past lessons onto those futures, lest
history repeat itself.
Sidney Dekker, Ten Questions about Human Error
Error as evolutionary cognitive
adaptation?
Once the nature of error is understood
and its occurrences anticipated, it is
then possible to devise protective
measures it is not the error that
should concern us; its the
consequences.
The utility of error:
Protective Measures
Examples
Error analysis & trending,
Mental models,
Precursors,
Defenses analysis
Material:
improper materials (impure sample)
improper procedure (wrong experimental protocol,
poor technical skill)
phenomenon influenced by observer
two different phenomena conflated due to lack of
experimental distinction
Observational:
insufficient controls (causes or effects misplaced)
incomplete understanding of instrument or method
perceptual bias (no double-blind study)
small, unrepresentative sample
Error analysis and trending (examples)
Conceptual:
reasoning error (computational, logical fallacy)
mistaken assumptions or background information
overgeneralization (unjustified scope of explanation)
lack of alternative explanations
Social:
communication failures (obscure publication, translation
hurdles)
faulty peer review; other mistaken judgments of
credibility
sociocultural cognitive biases (gender, ethnicity,
economic class, etc.)
poor science education, poor science journalism
Douglas Allchin Error and the Nature of Science 2004, American Institute of Biological Sciences
http://www.actionbioscience.org/education/allchin2.html
26
Practice of Preventing Events
Reduce Errors Frequency
Vision,
Beliefs,
&
Values
Identify & Fix
Latent
Organizational
Weaknesses
Mc
Human
Errors
Re
Mindfulness to Minimize
Initiating Action
No
Event
Work-as-done
Work-as imagined
W
Vision,
Beliefs, &
Values
1
2
3
4
Establish, maintain and question defenses
Error Precursors -
promoting mindfulness
Limited short-term memory Personality conflicts
Mental shortcuts (biases) Lack of alternative indication
Inaccurate risk perception (Pollyanna) Unexpected equipment conditions
Mindset (tuned to see) Hidden system response
Complacency / Overconfidence Workarounds / OOS instruments
Assumptions (inaccurate mental picture) Confusing displays or controls
Habit patterns Changes / Departures from routine
Stress (limits attention) Distractions / Interruptions
Illness / Fatigue Lack of or unclear standards
Hazardous attitude for critical task Unclear goals, roles, & responsibilities
Indistinct problem-solving skills Interpretation requirements
Lack of proficiency / Inexperience Irrecoverable acts
Imprecise communication habits Repetitive actions, monotonous
New technique not used before Simultaneous, multiple tasks
Lack of knowledge (mental model) High Workload (memory requirements)
Unfamiliarity w/ task / First time Time pressure (in a hurry)
Task Demands Individual Capabilities
Work Environment Human Nature
Equipment Labeling
Procedure /
Work Package
Worker Knowledge,
Skill, & Proficiency
FFD
Uneasy Attitude
Equipment Ergonomics
&. Human Factors
Tools
Roles &
Responsibilities
Housekeeping
Environmental
Conditions
FME Conditions
Lockout / Tagout
Walkdowns
Performance
Feedback
Task Assignment
HP Surveys
QC Hold Points
Independent
Verification
Interlocks
Personal
Protective
Equipment
Alarms
Forcing
Functions
SITE-LEVEL
RESULTS
WORK SITE
CONDITIONS
Post Job
Critiques
Condition
Reports
Root Cause
Anal ysis
Performance
Indicators
Maximize Control
effectiveness
Pre-job
Briefing
Just-in-time
Operating
Experience
SAFE
Dialogues
Turnover
Functions of Defenses:
Create awareness
Detect and warn
Protect
Recover
Contain
Enable escape
ORGANIZATION
PROCESSES
& VALUES
TASK
BEHAVIOR
Safeguards
Equipment
Reactor
Protection
Systems
Containment
Self checking Place-keeping
3-way Communication
Double
Verification
Procedure Use
& Adherence
Supervision
& Coaching
Mgmt. Monitoring
Stop When
Uncertain
Vital Parameters
Problem-solving
Methodology
Conservative
Decision-making
Team Skills
Peer checking
Meetings
Communication
Practices / Plan
Reviews &
Approvals
Change
Mgmt.
Problem
Solving
Scheduling /
Sequencing
Clear
Expectations
Self-Assessment
Rewards &
Reinforcement
Relationships
Trend Anal ysis
Role
Models
OE
Safety
Philosophy
Task
Allocation
JIT Training
Handoffs
High
Standards
Accountability
Simple/effective
Process Philosophy
Purpose: Encourage members of an organization
to report errors, near misses, technical, and
organizational problems to improve future
performance.
Just Culture defined A culture in which
reckless disregard for self or others is not
tolerated by the community and trust that
errors will not be met with blame.
Put a good person and a bad system and the system wins every time
Dr. Gerry Rummler
A Just Culture:
Nuclear Energy Performance Trends
1985-2008
Rx Trips/
Scrams
Cost (/kwh)
Significant
Events/Unit
Capacity Factor
(% up)
We are trying to manage
20
th
century organizations
using 19
th
century thinking
Gary Hamel
U of Michigan Ross School
Core Competency
See
http://blogs.wsj.com/management/
Successful organizations are
not error free but error resilient!
The signature of highly reliable organizations is
not that they are error-free, but that errors
dont disable them. Resilience is a combination of
keeping errors small and of improvising
workarounds that keep the system functioning.
These organizations develop capabilities to
detect, contain, and bounce back from those
inevitable errors that are part of an
indeterminate world.
Deming The New Economics for Industry, Government, Education (1993)
Dr. W. Edwards Deming
Theories are necessary to
learn and improve
Experience by itself teaches nothing ...
Without theory, experience has no meaning.
Without theory, one has no questions to ask.
Hence without theory there is no learning.
Mental Models
Perceptual Skills
Sense of typicality
Routines (local rules)
Declarative Knowledge
Cognitive systems distinguish
experts from novices
Dr. Gary Klein
We put too much emphasis on reducing errors
and not enough on building expertise.
Pedantry and mastery are opposite
attitudes toward rules. To apply a rule to
the letter, rigidly, unquestioningly, in cases
where it fits and in cases where it does
not fit, is pedantry... To apply a rule with
natural ease, with judgment, noticing the
cases where it fits, and without ever
letting the words of the rule obscure the
purpose of the action or the opportunities
of the situation, is mastery.
A deep knowledge of rules
and their application is a
hallmark of the expert
Dr. George Polya,
Stanford,
professor of
mathematics
How to Solve It
(1945) p. 148
Problems are interconnected,
environments are turbulent, and
the future is indeterminate in so
far as managers can shape it by
their actions. What is called for
are the analytic techniques of
operations research and the active,
synthetic skill of designing a
desirable future and inventing ways
of bringing it about.
Dr. Russell Ackoff,
Pioneer Operations
Research, Wharton
School
New management must
be interdisciplinary
"There is a beauty in discovery. There is mathematics
in music, a kinship of science and poetry in the
description of nature, and exquisite form in a
molecule. Attempts to place different disciplines in
different camps are revealed as artificial in the face
of the unity of knowledge. All literate men are
sustained by the philosopher, the historian, the
political analyst, the economist, the scientist, the
poet, the artisan and the musician.
Dr. Glenn Seaborg, On being appointed UC Berkeley chancellor 1 958
Seek unity of knowledge
(socio & technical)
There must be no barriers to
freedom of inquiry. There is no
place for dogma in science. The
scientistmust be free to ask
any question, to doubt any
assertion, to seek for any
evidence, to correct any
errors the only way to avoid
error is to detect it and the
only way to detect it is to be
free to inquire.
The culture must
support free inquiry
Dr. Robert Oppenheimer
Barnett, L., "J. Robert Oppenheimer," Life, Vol. 7, No. 9, International Edition, October 24, 1949,
pp.52-59, p.58
The desire, and the ability,
of an organization to
continuously learn from any
source, anywhere; and to
rapidly convert this learning
into action is its ultimate
competitive advantage.
Organizations ability
to learn will determine
their future survival
Jack Welch,
former CEO of General Electric
A learning organization is skilled at
creating, acquiring, and transferring
knowledge, & modifying its behavior to
reflect new knowledge. . .
Dr. D.A. Garvin
Harvard
Business School
Garvin, D. A., Edmondson, A. C., Gino, F. (2008), Is Yours a Learning Organization?, Harvard Business
Review, March 2008
Organizational learning and adaptability
depend on:
supportive learning environment
concrete learning processes and practices
leadership behavior that provides
reinforcement
Do we pursue
knowledge the social
sciences of
organizations as
rigorously as we
pursue knowledge of
physical science and
technology?
Are we managing for the
past or the future?
Neither the sausage maker nor the chemical plant
manager is immune from errors that can have far -
reaching consequences. The three major
recommendations we offer are that managers should
aggressively seek to know what they dont know, design,
reward and incentivize systems to recognize the cost
of failure and the benefits of reliability, and
communicate the big picture to everyone.
Roberts & Bea, Must Accidents Happen?, Academy of Management
Executive, 2001
Dr. Robert Bea Dr. Karlene Roberts
1. Theory derived from research in
analogous environments.
2. Applied methods & techniques
(individuals, teams, organization)
3. Reflective learning
New paradigm:
A framework for mindful,
adaptive practice
Whats the goal?
Using high reliability research and practice as a
reflective framework for the pursuit of excellence
Indentifying demonstrated strategies, methods and
techniques of implementation
Designing experiments to understand how people
adaptively create safety and performance in DOE
systems
Contributing to the advancement of sociotechnical
theory and practice to shape innovative organizations
of tomorrow
Organizations designed to create the
science and technology of the future
http://hsshpi.wordpress.com/
(new WIKI soon to come!)
Introduction
February 26, 2010
Welcome to our new blog on high reliability. In the mid 1990s the U.S. Department of
Energy began a journey toward high reliabilityfor managing our safety critical science and
technology missions. Over some 15 years our internal deliberations have benefited from
external perspectives, gradually engaging other federal agencies, private industry and the
academic community in our conversations. This new blog is a next step toward broader
discussion within our DOE community and with others who are likewise involved in
seeking to assure high levels of performance and safety for their safety critical
organizations. We welcome your questions, contributions, and comments.
Read the rest of this entry
Why does Reliability matter?
(Two reasons:)
Reason 1 - Designing the Future
In Memory
Laboratory Assistant, UCLA
Died in laboratory fire, January 2009
Reason 2 - Sheri Sangji
(and all the other Sheris)
It is easier to perceive error
than to find truth, for the former
lies on the surface and is easily
seen, while the latter lies in the
depth, where few are willing to
search for it.
Johann Wolfgang Von Goethe
Congratulations JGI!
2010 JGIs second ERGO Cup award
W. Earl Carnes
Senior Advisor, High Reliability & INPO
Liaison
301-903-5255
earl.carnes@hq.doe.gov
HS-31, GTN Rm C-156
U.S. Department of Energy
1000 Independence Ave, SW
Washington, DC 20585-0270