Stat 435/835

Statistical Methods for

Process Improvement
Course Overview
Stefan Steiner, shsteine@uwaterloo.ca

Background

Numerical and graphical data summaries (Stat

231)
Linear regression (Stat 231 [+331])
Design of experiments (Stat 332 [+430])
Analysis of Variance (Stat 332)

Practice appropriate application

Develop statistical thought process

Course Overview

Capstone statistics course

No new statistical methods introduced
But, we use what you have previously learnt

Course Learning Objectives

think strategically about how to achieve cost-effective

variation reduction
reduce variation by following a step-by-step algorithm
learn how to use appropriate statistical plans and tools
to achieve the goal of Statistical Engineering

Course Overview

Learn about the Statistical Engineering

algorithm, strategies and approaches for solving
chronic excess variation problems

Understand sources of variation and their role in

process improvement
3

Learn how to better use empirical methods; that

is, learn effective and efficient ways to plan,
execute and analyze the results of a process
investigation
Apply the methodology to Watfactory, a virtual
manufacturing process, to aid in learning
Tell me, I will forget
Show me, I may remember
Involve me and I will understand

Course Overview

Course Learning Objectives (cont.)

 This course covers the material in textbook

Statistical Engineering: An algorithm for
reducing variation in manufacturing
processes published by Quality Press 2005
You are expected to read the textbox on your
own
Download electronic version and/or
Borrow text for the term from me

Course Overview

Textbook

The course (though not the textbook) is

designed around a virtual process called
Watfactory
Watfactory is a web based virtual process to
produce camshafts
demo later

Course Overview

Watfactory

Watfactory website
www.student.math.uwaterloo.ca/~watfacto/login.htm

Watfactory Project

nine weekly written progress reports

two class presentation on your progress
two management reviews of another team

See course outline and the report and

presentations guidelines for more
information

Course Overview

You will be divided into teams and assigned

different versions of Watfactory to improve
Watfactory projects involve:

Videos

A suggested schedule is given in the written

course outline

The videos cover all the material from the

textbook as well as the Watfactory virtual
process

Course Overview

You are also expected to watch the series

of the videos on your own time

 General purpose statistical software

Most commonly used package in the quality
improvement area
Very easy to use
data window looks like a spreadsheet
pull down menus to access numerical analysis and
graphs
better than Excel for statistics/graphics

Used throughout these course notes and in

the corresponding book

(Book Chapters in Brackets)

Introduction Overview and Goals (1-3, 5)

Watfactory
The Statistical Engineering Algorithm (4)
Problem Selection and Definition (6)
Measurement System Analysis (7)
Choosing a Variation Reduction Approach (3, 8)
Finding a Dominant Cause using the Method of Elimination and
Families of Variation (9)
Tools for Finding a Dominant Cause (10-12)
Verification of the Dominant Cause (13)
Revisiting the Choice of Variation Reduction Approach (14)
Determining the Feasibility of an Approach (15-20)
Implementation and Holding the Gains (21)
Wrap Up and Conclusions

Course Overview

Course Topics

Course Overview

MINITAB Statistical Software

10

Introduction
Problems are only opportunities in work clothes
Henry J. Kaiser

Variation Definition
deviation of output from target
changing value of output from part to part

V6 piston diameters
target diameter = 101.591 mm
measured diameters for 3 consecutive pistons:
101.593, 101.589, 101.597

Chap. 1: Introduction

Variation is both

Excess output variation leads to

Poor performance
Scrap and/or rework
Low customer satisfaction
Extra costs

Chap. 1: Introduction

Consequences of Variation

sProcess improvement
3

Reducing Variation
Better centering to target
Reducing variation among the parts

Reducing variation among parts is usually

harder than moving the process center

Chap. 1: Introduction

We can improve the process by

 Pull is a critical alignment characteristic

Target pull: 0.23 Newton-meters
Almost all trucks in last 2 months were
within specs -0.12 to 0.58 Nm
Goal: reduce pull variation about the target

Chap. 1: Introduction

Truck Pull

Cast iron engine blocks were tested for

leaks
Current scrap rate was 2-3%
Goal: reduce leak rate to less than 1%

Chap. 1: Introduction

Engine Block Leaks

 Camshaft lobe geometry is critical

Base circle run-out is a positive measure
of the max. deviation from an ideal circle
Goal: reduce average run-out
Issue: physical lower limit of zero

Chap. 1: Introduction

Camshaft Lobe Runout

Breakage of sand cores occurred in handling

Goal: increase average core strength
Issue: cores that were too strong led to
casting defects

Chap. 1: Introduction

Sand Core Strength

 Excessive main diameter variation

Histogram suggests process off target
Goal: move average diameter to target
Issue: asymmetric costs
7
6

4
3
2
1

0
-5

1front

Paint Film Build

Vehicle paint appearance is critical
Film build lower specification is 15 thou
Goal: reduce film build variation
Issue: reducing variation would allow decrease
in average film build and cost savings

Chap. 1: Introduction

Percent

Chap. 1: Introduction

Crankshaft Main Diameter

10

 Customer complaints about frost

build up in frost free fridges
Goal: eliminate frost build up
Issues:
difficult to measure frost except
during customer usage
causes found to be in usage
environment

Chap. 1: Introduction

Refrigerator Frost Build

11

Describing Processes
If I had to reduce my message to management to just a few
words, Id say it all had to do with reducing variation
W. Edward Deming, 1900-1993

Process

Manufacturing processes
e.g. production of automobiles or automobile parts

Service processes
e.g. credit applications, customer returns, Math
faculty admissions

Chap. 2: Describing
Processes

A series of actions which are carried out

in order to achieve a particular result.
(Collins Dictionary )

Measurement processes
e.g. gauges, operators, etc. produce measurements

Process Map
Melting

Pouring

Shakeout

Machining

Molding

Each time the (e.g. exhaust manifold)

process operates it creates a
unit/part/realization

Chap. 2: Describing
Processes

Core Making

Process Outputs and Inputs

characteristics related to performance or ease
of assembly, e.g. strength of casting,
dimensions, etc.

Inputs: features of the process itself

e.g. operators, pouring temperature,
properties of the sand, etc.

Chap. 2: Describing
Processes

Outputs: characteristics of the realizations

of interest to the customers

Inputs and Outputs can be

continuous, binary, ordinal, etc.

Critical to Quality (CTQ)

e.g. piston head diameter, credit application
decision

Often we can make the process better if

we reduce variation in the CTQ(s).
CTQs typically have a target value and
specification limits
e.g. 595 5 microns from nominal for piston
head diameter

Chap. 2: Describing
Processes

Every manufactured product has 1+ critical

to quality (CTQ) output characteristics

Output Distribution

graphically by histogram
numerically by the center (average), standard
deviation, min, max, etc.

A histogram shows the distribution of the

output values, the bar heights give the
relative frequency for each range of
output values

Chap. 2: Describing
Processes

We are interested in the distribution of

output values from the process
We can summarize the output distribution

Describing Variation

Chap. 2: Describing
Processes

Truck alignment (pull): target 0.23, specs 0.12 to 0.58, well centered good process

Types of Problems
Excessive variation

Poor targeting
30

20

10

20

10

0
-3

-2

-1

deviation
Combination

deviation

40

Frequency

30

Chap. 2: Describing
Processes

Frequency

Frequency

30

Defect rate too high

20

10

8
0
0

out-of-round

Types of Problems
Chronic versus sporadic problems

Problems with a continuous output

characteristic e.g. time, length, etc.
excessive variation (high scrap and/or rework)
poor targeting of the process center

Problems with a binary output characteristic,

e.g. pass/fail, defective/not defective
defect rate too high

Chap. 2: Describing
Processes

chronic problems are persistent and resist solution

sporadic problems are urgent and short-lived
(firefighting)

Measure of Variation (StDev)

We quantify variability (across units) as

y  y
n

i 1

n 1

where yi is output for ith part i 1,2,..., n and

y is the average
Stdev is expressed in the same measurement
units as the process output
For bell shaped histograms almost all values
will fall within average r 3stdev

Chap. 2: Describing
Processes

stdev

10

Fixed and Varying Process Inputs

control plan
iron pouring temperature target value
process or product design changes

A varying input naturally changes from part to

part or time to time, e.g.

core dimensions change from casting to casting

operators change each shift
raw material characteristics change each batch
environmental conditions

Chap. 2: Describing
Processes

A fixed input changes only when we deliberately

change it, e.g.

11

 Variation in the output(s) as the process

runs must have a cause!
Only varying (not fixed) inputs can be
causes of this output variation
Some causes have a large (or dominant)
effect others have little or no effect
Denote output (y), fixed inputs (z) and
varying inputs (x), then we might model
f  x1 , x2 ,..., z1 , z2 ,...

Chap. 2: Describing
Processes

Causes (of Variation)

12

What is a Cause?
Scatterplot of output vs cause

cause

cause

Can the product design (process design) be

a large cause of output variation?

Chap. 2: Describing
Processes

output

output

Scatterplot of output vs cause

13

Dominant Cause of Variation

We shall assume (to start) that for every
problem, there is a SINGLE DOMINANT CAUSE
Secondary causes can be identified, but the
tools and strategies used in the search for a
cause work best if there is a single dominant
cause
The assumption is more likely to hold with a
focused problem
e.g. one with a single failure mode

Chap. 2: Describing
Processes

Pareto Principle applied to causes see the next page

14

Pareto Principle
First proposed by Vilfredo Pareto in 1906

Since then principle has been shown to be

widely applicable
Here we apply it to causes of variation
Most of the output variation can be explained
by just one or a few causes (varying inputs)
Vital few, trivial many

Chap. 2: Describing
Processes

80% of Italian land owned by 20% of the people

80/20 rule

15

 Suppose Y f  x   R
where x represents a single cause
Then, assuming independence between
the cause x and all other causes, we have
stdev Y 

stdev  due to x   V R2
2

Chap. 2: Describing
Processes

Model for Single Cause

16

Effect of Square Root Sum of

Squares Formula
sd (due to cause) 2  sd  due to all other causes 

100
90

Chap. 2: Describing
Processes

Percent reduction in overall variation

sd (output)

80
70
60
50
40
30
20
10
0

17
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
standard deviation(due to cause) / standard deviation(total)

Chap. 2: Describing
Processes

Dominant Cause Continuous

Output

Continuous cause

Discrete cause
18

Dominant Cause with

Binary Output (Ggood, Bbad)
GG

G G

B B

G
B

Input 2

G
B

G
G

Chap. 2: Describing
Processes

Input value

B
B

Input 1

19

 There is an interaction between 2+ inputs if the

effect on the output of changing either input
depends on the level of the other input
Interaction is not to be confused with correlation
between two inputs
a correlation exists between two inputs if they
vary together in some way, e.g. when input1 is
low, input2 also tends to be low
note two inputs can be correlated whether they
have an effect on the output or not

Chap. 2: Describing
Processes

Interaction and Correlation

20

Cause and Output Relationship

We assume that reducing the variation in a

dominant cause will reduce variation in the output
However, correlation does not guarantee this!
Verify assumption later

Chap. 2: Describing
Processes

In the search for a dominant cause we look for a

strong correlation between a varying input and
the output, such as

21

To reduce variation Juran suggests two steps

Diagnostic journey find the cause(s) of
the variation
Focus on varying inputs (xs)

Remedial journey find a solution

To improve we must change something
Focus on fixed inputs (zs)

Chap. 2: Describing
Processes

Variation Reduction Steps

22

To change the long term output variation

(i.e. solve the problem) we will need to
change one or more fixed inputs!
Change to a fixed input might help if it
reduces the variation in the dominant cause
changes the relationship between a dominant
cause and the output
moves the process output center

Chap. 2: Describing
Processes

Solutions

23

Seven Variation
Reduction Approaches
A fool can learn from his own experiences;
the wise learn from the experience of others
Democritus, 460-370 B.C.

Fix the Obvious Based on Knowledge of a

Dominant Cause
Desensitize the Process to Variation in a
Dominant Cause
Feedforward Control Based on a Dominant Cause
Feedback Control
Make Process Robust to Noise
100% Inspection
Change the Process Center

Chap. 3: Variation
Reduction Approaches

The Seven Variation Reduction

Approaches

Fix the Obvious Based on

Dominant Cause
Reduce variation in the dominant cause
Improved Process

output

output

Chap. 3: Variation
Reduction Approaches

Existing Process

input

input

Truck Pull
In the early phases of improving the truck
alignment process, the team looked at right caster
stratified by alignment gauge

4.3

3.8
15

20

25

day

30

As the trucks enter the gauges haphazardly the

dominant cause is the gauges
An obvious solution was to recalibrate the gauges
(and monitor them over time)

Chap. 3: Variation
Reduction Approaches

avg right caster

4.8

 Customers complained of wheel trim and hubcap

damage
A dominant cause of the broken retaining legs was
found to be a combination of cold weather and
contact with curbs.
An obvious solution was to replace the inherently
brittle existing ABS hubcap with a new design
made of mineral reinforced polypropylene

Chap. 3: Variation
Reduction Approaches

Hubcap Damage

Desensitization

Improved Process

output

output

Existing Process

Chap. 3: Variation
Reduction Approaches

Desensitize a process to variation in a dominant

cause

6
input

input

 Problem: cast iron engine block subsurface porosity

Dominant cause: iron pouring temperature. Low
temp. occurred during (un)planned stoppages
Using an experiment the team explored the effect of
a new core wash
400

porosity

300

200

100

0
regular

alternate

Chap. 3: Variation
Reduction Approaches

Engine Block Porosity

wash

Solution: change the block core wash to reduce the

effect of the iron temperature variation

output

output

Monitor the dominant cause and predict the

future behavior of the output
If the prediction is far enough from the target,
adjust the process
Existing Process
Improved Process

Chap. 3: Variation
Reduction Approaches

Feedforward Control

8
input

input

 Pull is an important characteristic as it indicates

how well a truck will track on a standard highway
Variation in truck frame geometry is a dom. cause
of variation in the key alignment characteristic left
caster that affects pull

Solution: Adjustment for each alignment assembly

measure geometry from bar coded label on each frame
predict left caster and pull using a predictive equation
drill cam to adjust predicted pull closer to target

Chap. 3: Variation
Reduction Approaches

Truck Alignment (Pull)

Monitor the output characteristic and predict

future behavior from current and past observations
If the prediction is far enough from the target, make
an adjustment to the process
Improved Process

Target

output

output

Existing Process

Chap. 3: Variation
Reduction Approaches

Feedback Control

Target

10
time

time

V6 Piston Diameter

Solution: Informal feedback controller (one on each

stream)
Every 15 minutes select and measure two pistons
If their average is outside the range 2.7 to 10.7 (target is
6.7 microns) adjust the process center to compensate

Chap. 3: Variation
Reduction Approaches

Excess piston diameter variation was a problem

Stratifying the process by streams found structural
variation in the diameters

11

Make the Process Robust

output

Chap. 3: Variation
Reduction Approaches

Change fixed inputs to reduce the effects of

unidentified causes.

improved process

original process

control input settings

12

Door paint thickness variation was a problem

Dominate variability acted vehicle-to-vehicle
An investigation to find the cause failed
An investigation to search for more robust settings
was conducted
An experiment involving five fixed inputs was conducted
Each experimental run consisted of painting five
consecutive cars
Performance measure was the log standard deviation of
thickness over the five cars

Chap. 3: Variation
Reduction Approaches

Paint Thickness

Solution: Change the process settings

high Zone X voltage, high conductivity, low temperature

13

100% Inspection

10
9
8

Percent

Lower
Inspection
Limit

Upper
Inspection
Limit

6
5
4
3

Chap. 3: Variation
Reduction Approaches

Reduce the variability by identifying and then

scraping or reworking all parts that have values of
the output beyond selected inspection limits

2
1

14

output

 Blocked exhaust manifold ports are very rare, but

result in catastrophic failure of the engine
A blocked port is relatively difficult to detect since
it is not visible
Search for a cause is difficult because blocked
ports are so rare
Ten year search was fruitless
Automatic 100% inspection of all manifolds using
ultrasound was expensive, but outweighed the
potential cost of a blocked port reaching the
customer

Chap. 3: Variation
Reduction Approaches

Blocked Exhaust Manifolds

15

Change the Process Centre

Adjust process center to move it closer to the target
Improved Process
Process Target

Percent

Percent

Process Target

output

output

Chap. 3: Variation
Reduction Approaches

Existing Process

16

 Leaky battery seals resulted in rework and

customer complaints
Low tensile seal strength was the cause of leaks
The problem was reformulated to increase the
tensile strength of the seal
h
hig

lo w

h
hig

low

h
hig

lo w

440

seal strengt

420

400

380

360
time

temp

speed

An experiment looked at the effect of temp., melt

time and elevator speed on the tensile strength
Solution: Low melt temp. increases seal strength

Chap. 3: Variation
Reduction Approaches

Battery Seal

17

The Seven Variation Reduction

Approaches
Feedback Control

Fix the Obvious by Reducing

Variation in a Dominant Cause
Output

control

Feed-forward Control
Dominant
Cause

Making a Process Robust

Output

Process

???

Process

control

Desensitize Process
Dominant
Cause

Process

Output Inspection
Output

Process

Output

Chap. 3: Variation
Reduction Approaches

Process

Output

Process

Change the Process Center

Process

18

Statistical Engineering:
An Algorithm for Reducing
Variation in Manufacturing
Processes
Begin with the end in mind
Stephen Covey

A union of engineering and statistics applied to

chronic manufacturing problems
Statistical methods are needed to plan
investigations and to analyze the collected
data
Engineering methods are needed to help plan
the investigations, interpret the results and to
act on the acquired information
INCREASED PROCESS KNOWLEDGE
p
OPPORTUNITIES FOR PROCESS IMPROVEMENTS

Chap 4: Statistical
Engineering Algorithm

Statistical Engineering

The Key is Knowledge

The greatest obstacle to discovery is not

ignorance it is the illusion of knowledge
Daniel Boorstin
By increasing knowledge of how and
why a process behaves as it does, we
will discover cost effective changes to
the process that will reduce variation

Chap 4: Statistical
Engineering Algorithm

There is no substitute for knowledge

W. Edwards Deming

Quickly find a low cost solution

to a chronic problem

Chap 4: Statistical
Engineering Algorithm

Goal of Algorithm

StatEng Algorithm
Statistical methods are needed to plan
investigations and to analyze the collected
data
Engineering knowledge is needed to help plan
the investigations, interpret the results and to
act on the acquired information

Requirements for success

A high volume manufacturing process

A clearly defined chronic process problem
A small team of dedicated problem solvers
Management support and understanding

Chap 4: Statistical
Engineering Algorithm

Uses engineering knowledge and statistical

methods to reduce variation

StatEng Algorithm
Check the Measurement System

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Assess Feasibility and Plan

Implementation of Approach

Implement and Validate Solution

and Hold the Gains

Chap 4: Statistical
Engineering Algorithm

reformulate

Define Focused Problem

 Shainin Red X Strategy

Six Sigma (DMAIC, Breakthrough
Cookbook)
Taguchis Parameter and Tolerance Design
Demings PDSA Cycle
Statistical Engineering is more focused and
prescriptive
Statistical Engineering reflects the iterative
nature of real problem solving.

Chap 4: Statistical
Engineering Algorithm

Competing Algorithms

A systematic approach to Problem Solving /

Variation Reduction is good because it:
Prevents jumping to incorrect solutions
Is a good communication tool
Encourages teamwork
Is teachable
Is manageable

Chap 4: Statistical
Engineering Algorithm

Structured Problem Solving

Define Problem and Check

Measurement Stages
Benefits of establishing a problem
baseline can be enormous
Allows us to know if the problem should be priority
and later whether we have solved problem
Effects design of future investigations

The measurement system is critical

Provides our only view of the process
Checking the measurement system starts the search
for a dominant cause
Often (in our experience) a source of trouble

Chap 4: Statistical
Engineering Algorithm

Part of other algorithms, but

Third Stage
Check the Measurement System

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Chap 4: Statistical
Engineering Algorithm

reformulate

Define Focused Problem

Assess Feasibility and Plan

Implementation of Approach

10
Implement and Validate Solution
and Hold the Gains

Choosing a Variation Reduction

Approach Stage

Fix the Obvious Based on a Dominant Cause

Desensitize the Process to Dominant Cause Variation
Feedforward Control Based on a Dominant Cause
Feedback Control
Make Process Robust to Noise
100% Inspection
Change the Process Center

Choice of approach effects how we proceed

Chap 4: Statistical
Engineering Algorithm

Begin with the end in mind Stephen Covey

7 approaches to reduce variation

11

Fourth Stage
Check the Measurement System

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Chap 4: Statistical Engineering

Algorithm

reformulate

Define Focused Problem

Assess Feasibility and Plan

Implementation of Approach

Implement and Validate Solution

and Hold the Gains

12

 Focus on varying inputs

Use families of causes and method of
elimination (more on this later)
Based (mostly) on sequence of observational
studies

Often the most time consuming stage

Not always needed, but usually worth it

Chap 4: Statistical
Engineering Algorithm

Finding Dominant Cause Stage

13

Assessing Feasibility and

Implementation Stages
e.g. not all solutions require knowledge of a
dominant cause

Use designed experiments on fixed inputs

to assess possible process changes
Note: we delay the use of (expensive)
designed experiments until the assessing
feasibility stage of the algorithm

Chap 4: Statistical
Engineering Algorithm

How to assess feasibility or implement is

different for each of the 7 approaches.

14

StatEng Algorithm Keys

Structured (Stage by Stage) Algorithm
prevents jumping to incorrect solutions
is a good communication tool
encourages teamwork
is teachable and manageable

Selecting a working (tentative) solution approach early on to

drive what we do next
Seven possible variation reduction approaches

Fix the Obvious (or Reformulate) Using a Dominant Cause

Desensitize the Process to Variation in a Dominant Cause
Feedforward Control
Feedback Control
Make Process Robust
100% Inspection
Change the Process Center

Chap 4: Statistical
Engineering Algorithm

15

An important consideration in the algorithm is

whether or not to search for a dominant
cause.
looking for a dominant cause is strongly
recommend!
Separating the search for a dominant cause
from the search for a solution
Specific tools and strategies are associated
with the various stages in the algorithm
A series of investigations is (normally) required
to find a solution

Chap 4: Statistical
Engineering Algorithm

StatEng Algorithm Keys (cont.)

16

A series of investigations are required within

the StatEng algorithm
Problem definition
Measurement system analysis
Searching for a dominant cause
Verification of the dominant cause
Determining if a proposed approach is
feasible
Testing a proposed solution

QPDAC
(Question, Plan, Data,
Analysis and Conclusion)

Framework
There is no substitute for knowledge
W. Edward Deming, 1900-1996

Chap 4: Statistical
Engineering Algorithm

Process Investigations

17

Observational/Experimental
Plans
does not interfere with existing process
may measure inputs/outputs not usually measured
usually low cost (relative to experimental plan)

Experimental plan: deliberately manipulate the

values of one or more inputs (fixed or varying)
usually high cost
logistical challenges
may need to contain produced parts as they may be of
suspect quality

StatEng Algorithm

Observational plan: observe the current process

in action

For each investigation, we propose the QPDAC

(Chap. 5) framework
Specify a clear Question(s) that tells us what
we want to know about the process
Develop a Plan that specifies how we will
collect data to try to answer the question
Collect the Data according to the Plan
Perform Analysis to summarize the data
Draw Conclusions from the investigation to
(try to) answer the question

StatEng Algorithm

QPDAC Statistical Method

 We want to infer how the process will

operate in the future from data collected
over a short period of time
It's tough to make predictions, especially
about the future Yogi Berra
How we collect the data and its quality are
crucial
Process consistency is needed to make
reasonable predictions

StatEng Algorithm

Issues in Process Studies

What are the parts and population available for the

investigation?
i.e. over what time frame will we conduct the investigation?
defines the study population
How will we select units to be included in the sample?
includes the choice of the number of parts
defines the sampling protocol


target population

study population

StatEng Algorithm

Key Decisions in the Plan

of an Empirical Investigation

time

0
sample

What inputs and outputs will we measure or deliberately

change on the selected parts?

Choosing the Problem

Our plans miscarry because they have no aim.
When you dont know what harbor youre aiming for,
no wind is the right wind.
Lucuis Annaeus Seneca, 5 BC-65 AD

First Stage of the Algorithm

Check the Measurement System

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Assess Feasibility and Plan

Implementation of Approach

Implement and Validate Solution

and Hold the Gains

Chap. 6a: Choosing a

Focused Problem

reformulate

Define Focused Problem

 Management should choose projects/problems

based on customer and/or business requirements
(use Pareto Principle, 80/20 rule)
greatest $ return
lowest cost of problem solving
likelihood of success
availability of trained and knowledgeable people
Need management input/decisions to prioritize
DO NOT start a large number of projects
simultaneously!

Chap. 6a: Choosing a

Focused Problem

Projects

Problem Definition
Statistical Engineering requires a focused problem
One project can generate several Statistical Engineering
problem solving efforts
Example leaking engine blocks
Project: Reduce scrap rate due to casting defects in
machined engine blocks
Problems: Eliminate three different failure modes
(center, cylinder bore and rear intake wall) that caused
leaks
Focusing may require studies, new measurement
systems, redefinition of the problem(s).

Chap. 6a: Choosing a

Focused Problem

general problems may not have a single dominant cause

Link Between Projects,

Problems and Investigations
Problem A

Problem B

...

Apply StatEng
Algorithm

Question A1
Baseline

Question A2
Measurement

Question A3

...

Translate management defined projects into

specific problems
Use StatEng algorithm to guide choice of
investigation different at each stage
Use QPDAC framework to help plan, conduct and
analyze each individual investigation

Chap. 6a: Choosing a

Focused Problem

Project

Managements goal was to reduce the rod

scrap rate from 3.2% to less than 1.5%
would be easier to address a more specific
problem
defined in terms of a binary output (scrap or not),
we prefer a continuous output

Chap. 6a: Choosing a

Focused Problem

Connecting Rods Project to Problem

Chap. 6b: Problem

Baseline

Rod Scrap by Day

Scrap rate fairly stable over time

Connecting Rod Scrap Locations

100

Count

60
40

50

Percent

80

100

20

Count
Percent
Cum %

0
nd
gri

85
64.9
64.9

bo

re

24
18.3
83.2

bro

h
ac

14
10.7
93.9

ly
mb
se
as

6
4.6
98.5

ers
Oth

2
1.5
100.0

Grinding (68%) was the dominant location for scrap detection

looking more closely (not shown here), 90% of the scrap at grind
was due to undersized rods

Rod thickness was selected to define the baseline

if thickness variation can be reduced so that undersized rods are
eliminated, scrap reduction is approximately 3.2% x 0 .68 x 0.9 =
1.96%, so overall scrap rate will be approximately 1.25 % (Goal
met)

Chap. 6a: Choosing a

Focused Problem

Defect

 Identify and address the most important failure

modes
Replace a binary or discrete output by a
continuous one, if possible
Define the problem in terms of an output that
can be measured locally and quickly (e.g.
refrigerator frost buildup)
Choose the problem goal to meet the
management project goal

Chap. 6a: Choosing a

Focused Problem

Key Elements of Focusing a Project

to One or More Problems

Process Certification

ensure basic good process management

follow standard operating procedures as written
include safety, training, housekeeping, maintenance
need to have a defined process before improvements
can be made

Elements covered by Quality system standards

such as ISO 9000/QS 9000
Statistical control (i.e. a stable process as defined
by a control chart) is not required for Statistical
Engineering to work

Chap. 6a: Choosing a

Focused Problem

Process certification is a desirable prerequisite to

Statistical Engineering
FIX THE OBVIOUS!

10

 To define the problem we need to select an output

characteristic (or more than one) that can be used to
summarize the size and nature of the problem
Select a critical process output
continuous characteristic (dimension, time, ...)
discrete characteristic (defect count, scrap, ...)
We can summarize the output using a performance
measure, e.g.
mean, standard deviation, histogram, run chart,
capability ratio, ...
scrap/rework rate, run chart, cost, ...

Quantifying the Baseline

If you know a thing only qualitatively, you know it
no more than vaguely. If you know it quantitatively grasping some numerical measure that distinguishes
it from an infinite number of other possibilities you
are beginning to know it deeply.
Carl Sagan, 1932-1996

Chap. 6a: Choosing a

Focused Problem

Selecting an Output

11

First Stage of the Algorithm

Check the Measurement System

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Assess Feasibility and Plan

Implementation of Approach

Chap. 6b: Problem

Baseline

reformulate

Define Focused Problem

Implement and Validate Solution

and Hold the Gains

To complete the first stage of the StatEng algorithm,

we must establish the problem baseline, i.e. quantify
the size of the current problem
The baseline performance is used to
set goals [when is the project completed?]
track progress
help in the search for a dominant cause!
used to plan investigations
used to help in the analysis of the results of
investigations

validate success of a solution

Chap. 6b: Problem

Baseline

Determining the Problem

Baseline

 We conduct a study (i.e. sample and measure parts from

the process) to determine the problem baseline
The specific goals of this baseline investigation are to
estimate/determine the
distribution of output values process center and
process standard deviation, etc.
full extent of variation (FEoV) in the output
nature of the process variation over time (time family
of output variation)
The time family of the output variation provides strong
clues about the nature of the dominant cause (the
dominant cause must act in the same time family as the
output variation)

Chap. 6b: Problem

Baseline

Problem Baseline Investigation

Some outputs (causes) change a lot from one part to

the next, others change more slowly over time.
e.g. raw material properties usually change slowly
whereas piston dimension is different from part to
part
What is slow and fast depends on your perspective
and specific process
e.g. plant environment (daily/seasonal changes),
operators (change each shift)
There are many time families
part to part, hour to hour, shift to shift, day to day,
week to week, etc.

Chap. 6b: Problem

Baseline

Time Families of Variation

 Problem: Excessive scrap due to diameter variation in a

piston manufacturing process.
To assess the time families part to part and hour to
hour suppose we measure diameter on three
consecutive pistons once per hour for 12 hours
output varies slowly

output varies quickly

Chap. 6b: Problem

Baseline

Time Family Example

Time Families Example

output varies quickly

Chap. 6b: Problem

Baseline

output varies slowly

 Knowing the output time family is extremely

useful for planning, it helps us
select an appropriate time frame (i.e. study
population) for future observational
investigations
define a run for future experimental
investigations
Output time family also allows us to eliminate
varying inputs that act in other time families as
suspect dominant causes

Chap. 6b: Problem

Baseline

Uses of Time Family Knowledge

Establishing the Baseline

capture effect of all major sources of variation

e.g. different machines, raw material, operators, etc.
consist of 100s (continuous output) or 1000s of parts
(binary output)
use a systematic sampling plan designed to allow us to
assess a variety of time families

Chap. 6b: Problem

Baseline

Goal: assess process performance (center and

variation), and output time families
Investigation should

Appropriate time frame for baseline data is key

longer is better, but is more expensive
how long is long enough?

 Select 20 consecutive rods twice haphazardly

each day for five days, total of 200 rods
Measure the thickness of each rod at each of the
four positions
total of 800 thickness measurements
Questions
are five days enough? How can we tell/check?
are 800 measurements enough?
why are the two batches of 20 consecutive rod
chosen haphazardly from within each days
production?

Chap. 6b: Problem

Baseline

Connecting Rod Baseline

10

Each row represents a

different rod and position
Each column gives the
values for a different
input
MINITAB worksheet
Most convenient format
for data analysis
Not the default way to
store data in Excel

Chap. 6b: Problem

Baseline

Row/Column Format Baseline Data

11

Rod Baseline Results

mean: 34.6
standard deviation: 11, min and max: 2, 59
Histogram with specification limits 10 and 60
Histogram of thickness
10

70

60

60

Frequency

50

Chap. 6b: Problem

Baseline

Numerical Summaries, thickness = deviation (in

thousands of an inch) from nominal (0.9 inches)

40
30
20

12

10
0

16

24

32
thickness

40

48

56

MINITAB Histogram

Chap. 6b: Problem

Baseline

Graph ,

Adding reference lines

for specification limits

13

Setting the Problem Goal

Want to eliminate undersized rods

Specification range is 10 to 60 thou

Set goal to reduce thickness standard deviation
to less than

60  10
6

8.3

Corresponds to a ~25% reduction from the

baseline standard deviation of 11

Chap. 6b: Problem

Baseline

process well centered already, so need to reduce

thickness variation

14

We can learn a lot about the process and the

nature of the dominant cause by stratifying the
output in a number of ways
by time family, e.g. by day, batch, etc.
by location family, e.g. position
To graphically compare the distribution of output
(or input) values stratified into subprocesses use
an individual values plot with groups (plot on left
on next slide), or
a box plot with groups (plot on right on next
slide) if the number of observations is large

Chap. 6b: Problem

Baseline

Stratifying the Output

15

 Position 3 lower on average

Would the undersized rods (scrap) problem be
solved if we could increase the average thickness
at position 3?

Chap. 6b: Problem

Baseline

Rod Baseline Comparing

Different Positions

16

MINITAB Individual Values

Plot

Chap. 6b: Problem

Baseline

Graph /sW

17

MINITAB Boxplot (With

Groups)

Chap. 6b: Problem

Baseline

Graph 

18

shows a five number summary of the distribution

min, max, median, 25th and 75th percentiles
a summary of a histogram turned on its side
outlying observations are shown with a separate symbol
(rule for outlier vs. min or max varies with software)
Boxplot of thickness
60

max

50

75th percentile

thickness

40

Chap. 6b: Problem

Baseline

Box (and Whiskers) Plot

median
30

25th percentile

20

19

10

min
outliers

Chap. 6b: Problem

Baseline

Rod Baseline Day to Day

Pattern

We see little output variation from day to day, i.e.

the variation in thickness is large and roughly the
same in each of the five days

20

Little variation from batch to batch

20 consecutive parts give the FEoV
helps us choose time frame for future investigations
tremendous clue about the dominant cause

Chap. 6b: Problem

Baseline

Rod Baseline Time Pattern

21

Chap. 6b: Problem

Baseline

Rod Baseline Time Series Plot

22

The proposed sampling plan for a baseline

investigation is systematic
As a result, the elapsed time between parts
follows a consistent pattern but is not the same
for all parts
The standard time series plot is not ideal.
A multivari chart is designed for this sort of data
We look at multivari investigations later when
searching for the dominant cause

Chap. 6b: Problem

Baseline

Multivari Chart

23

Chap. 6b: Problem

Baseline

Rod Baseline Multivari Charts

24

Chap. 6b: Problem

Baseline

MINITAB Instructions
Multivari

25

 For a multivari chart always using the option

Display individual data values
Note that the factor used to define horizontal
axis is the last factor in the list

Chap. 6b: Problem

Baseline

Multivari Dialog Box

26

An estimate of the long term rod thickness variation

(standard deviation, denoted V y) is 11
To meet the goal we need to reduce the output
variation to around 8
Full extent of output variation (FEoV) is 2 to 59
Output varies in the part to part family
Subsequent investigations conducted over a short
time interval should result in the output FEoV
Dominant cause must act in the part to part and
position to position families
Can almost solve the problem by increasing the
average thickness at position three by around 10
units

Chap. 6b: Problem

Baseline

Rod Baseline Conclusions

27

Baseline Over Too Short a Time

Suppose we see the following hypothetical

pattern of output by day
60

40
30
20
10
1

day
Large day to day effect
it is hard to tell what will happen tomorrow
we need to collect data over many more days to
be sure that the baseline variation represents
the long term behavior of the process

Watfactory
An Online Virtual Manufacturing
Environment
Tell me and I will forget.
Show me and I may remember.
Involve me and I will understand.
Chinese Proverb

Chap. 6b: Problem

Baseline

thickness

50

28

Watfactory (Camshaft)
Manufacturing Process

Watfactory

Watfactory is designed to model a

manufacturing process that produces
automotive camshafts

Consider a single output, denoted y300

The target for y300 is zero (measured from
nominal) and specification limits are -10 to 10

Watfactory Process Map

Components

Component B

Assembly

Varying Inputs
x4, x5, x6

Step 100

Component C

Assembly

Varying Inputs
x7, x8, x9
Component D
Varying Inputs
x10, x11, x12
Component E
Varying Inputs
x13, x14, x15

Welding
Step 200

Varying Inputs
x16, ..., x25
Fixed Inputs
z1, ..., z6

Step 100
Output Step 150
y100

Welding
Varying Inputs
x26, ..., x30
Fixed Inputs
z7, ..., z12

Heat Treatment

Machine 1

Step 200
Output

Machine 2

y200

Machine 3

Varying Inputs
Machine #: x31
x32, ..., x45
Fixed Inputs
Can be Set by Machine
z13, ..., z18

Step 250

Step 300

Stream 1
Machine A

Stream 1
Machine B

Stream 2
Machine A

Stream 2
Machine B

Final Output
y300

Varying Inputs Varying Inputs

Stream #: x46
Stream #: x46
x47, ..., x53
x54, ..., x60
Fixed Inputs
Can be Set by Stream
z19, ..., z24
z25, ..., z30

There are three types of process characteristics

one output (y), can be measured at y100, y200, y300
60 varying inputs (xs), change as the process runs
30 fixed inputs (zs), normally constant, but changeable

Watfactory

Component A
Varying Inputs
x1, x2, x3

 Management has determined that the final

output (y300 - straightness measured in microns
from nominal) exhibits too much variation
Your teams goal is to find a cost effective way to
reduce the variation in y300 so that (virtually) all
camshafts are within the specification limits
You have a budget of $10,000 to find a solution
Your team will follow the Statistical Engineering
algorithm (covered in the textbook and
associated videos) and conduct a series of
process investigations looking for a way to reduce
variation in y300

Watfactory

Watfactory Process Game

Process runs 3 shifts, 5 days a week, 1 part per minute

i.e. 1440 camshafts are produced per day
Varying Inputs (x1, , x60)
Type (continuous/categorical)
Process step in which they act (assembly, welding,
heat treatment)
History (pattern of variation over time)
e.g. x25 is the operator in the assembly step,
x42 the cooling temperature in the welding operation
x50 the heating time in the heat treatment step
Fixed inputs (z1, , z30)
Current level, possible range
e.g. z22 is coil length in the heat treatment step

Watfactory

More Process Information

Varying
Input

Description

Type
(# levels)

x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
x13
x14
x15
x16
x17
x18
x19
x20
x21
x22
x23
x24
x25
x26
x27
x28
x29
x30

dimension A
diameter A
hardness A
dimension B
diameter B
hardness B
dimension C
diameter C
hardness C
dimension D
diameter D
hardness D
dimension E
diameter E
hardness E
temperature
fixture
humidity
ball size
orientation
position
pressure
force
offset
operator
temperature
fixture
operator
power
static

continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
categorical (5)
continuous
continuous
categorical (3)
continuous
continuous
continuous
continuous
categorical (3)
continuous
categorical (5)
categorical (4)
continuous
continuous

Observed Varying
Range
Input
unknown
unknown
-2.2, 7.2
unknown
-17.1, 22.2
-14.3, 18.5
unknown
-20, 26.6
unknown
unknown
unknown
-7.5, 19.6
unknown
unknown
-12.6, 21.2
unknown
1, 2, , 5
-3.0, 12.2
unknown
1, 2, 3
unknown
-10.3, 22.4
unknown
-2.5, 6.9
1, 2, 3
-10.6, 15
1, 2, , 5
1, 2, 3, 4
unknown
unknown

x31
x32
x33
x34
x35
x36
x37
x38
x39
x40
x41
x42
x43
x44
x45
x46
x47
x48
x49
x50
x51
x52
x53
x54
x55
x56
x57
x58
x59
x60

Description

Type
(# levels)

Observed
Range

machine #
squeeze time
feed rate
temperature
dimension 1
electrode force
humidity
dimension 2
mandrel position
weld time
load time
cooling temp.
spacing
operator
fixture
stream #
power density
induction level
frequency
heating time
operator
depth
coupling degree
surface area
coil
current
hold time
air gap
inductance
quench temp.

categorical (3)
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
continuous
categorical (5)
categorical (12)
categorical (2)
continuous
continuous
continuous
continuous
categorical (4)
continuous
continuous
continuous
categorical (8)
continuous
continuous
continuous
continuous
continuous

1, 2, 3
unknown
unknown
-17, 29.6
unknown
unknown
unknown
0.2, 10.4
-1.5, 12.1
unknown
-1.6, 9.4
unknown
unknown
1, 2, , 5
1, , 12
1, 2
unknown
-20, 26.2
-14.5, 29.5
0.9, 8.4
1, 2, 3, 4
unknown
unknown
-14, 21
1, , 8
unknown
unknown
unknown
-8.8, 13.4
-4.3, 8.8

Watfactory

Varying Inputs Information

Watfactory

Input Time Family Information

Watfactory Login

Watfactory

Web site: www.student.math.uwaterloo.ca/~watfacto/login.htm

Login ID and Password are given at registration (each

team has access to a different copy of the process)
A guest login (to a different version of the process) is
also available

Gives summary information on

virtual date
remaining funds
y300 specification limits
links to more information

You can
request data from previous
studies
change your password
see investigation/solution
history

Watfactory

Team Home Page

Observational: prospective, retrospective

Experiments: with varying inputs, fixed inputs or both
Offline experiments: e.g. component swap
Solutions: process changes

Watfactory

Available Empirical Investigations

10

For each investigation you need to specify

Type of investigation
(observation/experimental,)
What input(s): x1, , x60 (if any) and/or
output(s): y100, y200, y300 to measure
How many parts and which parts (camshafts) to
measure
For experimental plans you also need to specify
which fixed inputs (z1, z30) and/or varying
inputs (x1, , x60) to control to which levels and
when

Watfactory

Conducting Investigations

11

There is a cost (in $ and time) for each investigation.

Cost influences:
Type of investigation
Prospective/observational studies are cheaper
Number of parts
Which inputs/outputs are selected. Cost/part
measuring each input and output, e.g. $1/part for y300
tracing parts through the process, i.e. matching inputs
and/or outputs measured at different processing steps
The cost for each investigation you conduct is recorded!
Costs can be determined before running an investigation

Watfactory

Investigation Costs

12

Prospective Measurement Costs

Components

100

150

Varying Measurement Process Varying Measurement

Input
Step
Input
Costs Per
Costs Per
Part
Part
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
x13
x14
x15
x16
x17
x18
x19
x20
x21
x22
x23
x24
x25
x26
x27
x28
x29
x30

3
2
5
2
3
5
1
1
5
1
3
5
3
2
5
1
1
2
1
1
1
2
6
1
1
1
1
1
5
2

200

250, 300

250

300

x31
x32
x33
x34
x35
x36
x37
x38
x39
x40
x41
x42
x43
x44
x45
x46
x47
x48
x49
x50
x51
x52
x53
x54
x55
x56
x57
x58
x59
x60

2
1
1
1
2
2
1
1
1
1
4
2
2
1
1
1
1
2
2
2
1
1
1
2
1
2
12
1
1
2

Watfactory

Process
Step

13

 Tracing costs are applicable when inputs and/or

outputs are measured at different process steps.
This cost accounts for the additional expense of
tracing parts through the manufacturing process.
Tracing costs are in addition to the standard
measurement costs for any input or
Input Based Tracing Costs Per Part
Cost Links to Output
Inputs
Components (x1, , x15)
6
y100
Step 100 (x16, , x25)
3
y100
Step 150 (x26, x30)
6
y200
Step 200 (x31, , x45)
3
y200
Step 250 (x46, , x53)
6
y300
Step 300 (x54, , x60)
3
y300

Output Based Tracing Costs Per Part

Upstream Downstream Cost
Output
Output
y100
y200
12
y100
y300
22
y200
y300
10

Watfactory

Tracing Costs (per part)

14

Virtual time elapses when you conduct

investigations in Watfactory
Time elapsed depends on your choice of study
population
time elapsed is rounded up to nearest shift
minimum investigation time is 1 shift

Watfactory

Investigation Time

Your team home page shows your current virtual

time in terms of weeks, days and shifts since the
start
15

Other Investigations

measurement assessment
retrospective
assembly versus components
component swap
experiments with varying inputs, fixed inputs or both

We cover these costs and time elapsed later

when the investigation is needed

Watfactory

There are also special investigation costs and time

associated with the other types of investigations
such as

16

Goal is to reduce variation in y300

a solution requires a process change
Possible solutions include
changing 1+ fixed input (z1,z30)
adding 100% inspection
reducing varying input variation (x1,,x60)
adding a feedback controller
adding a feedforward controller
Solution cost (per part) depends on the type of
solution selected

Watfactory

Possible Solutions

17

Hints and Suggestions

look for a dominant cause(s) of output
variation
use a series of studies
focus on fixed inputs that act in the same
processing step as the dominant cause(s)

Watfactory

Use the StatEng algorithm

To find a solution

Use only process knowledge obtained

inside Watfactory (realistic, but not real)
18

You will conduct a series of empirical investigation in

Watfactory
Often the plan for the next investigation will best be
determined using knowledge gained from previous
investigations
As a result, it is helpful to stay organized
Suggestions
Create a new Minitab project with a sensible name
for each investigation
Summarize the plan and conclusions from each
investigation in a single document

Watfactory

Organization of Data and Results

19

Watfactory Project Reports

plan investigation
collect the data in Watfactory
conduct an analysis to draw conclusions
write a short report that summarizes your rationale
for choices made in the plan and gives a summary of
your conclusions

Use the QPDAC (Question, Plan, Data, Analysis

and Conclusion) framework to organize each
report

Watfactory

Summarize progress in 9 written reports

Each report describes 1+ investigation

20

Baseline/prospective investigation
Measurement system assessment
Assembly/disassembly and component
swap investigations
Retrospective investigations
Experimental (with varying and/or fixed
inputs) investigations
Possible solutions

Watfactory

Available Watfactory Videos

21

1st Watfactory Investigation

Establishing the Baseline
, 4 BC AD 65

Prerequisites

Chapter 6 covers the baseline investigation

Watfactory Baseline

Watched videos and read textbook for Chapters

1-6

Current Algorithm Stage

reformulate

Define Focused Problem

Check the Measurement System

Feedback Control
Make Process Robust
100% Inspection

Change Process (or Sub-process) Center

Find and Verify a Dominant

Cause of Variation

Watfactory Baseline

Choose Working Variation Reduction Approach

Fix the Obvious
Desensitize Process
Feedforward Control

Assess Feasibility of Approach

Implement Approach

Validate Solution
and Hold the Gains

Complete the first stage of the Statistical

Engineering algorithm by
Estimating the process variability, i.e. V y , and
center P y
Determining the full extent of variation (FEoV) in
the output
Determining the time pattern in the output
variation, e.g. does the output vary a lot from part
to part, hour to hour, shift to shift, day to day,

Watfactory Baseline

Baseline Goals

Select a plan to address the baseline goals

Decide what inputs/outputs to measure
Choose the study population
period of time when you collect data

Select a sampling protocol and sample size

Watfactory Baseline

Investigation Plan

Costs
See prospective investigation costs in the
Watfactory introduction video or Watfactory
diagnostic journey written summary
Recall measurement and tracing costs

Elapsed Time
Depends on study population
Should not be more than 1 week (at least for first
baseline investigation)

Watfactory Baseline

Investigation Cost and Time

Watfactory Baseline

Watfactory Baseline

Baseline Investigation Selection

Watfactory Baseline

Random Sampling Example

Watfactory Baseline

Systematic Sampling Example

10