Design of Experiments

Pat Hammett

Introduction to Design of Experiments (DOE)

Experimental Design Variables

Process Input Variables (Factors) Control Variables (Design Factors) - signal factors (location or mean effects) - noise factors (dispersion or variation effects) Manufacturing Process Process Output, or Response Variable, or Product Characteristic Process Output Variables

Uncontrollable Variables - environmental factors those that are difficult or very expensive to control. (e.g., ambient temperature)
2

TQM - University of Michigan

Design of Experiments

Pat Hammett

Duality of Signal & Noise Factors

Effect of Input Variables on a Process Output Adjustment Factors affects the mean of the process without affecting variation knob. Need these to center a process. (especially if Pp is high, Ppk low) Example: tool position knob in machining

Signal Noise Duality Factors location & Factors (dispersion effects) dispersion (location effects)

Dispersion Effect?
Yes No Yes Duality Factor Adjustment Factor No Noise Factor Robust Factor
3

Location Effect?

Operating Windows
Defined:

Output Variable

Input Variable
4

TQM - University of Michigan

Design of Experiments

Pat Hammett

Summary of Terminology
n

Most input variables have a robust operating window in which the output is insensitive or able to meet manufacturing specifications. (i.e., relatively few pure linear relationships).
Output Variable

Input Variable
5

Experimental Design and DFSS

n

In Design for Six Sigma (DFSS), use DOE to:

n

Identify key input variables which affect the mean or variation of a key product output variable. After determining key inputs, mfgs should either:
n n n

Identify robust levels and establish controls to maintain, Fix the settings for an input variable, or Remove input variable effect by re-designing process.

If an input variable does not affect the output, then a mfg may either:
n n

Ignore the variable. Establish control plan to insure that input continues to have no impact on output.
6

TQM - University of Michigan

Design of Experiments

Pat Hammett

Conducting an Experiment
n

Identify the process output(s) of concern. Identify process input variables which you believe affect the process output. Select input variables to study.
n

Use FIRM Approach (next slide)

Select levels or settings for each process input variable. Run the experiment at various combinations of levels. Analyze the data for significant main effects (input variable effects) and interaction effects.
7

FIRM Approach to DOE Variables

n

For each potential input variable identified, determine a model strategy.

n n n n

F - fix the settings for input variable I - ignore the settings for an input variables R - randomize the settings for an input variable M - model the variable - identify settings for your variables in which you want to test.

Remember the cost of experimentation can grow significantly if too many variables and/or levels of variables are selected.
8

TQM - University of Michigan

Design of Experiments

Pat Hammett

Experimental Combinations
n

The number of experimental combinations is based on the:

n n n

number of factors (input variables), number of levels for each factor, number of factor interactions you wish to study.

Suppose you wish to study three variables (factors) with two levels for each variable.
Experimental Combinations = LF = 23 = 8 combinations (Full Factorial Design) L = # of Levels, F= # of Factors

How many total combinations would you have if you tested 2 factors at 2 levels and 1 factor at 3 levels?

Example 1: Catapult Experiment (In-class)

n

Process Output Variable

Input Variables

Model (FIRM)

10

TQM - University of Michigan

Design of Experiments

Pat Hammett

Example 2: Detergent Box Filling Experiment

Experiment: Determine the effect of three process input variables or factors on the fill rate of a box-filling machine for detergent powder. (goal or target = fill rate = 3.0 kg/s) Process Output Variable or Response: Fill Rate Process Inputs (Factors):Settings of Process Parameters (Levels) Diameter of Chute Short - 4 Long - 6 Length of Chute Detergent Particle Size Medium - 8 Small Tall - 12 Large

Are these input variables controllable or uncontrollable?

11

Input Variable Combinations

(full factorial model)
Diameter Length Size
Small Medium Large Small Large Small Long Medium Large Small Tall Large

Fill Rate
1.0 1.3 1.2 1.4 2.8 3.2 3.5 3.3
12

Short

Tall

TQM - University of Michigan

Design of Experiments

Pat Hammett

Average Fill Rates by Input Variables

Process Inputs (Factors): Diameter of Chute Average Fill Rates Small: 1.23 Large: 3.20 (1.0+1.3+1.2+1.4)/4 (2.8+3.2+3.5+3.3)/4 Medium: 2.10 Tall: 2.35 (1.0+1.3+2.8+3.2)/4 (1.2+3.5+1.4+3.3)/4

Length of Chute

Detergent Particle Size

Small: 2.12 Large: 2.30 (1.0+2.8+1.2+3.5)/4 (1.3+3.2+1.4+3.3)/4

Which factor(s) are location effects? How might you determine if a factor is an adjustment factor?
13

Using Excel to Solve the DOE

Must code variables for regression: diameter (-1 = small ; 1= large) length (-1 = medium ; 1 = tall) particle size (-1 = small ; 1 = large) Code: -1 or 1
Length -1 -1 -1 -1 1 1 1 1 Diameter Size Fill Rate -1 -1 1 -1 1 1.3 1 -1 2.8 1 1 3.2 -1 -1 1.2 -1 1 1.4 1 -1 3.5 1 1 3.3

To test for significance of variables: Use Regression Function (Under Data Analysis of the Tools Menu) See above for input range in regression function.

Note: You can only use the regression function in excel for DOE if you only have two levels for each factor.
14

TQM - University of Michigan

Design of Experiments

Pat Hammett

Regression Output For Example

SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations ANOVA df Regression Residual Total 3 4 7 SS 8.01375 0.135 8.14875 t Stat 34.06367 2.116951 15.20356 1.347151 P-value 4.43E-06 0.101691 0.000109 0.249188 MS F Significance F 2.67125 79.14815 0.000511769 0.03375 0.991681926 0.983433042 0.971007823 0.183711731 8

Note: the larger the R2, the more you explain the Variation in your output.

Intercept Length Diameter Size

Coefficients Standard Error 2.2125 0.064951905 0.1375 0.064951905 0.9875 0.064951905 0.0875 0.064951905

If alpha = 0.05, which variables are significant?

15

Factor interactions?
n

Simultaneous effect of two or more factors on the response variable. Two-way interactions (involve two variables)
n

Box-Filling Example: Chute Diameter x Chute Length Note: beyond the scope of this class.
Length = tall
3

Three-way interactions (involve three variables)

n

Average Fill Rate

3.5

2.5 2 1.5 1 0.5 0

Length = medium

Is there an interaction between diameter & length?

large
16

small

Chute Diameter

TQM - University of Michigan

Design of Experiments

Pat Hammett

Model with 2-Way Interactions

Length -1 -1 -1 -1 1 1 1 1
ANOVA df Regression Residual Total Coefficients 2.2125 0.1375 0.9875 0.0875 0.0625 -0.0875 -0.0375 6 1 7 SS MS F Significance F 8.1175 1.352917 43.29333 0.115816468 0.03125 0.03125 8.14875 Standard Error 0.0625 0.0625 0.0625 0.0625 0.0625 0.0625 0.0625 t Stat 35.4 2.2 15.8 1.4 1 -1.4 -0.6 P-value 0.0180 0.2716 0.0402 0.3949 0.5000 0.3949 0.6560

Diameter Size LengthxDiameter -1 -1 1 -1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 -1 1 -1 1 -1 1 1 1 1

LengthxSize 1 -1 1 -1 -1 1 -1 1

DiameterxSize 1 -1 -1 1 1 -1 -1 1

Fill Rate 1 1.3 2.8 3.2 1.2 1.4 3.5 3.3

Intercept Length Diameter Size LengthxDiameter LengthxSize DiameterxSize

Are any interactions significant?

17

Identifying Two-Way Interactions

(note: data below differ from prior experiment)
Conditions: (1) Lines Not Parallel; (2) A or V conditions. Interaction Effect Shown Below: Here, Average Fill Rate increases when diameter is large & chute is tall
Two-Way Interaction Plot

Average Fill Rate

3.5 3 2.5 2 1.5 1 0.5 0

Chute Length = Tall

Chute Length = Medium

small

Chute Diameter

large
18

TQM - University of Michigan

Design of Experiments

Pat Hammett

Catapult Example
n

The table below represents raw data from an experimental design on the catapult. Factors = return height (3 levels: low,medium,high) & ball position (2 levels: position 1 and position 2) Output Variable is launch distance (n=4 samples / comb.)
A 1 2 3 4 5 6 7 8 9 Pos 1 B Low 71 75 77 76 51 53 48 50 C Medium 74 71 69 74 50 52 46 48 D High 67 69 66 68 45 41 40 42
19

Pos 2

Two-Factor ANOVA using Excel

n

Under Data Analysis Menu: Select Two-factor ANOVA w/ Replication

n n

Note: you may have to use add-in function for tool pack. Two-Factor ANOVA inputs for this example: n input range: A1:D9, rows per sample: 4 alpha: 0.05
A 1 2 3 4 5 6 7 8 9 Pos 1 B Low 71 75 77 76 51 53 48 50 C Medium 74 71 69 74 50 52 46 48 D High 67 69 66 68 45 41 40 42

Pos 2

20

TQM - University of Michigan

10

Design of Experiments

Pat Hammett

Excel Output for Example

Anova: Two-Factor With Replication SUMMARY Pos 1 Count Sum Average Variance Pos 2 Count Sum Average Variance Total Count Sum Average Variance 8 501 62.6 172.8 8 484 60.5 156.6 8 438 54.8 188.5 4 202 50.5 4.3 4 196 49.0 6.7 4 168 42.0 4.7 4 299 74.8 6.9 4 288 72.0 6.0 4 270 67.5 1.7 Low Medium High Total

(from Two-factor ANOVA w/ Replication using excel Data Analysis)

12 857 71.4 13.7 12 566 47.2 19.2

21

Excel Output for Example

n

(from Two-factor ANOVA w/ Replication using excel Data)

Note: the sample variable is the row variable. In this case, ball position.

ANOVA Source of Variation Position Sample Height Columns Interaction Within Total

SS 3528.38 265.583 6.25 90.75 3890.96

df

MS F P-value F crit 1 3528.375 699.843 7E-16 4.414 2 132.7917 26.33884 5E-06 3.555 2 3.125 0.619835 0.5491 3.555 18 5.041667 23

22

TQM - University of Michigan

11

Design of Experiments

Pat Hammett

Settings & Design for Six Sigma

n

Suppose your objective is to hit a target 50 away with a tolerance of +/- 8. What level(s) for each input variables would you need to produce less than 3.4 DPM?
n

Note: try to make your process as robust as possible to your input variables.

23

Scatter Plot of Height Settings

n

How would you test if these settings differ?

Catapult Data
Launch Distance (inches)

55 50 45 40 35

Low

Medium
Return Height

High

24

TQM - University of Michigan

12

Design of Experiments

Pat Hammett

Identifying Settings & Six Sigma

n

How might you improve this process to meet your six sigma quality objectives? Should you rely on the variance from your experiment to determine if you are able to achieve your six sigma quality objectives?
Low 51 53 48 50 Medium 50 52 46 48 =ftest() =ttest() Average StDev Pp Ppk 0.73 0.40 49.75 2.31 1.1521 1.1161
25

Pos 2

Example 3: Injection Molding

(plants wants to reduce part shrinkage)
Control Factors Cycle Time Mold Temperature Holding Pressure Holding Time Level 1(Old) 10 mins 100 (C) 1 atm 10 sec Level 2 (New) 8 mins 110 (C) 0.8 atm 12 sec Statistical Results p-value = 0.554 p-value = 0.325 p-value = 0.756 p-value = 0.003 (level 2: less shrinkage)

Uncontrollable Factors % of Regrind Pellets Ambient Temperature

Level 1 5% 25 (C)

Level 2 10% 28 (C)

What settings would you use for each variable? Cycle Time: Mold Temp.: Hold Press.: Hold Time:
26

TQM - University of Michigan

13

Design of Experiments

Pat Hammett

An Alternative to ANOVA Method: Taguchis Signal-to-Noise Ratio

Signal To Noise Ratio : Example : Nominal is Best = SN(Target) = 10 log ( where
n

) s2 y = average response, s = standard deviation of the response

y2

Taguchi Experiments -- often do not use statistical inference to determine significance. Rather they compute signal to noise ratios for all variables and use settings which produce best ratios. Taguchi combines mean and variance into a single response -- traditional DOE separates them.
n

Note: ANOVA assumes homogeneity of variance.

Why does Taguchi use log transformation?

27

Taguchi & His Methods

n

Genichi Taguchi
n

a Japanese engineer who has developed both philosophies and methodologies for quality improvement. often credited with playing a significant role in bringing statistically designed experiments to industry. his work has received strong reactions from academia and industry. Typical academic reactions to Taguchi in the use of:
n n

statistical techniques - much controversy over its lack of statistical rigor. quality engineering philosophy - much praise for its simplicity.

Industrial reaction to Taguchi Concepts n Taguchi concepts are often considered as more user-friendly than traditional design of experiments methods.
28

TQM - University of Michigan

14