Mse

Sigma Science: Measurement System Evaluation
7.1
Measurement System Evaluation

(MSE)
At the end of this chapter, you will be able to:
Evaluate the requirements and characteristics of

measurement systems as they are used to either check
compliance or gauge process improvement.
Understand the techniques useful in evaluating

measurement systems.
Initiate an ongoing, data based system to

appropriately validate and calibrate measurement
equipment.
If you cannot measure, you cannot improve!
050207
Ross, Sanders & Cooper
7.2
ISO 9001 Requirements

Section 4.11: Control of inspection, measuring, and test
equipment.
Determine the measurements to be made, accuracy
required, and select appropriate inspection,
measuring, and test equipment that is capable...
Identify, calibrate and adjust all inspection,
measuring and test equipment at intervals against
standards.
Identify equipment, maintain records, safeguard
equipment...
Define, establish, document, and maintain calibration
process, including frequency of checks, acceptance
criteria and actions to be taken when results are
unsatisfactory.
Equipment shall be used in a manner which
ensures that measurement uncertainty is known &
is consistent with the required measurement
capability.
050207
7.3
Why Measure?
To make a decision:
Meet standards & specifications
Compliance requirement
Detection/reaction oriented
Short-term results
Compare measurement variation to specifications
To learn about the process/product and how to
improve it:
How much to improve?
Prevention oriented
Long-term strategy
Compare measurement variation to the change
you need to detect (e.g., product variation)
To ensure consistency
050207
7.4
Measurement is a process. The objective of the MSE is to

learn as much as possible about the measurement
process in a short amount of time, identify
improvements needed and then establish on-going
control.
The snapshot study can provide useful information about
the current operation and the potential work required for
a measurement process.
Just as with any other process, we are concerned with the
level of the process and the variability in the process.
Also, the measurement process may shift, drift, etc. over
time.
In order to improve the measurement process, Y=f (x)
must be understood. What xs are driving variation in the
measurement process? Process maps are used to understand
the sources of variation in the measurement process.
Just as control charting identifies and quantifies sources of
variation affecting a process, so an MSE identifies and
quantifies the sources of variation that affect a
measurement system.
050207
7.5
Example 1: Material Batches

Over the course of one day, each hour, five samples (one
per batch) are selected from the line making batches of a
material. A key material characteristic is measured with a
specification of 100 20.
Questions:
Based on the above sampling plan, what assumptions
were probably being made about the measurement
process? About within batch variation?
Provide a set of questions that could have led to this
sampling plan.
What sources of variation are captured within
subgroup? Between subgroup?
050207
7.6
Evaluate the information in the control charts. What

questions are generated? Develop a subsequent data
acquisition plan.
100
UCL=91.4
Range of Y
80
60
Avg=43.2
40
20
12
16
20
24
130
UCL=124.5
Mean of Y
120
110
100
Avg=99.6
90
80
LCL=74.7
70
050207
12
16
20
24
Hour
7.7
Sampling Plan (Tree Diagram)
050207
7.8
Mapping the Measurement Process

Constant from titration to titration
Changes titration to titration
(N) = Noise, (C) = Controlled, (NC) Noise that could be controlled, but we do not
(NM) = not currently measured, (CM) = currently measured
Changes within titration
y = grams of
sample (CM)
Transfer
sample to 100
mL volumetric
flask
y1 = uniformity of
solution (NM)
y2 = final volume (CM)
Charge solvent
and dissolve
(NC) Sample on sides

(NC) Cleanliness of flask
(NC) Sample on sides
(C) Sample shaken
(NC) Balance used
(NC) Time sample is
(C) Balance type
shaken
(N) Balance fluctuation
(NC) Time with lid off
(NC) Temperature of sample (NC) Temperature of
(N) Ambient temperature
sample
(NC) Shake sample
(NC) Time sample is shaken
(NC) Time with lid off
(NC) Deviation from target
weight
(C)Cleanliness of balance
(C)Time from last calibration
of balance
y1 = mL of solution
(CM)
y2 = waste (NM)
Pipette sample
into 250 mL
volumetric
flask
(N) Pipetting method

(NC) Cleanliness of
pipette
(NC) Cleanliness of flask
(NC) Pipetting skills
(NC) Pipette used
(C) Volume of pipette
used
(C) Temperature of
sample
y1 = mL of indicator (CM)
y2 = mL of reagent (CM)
y3 = mL of solvent (CM)
Charge
reagents
(NC) Shake reagents

(NC) Equipment used for
measuring indicator
(N) Accuracy of measuring
in graduated cylinder
(NC) Cleanliness of
graduated cylinders
(NC) Cleanliness of
equipment used for
indicator
(NC) Reagents made
(C) Graduated cylinder
used for salt solution
(C) Graduated cylinder
used for solvent
(C) Charge order
(C) Expiration date on
reagent
Y = %Active (CM)
y1 = mL titrated
(CM)
y2 = waste (NM)
Titrate
(C) Calculation used

(NC) Shake titrant
(NC) Shake flask before
titrating
(NC) Time sample sits
before titrating
(NC) mL of titrant added
before shaking
(N) Accuracy of burette
reading
(N) Final color of blue
(NC) Preparation of titrant
(C) Burette used
(C) Expiration date on
reagents
(NC) Amount of time since
reagents have been opened
or used
(C) Air bubbles in burette
In order to improve the measurement process, the focus

should be on discovering Y=f(x). What xs are driving
variation in the measurement process?
050207
7.9
Some Important Characteristics of a

Measurement Process
1
Discrimination: Measure of the smallest unit of

measurement effectively reported by the device (also
resolution).
Precision:
The variation between successive
measurements obtained under stipulated conditions.
o Repeatability: Precision where the conditions are
the same characteristic on the same part by the
same person using the same instrument.
o Reproducibility:
Precision where conditions
include using different operators on different
instruments (or labs).
o
Accuracy: The difference between the observed average
value of measurements and the accepted reference
value.
Bias: Condition where the central tendency (e.g.,

mean) of an estimate deviates from the true value.
Sensitivity:
The ability of the device to detect a
change in output (based on a minimum input.)
Stability: The ability of the measurement process to

maintain its characteristics (e.g., discrimination,
precision, accuracy, reproducibility) over time.
Applicability: Intended purpose for the measurement

system (e.g., compliance, improvement)
For definitions see ISO 8402 and 5752
050207
7.10
Just as control charting identifies and quantifies sources of

variation affecting a process, so an MSE identifies and
quantifies the sources of variation that affect a
measurement system.
To what can we attribute the total variation seen in the
previous control charts?
!Total =
2
!Product + !Measurement System

=
Measurement System
Operator
Within Sample
Measurement Device
Technique
050207
7.11
Investigating the Measurement Process

(Example 1 continued)
In order perform a short-term investigation of the

measurement process, data is collected as follows:
Five samples are randomly selected from the line

making batches.
Three operators are selected to measure the samples.

In this process, the operators are responsible for
measuring samples of the material batches.
Each operator measures each sample three times.
The operators are instructed to use techniques and

practices that they would normally use.
Assignment:
1. Draw the sampling tree for this data acquisition.
Discuss the types of information available in the data.
What information might not be available?
050207
7.12
2. Based on the dot frequency diagram shown below,

where are the largest sources of variation in the
measurement process?
Is the within subgroup
variation stable?
105
Each Box describes the three

measurements on the sample
100
Y
95
90
85
1 2 3 4 5
Operator 1
050207
1 2 3 4 5
Operator 2
1 2 3 4 5
Operator 3
UCL=7.9
Avg=3.1
10
95
Avg=94.6
LCL
=91.4
Mean of Y
100
90
85
050207
15
UCL
=97.7
Range of Y
8
7
6
5
4
3
2
1
0
7.13
10
15
7.14
Assessing the Discrimination of a

Measurement Process
Are there sufficient measurement units to see variation in
the sample? Does the measurement system have enough
resolution?
To assess discrimination, compare the number of
measurement units (smallest increment reported by the
measurement system) to the measurement error using the
range chart. How many measurement units are below the
upper control limit on the range chart? (Include 0 in your
count.) The recommendation is to have at least 5
measurement units under the control limit.
Range Charts (Subgroup Size =3)
NOT OK
1.25
UCL=1.159
OK
1.25
1.0
0
0.75
0.5
0
0.25
0.5
0
0.25
0.0
0
8 10
Unit is 0.5
050207
UCL=3.99
4.0
UCL=1.197
3.0
1.0
0
0.75
Avg
=0.450
OK
0.0
0
Avg
=0.465 2.0
1.0
Avg
=1.55
0.0
2
8 10
Unit is .01
8 10
Unit is 0.5
7.15
Precision:
Ideal Process Target
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
050207
7.16
Assessing the Precision/Repeatability of

the Measurement Process
How closely do the successive measurements repeat? Is
the amount of within subgroup variation consistently
small enough to be able to detect the differences we need
to see to improve the process?
1)
If the range chart is out-of-control, then a measurement

cannot be consistently repeated.
If the range chart is in-control, the precision of the
measurement process can be estimated.
The
measurement uncertainty (i.e., the measurement
standard deviation) is estimated as:
R
" =
d
m
2)
050207
If the X-bar chart is in-control, the dominant sources of

variation are within subgroup. In other words, the
measurement system cannot distinguish between the
samples!in the study. The measurement variation
overshadows any product variation.
7.17
Bias
Ideal Process Target
Instrument 1
Day 1
Operator A
050207
Instrument 2
Day 2
Operator B
7.18
Determining Bias & Lack of Reproducibility

in the Measurement Process
A measurement process is biased if differences exist in the
average of the measurements made by different persons,
machines, etc. when measuring the identical characteristic.
The average of measurements is different by a fixed
amount.
To identify operator bias in the measurement process,
subgroup averages are plotted in the same sample (part)
order by operator.
If the operators have different averages, but their patterns
are the same, we say there is an operator bias. On the
other hand, different patterns between operators signals
the existence of operator by sample interactions.
Each point represents the
average of the three
measurements on the part
Measurement
105
100
95
90
85
1
Operator 1
050207
Operator 2
Operator 3
7.19
Example 2: Reactor Charge

A reactor is charged with sacks of chemical A. The weight
of three sacks is measured and recorded for each of three
shifts.
Week
Shipment / Lot
Weigh Scale
Reactor
Day 1
Shift 1
2
3
Sack 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Measurement
3
61 62 63
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
61 62 63
Subgroup on
control chart
Assignment:
What is captured within subgroup?
Between subgroup?
Evaluate the information in the control charts.

What can you conclude about the measurement system?
050207
7.20
5
4
UCL=3.92
Range of Y
3
2
Avg=1.52
1
0
LCL=0
10
12
14
16
18
20
22
Mean of Y
117
UCL=116.35
116
115
Avg=114.79
114
LCL=113.23
113
2
050207
10
12
14
16
18
20
22
7.21
Sampling Tree
050207
7.22
Investigating the Measurement Process

(Example 2 continued)
In order to investigate the measurement process, data is

collected as follows:
Twelve sacks are randomly selected from a days

production.
A single operator is selected to measure the sacks.
Each sack is measured three times.
The operator is instructed to use techniques and

practices that they would normally use.
Assignment:
1. Draw the sampling tree for this data acquisition
strategy. How would you recommend the study be
conducted?
2. Discuss the types of information available in the data.
What information might not be available?
3. Based on the dot frequency diagram, what comments
can be made about the measurement system? Is the
within subgroup variation stable?
050207
Sack
1
2
3
4
5
6
7
8
9
10
11
12
7.23
Rep
2
114
115
114
115
116
114
115
115
114
115
114
115
1
115
114
114
115
115
114
114
113
115
115
115
114
3
115
114
114
115
114
114
115
114
115
113
114
114
116.5
116.0
115.5
115.0
114.5
114.0
113.5
113.0
112.5
10
11
12
Sack
050207
7.24
Example 2 continued: The Weigh Scale

Discrimination: Are there adequate measurement units to
detect the variation present?
Sack
1
2
3
4
5
6
7
8
9
10
11
12
1
115
114
114
115
115
114
114
113
115
115
115
114
Rep
2
114
115
114
115
116
114
115
115
114
115
114
115
3 Range Average
115
114
114
0
114.00
115
0
115.00
114
2
115.00
114
0
114.00
115
1
114.67
114
2
114.00
115
1
114.67
113
2
114.33
114
1
114.33
114
1
114.33
1
114.44
R and X
Complete the arithmetic and control charts. (Control chart

constants are reproduced at the end of the module.)
050207
7.25
Example 3: ID with Calipers

Production run of parts 1
1 hour in morning 1
Operator
Caliper
3
4 5
Part
1
1 2 3
2
4 5 6
3
7 8 9
10 11 12 13 14 15
16 17 18 19 20 21 22 23 24
5
43 44 45
Measurement
How many parts are included in the study?

Comment on how representative the parts are of all
production.
How many operators are in the study?
How many times is each part measured?
Complete and analyze the dot frequency diagram.
050207
7.26
Production run of parts 1

1 hour in morning 1
Operator
Caliper
3
4 5
Part
1
1 2 3
2
4 5 6
7 8 9
10 11 12 13 14 15
16 17 18 19 20 21 22 23 24
5
43 44 45
Measurement
Operator/
Caliper
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
Part
Range
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
21.12
23.20
22.83
25.07
22.37
21.40
23.19
22.97
24.88
22.15
21.19
23.30
23.08
24.93
22.25
21.15
23.48
22.99
25.86
22.32
21.52
22.82
21.96
24.72
20.90
20.91
23.96
23.48
24.88
22.06
21.58
23.64
21.22
25.59
22.51
21.93
22.51
22.82
24.50
23.15
20.11
23.38
22.30
25.75
22.14
0.46
0.44
1.77
0.79
0.19
0.53
0.68
1.01
0.38
2.25
1.08
0.66
1.18
0.87
0.19
Average
21.283
23.440
22.347
25.507
22.400
21.617
22.840
22.583
24.700
22.067
20.737
23.547
22.953
25.187
22.150
The control charts are shown on the next page. Identify

the data from the different operators.
050207
7.27
3.0
2.5
UCL=2.142
Range of Y
2.0
1.5
1.0
Avg=0.832
0.5
0.0
2
10
12
14
16
27
26
Mean of Y
25
24
23
Avg=22.89
22
21
20
2
10
12
14
16
Sample
How would your work on the measurement system

proceed?
050207
7.28
Example 4: CMM (Coordinate Measurement Machine)

Table teardown / Calibration / Morning 1
Operator
1
Part
Part Setup
1
3
1 2 3 4 5 6 7 8 9 10 11 12 13 14
11 12 13 14
Measurement 1 2 3 4 5 6 7 8 9 10 11 12 13 14
25 26 27 28
Subgroups
2.5
UCL=2.357
Range of Dim.
2.0
1.5
1.0
Avg=0.722
0.5
0.0
-0.5
2
10
12
14
16
Sample
1145
Mean of Dim.
1140
1135
UCL=1131.66
Avg=1130.31
LCL=1128.95
1130
1125
1120
2
10
12
14
16
Sample
050207
7.29
Can the CMM detect differences between these seven

parts?
Visually, are the operators consistent?
If there had been two CMMs at two facilities (one at each

plant), how would you modify the sampling scheme to
include them in the study?
050207
7.30
Stability (Consistency)
An On-going Evaluation of Measurement Systems
A process that is consistent, predictable and repeatable

over changing noise.
The results may not hold on a different day or once the

CMM
is
calibrated.
How
would
you
check
the
measurement system on an on-going basis? (i.e., How

would you check the measurement system stability?)
050207
7.31
Determining Bias & Lack of Reproducibility

in the Measurement Process
A measurement process is biased if differences exist in the
average of the measurements made by different persons,
machines, etc. when measuring the identical characteristic.
The average of measurements is different by a fixed
amount.
Bias effects include:
Operator bias - different operators get detectably

different averages for the same parts.
Reproducibility refers to the capability of operators to

get the same average and pattern of measurements
for the same parts.
Machine bias - different machines get detectably

different averages for the same parts.
Other biases - day to day, fixture to fixture, supplier to

supplier, etc. differences in averages.
Note: Bias effects may be inconsistent and therefore require different

actions.
050207
7.32
Example 5: Chemical Batches
A key material characteristic

specification is 110 7.5.
is
Day's production
Operator
measured.
The
Sample
1
Sub-set
1 2 3
4 5 6
7 8 9
10 11 12 13 14 15
16 17 18 19 20 21 22 23 24
Meas.
1 2 3
4 5 6
7 8 9
10 11 12 13 14 15
16 17 18 19 20 21 22 23 24
Operator Sample
1
1
1
2
1
3
1
4
1
5
2
1
2
2
2
3
2
4
2
5
3
1
3
2
3
3
3
4
3
5
1
106.04
100.20
110.36
106.96
102.76
113.17
107.07
117.38
114.11
110.14
106.05
100.14
109.49
108.04
102.75
2
107.12
101.32
106.17
109.24
101.03
112.73
106.75
114.69
113.85
110.18
107.43
99.18
111.31
107.32
102.29
3
105.34
99.15
110.75
109.80
102.87
115.20
106.29
116.72
113.80
110.55
106.79
100.62
110.99
108.63
99.17
Range
1.78
2.17
4.58
2.84
1.84
2.47
0.78
2.69
0.31
0.41
1.38
1.44
1.82
1.31
3.58
5
43 44 45
43 44 45
Average
106.167
100.223
109.093
108.667
102.220
113.700
106.703
116.263
113.920
110.290
106.757
99.980
110.597
107.997
101.403
Assess the adequacy of the measurement process.

(Discrimination, precision, etc.)
050207
7.33
120
115
110
105
100
1
Operator
Sample within Operator
6
5
UCL=5.05
Range of Y
4
3
2
Avg=1.96
1
0
2
10
12
14
16
120
Mean of Y
115
110
UCL=109.60
Avg=107.60
LCL=105.59
105
100
2
050207
10
12
14
16
7.34
Formulas for :
R and X-bar
LCL
=D R
R
3
UCL
=D R
R
4
CL
Individuals and Moving Range

(Typically, the sample size=2)
UCL R = 3.267(M R )
" MR %
'
CL X = X 3$$
'
#1.128 &
=XA R
2
#R &
" = %% (( (If appropriate charts are in-control.)
$d 2 '
Sample Size
A2
D3
D4
d2
2
3
4
5
6
7
8
9
10
1.880
1.023
0.729
0.577
0.483
0.419
0.373
0.337
0.308
0.076
0.136
0.184
0.223
3.267
2.574
2.282
2.114
2.004
1.924
1.864
1.816
1.777
1.128
1.693
2.059
2.326
2.534
2.704
2.847
2.970
3.078
050207
7.35
Example: Candy Bars

Customer complaints:
1. Filling is coming out and sticking to the wrapper.
2. Taste varies from bar-to-bar.
Marketing interprets this as a candy bar weight problem.
Management does not want to ship over-weight bars, nor
do they want any destructive testing done.
What issues are there?
Project is to improve uniformity of candy bar weight.

Weights are measured both on-line and in the QC lab.
Design sampling strategies to address the problems.
Design sampling plans for

measurement systems (weight).
understanding
the
Select a plan, collect and analyze the data. What do you

recommend?
050207
7.36
Notes:
050207

Mse

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Mse

Enviado por

Direitos autorais:

Formatos disponíveis

Sigma Science: Measurement System Evaluation

Measurement System Evaluation

Evaluate the requirements and characteristics of

Understand the techniques useful in evaluating

Initiate an ongoing, data based system to

If you cannot measure, you cannot improve!

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

ISO 9001 Requirements

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Measurement is a process. The objective of the MSE is to

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Example 1: Material Batches

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Evaluate the information in the control charts. What

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Sampling Plan (Tree Diagram)

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Mapping the Measurement Process

Changes within titration

(NC) Sample on sides

(N) Pipetting method

(NC) Shake reagents

(C) Calculation used

In order to improve the measurement process, the focus

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Some Important Characteristics of a

Discrimination: Measure of the smallest unit of

Bias: Condition where the central tendency (e.g.,

Stability: The ability of the measurement process to

Applicability: Intended purpose for the measurement

For definitions see ISO 8402 and 5752

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Just as control charting identifies and quantifies sources of

!Product + !Measurement System

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Investigating the Measurement Process

In order perform a short-term investigation of the

Five samples are randomly selected from the line

Three operators are selected to measure the samples.

Each operator measures each sample three times.

The operators are instructed to use techniques and

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

2. Based on the dot frequency diagram shown below,

Is the within subgroup

Each Box describes the three

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Ross, Sanders & Cooper

Sigma Science: Measurement System Evaluation

Assessing the Discrimination of a

Range Charts (Subgroup Size =3)

Ross, Sanders & Cooper