ISO/TS 16949:2009(E) and

AIAG MSA 4th edn. (2010)

Martin Gibson CStat, CSci, MSc, MBB

AQUIST Consulting
gg1000@waitrose.com

M G Gibson 2010

RSS Destructive Testing MSA

Making sense of MSA

Do you know how accurate and precise your measurement and

test equipment are?

Do you suspect that good work is sometimes condemned as bad
simply because of uncertainty in the measurement system; is
bad work ever released as good?
Do you know the cost of non-capable measurement systems?
Do you realise how important it is to understand measurement
systems uncertainty?
Does your auditor share your understanding of measurement
systems?
What can you do about it?

M G Gibson 2014

RSS Destructive Testing MSA

ISO/TS 16949:2009(E)
7.6.1 Measurement System Analysis
Statistical studies shall be conducted to analyse the variation
present in the results of each type of measuring and test
equipment system.
... applies to measurement systems in the control plan.
... analytical methods and acceptance criteria used shall
conform to those in customer reference manuals on MSA.
Other analytical methods and acceptance criteria may be used
if approved by the customer.
Questions:
What is the operational definition of statistical studies?
Do organisations, auditors, quality mangers understand statistical
studies?
Why do auditors ask?, Can you show me GR&R studies for each type of
measuring and test equipment system referenced in the control plan?
M G Gibson 2014

RSS Destructive Testing MSA

ISO/TS 16949 Scheme Update

IF SMMT Webinar, 5 Nov. 2013

Common problems found in ISO/TS16949 audits

Calibration and MSA (7.6 and 7.6.1)

Definition of Laboratory scope

Control of external laboratories
Traceability to national or international standards
MSA not done for all types of measuring systems
MSA only considering gauge R and R

Questions:
1. Why is MSA regarded as GR&R?

M G Gibson 2014

RSS Destructive Testing MSA

Ford Motor Company MSA requirements (2009)

4.35 (ISO/TS 16949 cl. 7.6.1)

All gauges used for checking Ford components/parts per

the control plan shall have a gauge R&R performed in
accordance with the appropriate methods described by
the latest AIAG MSA to determine measurement
capability.
Variable gauge studies should utilize 10 parts, 3 operators & 3
trials
Attribute gauge studies should utilize 50 parts, 3 operators & 3
trials
Questions:
1. Are some Customers leading the thinking?
2. Why just limited to products?
3. What are your Customer expectations?
M G Gibson 2014

RSS Destructive Testing MSA

Measurement System Variation

Observed Variation = Process Variation + Measurement System Variation

Bias
Accuracy
Linearity
Measurement
System
Variation

Calibration

Stability
Repeatability
Precision

Gauge R&R
Reproducibility

M G Gibson 2014

RSS Destructive Testing MSA

AIAG MSA 4th edn. (2010)

Accuracy, Bias, Stability, Linearity, Precision, Repeatability,
Reproducibility, GR&R
Attributes, Variables, & non-replicable data considered
Variables GR&R study
10 parts, 3 operators, 3 measurements

Parts chosen from 80% of tolerance

Destructive testing requires 90 parts from a homogeneous batch

Three analytical methods:

1.
2.
3.

Range basic analysis, no estimates of R&R

Average & Range provides estimates of R&R
ANOVA preferred, estimates of parts, appraisers, parts*operators
interaction, replication error due to gauge

Question:
1. Do organisations, auditors, quality mangers understand MSA?
M G Gibson 2014

RSS Destructive Testing MSA

AIAG ANOVA Models

Crossed vs. Nested
Y ijk = + Operator i + Part j + (Operator*Part) ij + k(ij)
Y ijk = + Operator i + Part j(i) + (ij)k

Crossed vs. Nested?

See Barrentine, Moen, Nolan & Provost, Bower, Burdick, Skrivanek

Fixed vs. mixed effects models?

Software?

MTB V16+ includes fixed, mixed effects, enhanced models, pooled

standard deviation approach not included.
SPC for Excel fixed effects
Other software packages?

Question:

Do organisations, auditors, quality mangers understand ANOVA?

M G Gibson 2014

RSS Destructive Testing MSA

GR&R Variables Data Acceptance Criteria

% Contribution
Measurement System Variation as a percentage of Total Observed
Process Variation using variances (additive)

% Study Variation
Measurement System Standard Deviation as a percentage of
Total observed process standard deviation (not additive)
% Tolerance
Measurement Error as a percentage of Tolerance
Number of Distinct Categories (ndc)
Measures the resolution of the scale

% Contribution

% Study Variation

ndc

% Tolerance

> 9% Unacceptable

> 30% Unacceptable

> 30% Unacceptable

< 5 Unacceptable

2-9% Acceptable

11-30% Acceptable

11-30% Acceptable

5-10 Acceptable

< 1% Good

< 10% Good

< 10% Good

> 10 Good

Do organisations, auditors, quality mangers understand the

metrics?
9

Non-replicable GR&R case study (Anon, 2002)

Ensure that all the conditions surrounding the
measurement testing atmosphere are:
defined, standardized and controlled
appraisers should be similarly qualified and trained
lighting should be adequate and consistently controlled
work instructions should be detailed and operationally

defined
environmental conditions should be controlled to an
adequate degree
equipment should be properly maintained and calibrated,
failure modes understood, etc.
10

Non-replicable GR&R case study (Anon, 2002)

If the overall process appears to be stable & capable, and all

the surrounding pre-requisites have been met, it may not
make sense to spend the effort to do a non-replicable study
since the overall capability includes measurement error if
the total product variation and location is OK, the
measurement system may be considered acceptable.
Ironically high Cp / Cpk gives poor ndc!
Question:
1. Do organisations, auditors, quality mangers understand this concept?
AIAG FAQs response:
If your process is stable and capable, the spread of this acceptable
process distribution includes your measurement error. There may be no
need to study your measurement error from a purely "acceptability"
viewpoint.
11

Questions for Making sense of MSA

What is the operational definition of statistical studies?
Do organisations, auditors, quality mangers understand

statistical studies?
Why do auditors ask for GR&R studies?
Why is MSA regarded as GR&R?
Are (some) Customers leading the thinking?
Why is MSA limited to products?
Do you know your Customer expectations?
Do organisations, auditors, quality mangers understand?

MSA, ANOVA, crossed vs. nested, fixed vs. mixed models, metrics, high
Cp/Cpk gives low ndc?

Is MSA seen just as a QMS requirement or a true part of

continuous improvement?
M G Gibson 2014

RSS Destructive Testing MSA

12

12

References
AIAG Measurement System Analysis, 4th edn., (2010)
Anon. Non-replicable GR&R case study, (circa 2002)
Barrentine, Concepts for R&R Studies, 2nd edn., ASQ, (2003)
Bower, A Comment on MSA with Destructive Testing, (2004) ; see also
keithbower.com
Gorman & Bower, Measurement Systems Analysis and Destructive Testing,
ASQ Six Sigma Forum Magazine, (August 2002, Vol. 1, No. 4)
Burdick, Borror & Montgomery, A review of methods for measurement
systems capability analysis; JQT, 35(4): 342-354, (2003)
Burdick, Borror & Montgomery, Design & Analysis of Gauge R&R Studies,
SIAM, ASA, (2005)
Moen, Nolan & Provost, using a Nested Design for quantifying a destructive
test in Improving Quality Through Planned Experimentation, McGraw-Hill;
1st edn., (1991)
Skrivanek, How to conduct an MSA when the part is destroyed during
measurement, moresteam.com/whitepapers/nested-gage-rr.pdf
13