Measurement Risk

and the Impact on Your Processes

Webinar hosted and presented by

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Agenda

 What is Measurement Risk?

 Where does Risk Creep into Your Process?
 Where does Risk Creep into the Calibration Process?
 Calibration Results: Impact on Your Process
 Question and Answer Session

Better by Your Every Measure • 800.828.1470 • www.transcat.com

What is Measurement Risk?
 Measurement Risk is the probability that measurements are
incorrectly influencing the acceptance decision on your product.
• This can impact your company in many ways:
– Acceptance of bad product, which can lead to negative results ranging
from consumer dissatisfaction to fatalities and lawsuits.
– Rejection of good product, which can lead to negative results ranging
from unnecessarily scrapped or reworked product to wasted resources
performing superfluous impact analyses.
 It is possible that some of your rework, scrap, recall or any
problems with released product are the result of unbridled
measurement risk.
 How can you determine if measurement risk is a problem in your
manufacturing process?
 Let’s refer to this in the positive term of Measurement Assurance.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Measurement Assurance encompasses a number of assumptions in your

manufacturing process:
1. Product Acceptance You made the right decision about the acceptance or
rejection of your product (or a component of your
product) during the manufacturing and quality
acceptance process.
2. Basis of Decision The decision was based (partially or completely) on
quantitative information provided by one or more
measuring instruments.
3. Suitability for You made the right choice in selecting instrument(s)
Intended Purpose that are appropriate for the measurements in your
process.
4. Instrument The instrument(s) were used correctly in the
Application manufacturing process.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Measurement Assurance encompasses a number of assumptions (con’t):

5. Beginning of The calibration of the instrument prior to your decision about
Period Reliability the product indicated it met its performance expectations.
6. End of Period The subsequent calibration of the instrument following your
Reliability decision about the product indicates that it continues to meet
its performance expectations.
7. Calibration Process Both calibrations were executed correctly.
8. Non-Conformance If a calibration indicates that the instrument failed to meet its
Review performance expectations, your quality system reviewed any
impact to the decision(s) made about your product(s) that were
based on the quantitative values presented by the instrument.
9. NCR Accuracy This impact study was thorough and did not miss the mark on
determining any detrimental affect on product.
10. Risk mitigation Any negative result from the impact study was properly
mitigated to remove or reduce product risk.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Measurement Assurance Reliability • If a value from 0.00 to 1.00 is

(Summary) assigned to each of these
Product Acceptance 1.00 factors, then a total reliability
figure can be calculated to
Basis of Decision 1.00 indicate the health status of
Suitability for Intended Purpose 1.00 your Measurement Assurance
program.
Instrument Application 1.00
• This is accomplished by
Beginning of Period Reliability 1.00 multiplying the assigned
End of Period Reliability 1.00 values.
Calibration Process 1.00 • If all factors have a reliability
of 1.00, then overall
Non-Conformance Review 1.00 Measurement Assurance has
NCR Accuracy 1.00 a reliability of 100%,
indicating that no
Risk Mitigation 1.00 Measurement Risk is present.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Risk Category Ex. 1 Ex. 2 Ex. 3

Reliability Reliability Reliability
Product Acceptance 1.00 1.00 1.00
Basis of Decision 1.00 1.00 1.00
Suitability for Intended Purpose 1.00 1.00 1.00
Instrument Application 0.95 1.00 1.00
B.O.P. Reliability 1.00 1.00 1.00
E.O.P. Reliability 1.00 1.00 1.00
Calibration Process 1.00 0.50 1.00
NCR Review 1.00 1.00 0.00
NCR Accuracy 0.90 1.00 0.00
Risk Mitigation 1.00 1.00 0.00
Measurement Risk 0.855 0.50 0.00

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Risk Category Ex. 1 Ex. 2 Ex. 3

Reliability Reliability Reliability
Product Acceptance 1.00 1.00 1.00
Basis of Decision 1.00 1.00 1.00
Suitability for Intended Purpose 1.00 1.00 1.00
Instrument Application 0.95 1.00 1.00
B.O.P. Reliability 1.00 1.00 1.00
E.O.P. Reliability 1.00 1.00 1.00
Calibration Process 1.00 0.50 1.00
NCR Review 1.00 1.00 0.00
NCR Accuracy 0.90 1.00 0.00
Risk Mitigation 1.00 1.00 0.00
Measurement Risk 0.855 0.50 0.00

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Risk Category Ex. 1 Ex. 2 Ex. 3

Reliability Reliability Reliability
Product Acceptance 1.00 1.00 1.00
Basis of Decision 1.00 1.00 1.00
Suitability for Intended Purpose 1.00 1.00 1.00
Instrument Application 0.95 1.00 1.00
B.O.P. Reliability 1.00 1.00 1.00
E.O.P. Reliability 1.00 1.00 1.00
Calibration Process 1.00 0.50 1.00
NCR Review 1.00 1.00 0.00
NCR Accuracy 0.90 1.00 0.00
Risk Mitigation 1.00 1.00 0.00
Measurement Risk 0.855 0.50 0.00

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Suitability for Intended Purpose (3) – This means that the

instrument selected:
• Measures the right parameter(s) (e.g., voltage, pressure,
temperature, length, et.al).
• Provides valid measurement results, meaning that the instrument is
accurate and stable enough to provide useful information.
• Can sustain the environmental conditions to which it must be
exposed while in use and still provide valid information to the user.
• Is user-friendly at a level commensurate with the
education/experience of the operator so that errors are not
introduced through improper use/application.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?
Instrument Application (4) – This means that the instrument is being used as
designed by the OEM.
• Incorrect use introduces errors into the measurement process, which will change
the decision of the outcome for the part/product.
SPRT Entering the wrong coefficients into the display, causing temperature
w/Indicator readings to be off (sometimes significantly).
Calipers Dropping the calipers, causing a nick or burr on the jaws to interfere
with the measurement.
Torque Wrench Operator error can introduce upwards of 10% error!
Dry Block Using the wrong size insert for the temperature probe; Not
Calibrator compensating for axial or radial uniformity errors; overloading the dry
block – all of these introduce errors into the perceived temperature
being generated by the dry block.
Oscilloscope Lack of understanding bandwidth constraints; using the instrument
above the frequency cutoff for bandwidth will provide low amplitude
readings, causing errors in your decisions about the product/test quality.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

Beginning of Period Reliability (5) – Some food for thought:

• Did you order the instrument and put it right into production without
having it calibrated first?
• If you relied on the “production cert”, this is not always a guarantee
that the instrument performs within the OEM’s stated performance
specifications; often this is simply a statement that the instrument
went through an operation check or production test that only
guarantees it is functional.
• Was the initial OEM cal ISO 17025 accredited? Often this is not the
case.
• For recalibration of the instrument, was the instrument returned to
you in a condition that is ‘close’ to being OOT? How long will it
maintain this condition during the current usage period?

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into Your Process?

End of Period Reliability (6) – This is the most critical

information that must be evaluated in order to close the loop
on your measurement assurance program:
• Was the instrument found to be OOT?
• Was the instrument found to be In Tolerance but close to being
OOT?
• What impact does the lab’s uncertainty have on this condition?
• How do you apply this information to the use of the instrument
during its last usage period?

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into the Calibration Process?

 What are the measurement parameters, or functions, on

the instrument you are submitting for calibration?
 Are there multiple ranges/values on the instrument?
 What is the accuracy of these ranges/values?
 Is the lab capable of performing calibrations for these
parameters/functions/ranges/values?
 How does their measurement uncertainty compare to the
tolerances of the instrument? Will they be able to provide a
valid result that is meaningful?

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Where does Risk Creep into the Calibration Process?

 What test points will they check on your instrument?

 What tolerances do they apply? Are they the same as the
ones stated by the OEM?
 If you have decided to use tolerances that are different than
those stated by the OEM, does your calibration provider
know this and apply them to your calibration?
 Is the calibration technician competent to perform the
calibration? How would you know?
 Is the calibration report easy or difficult to understand?

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Calibration Results: Impact on Your Process

NCR Review: Whether or not your quality system mandates

it, this step can save you money, can prevent safety
problems from occurring, and can prevent negative
marketing resulting from bad product being released.
• Again, this evaluation is often under-executed resulting in the false
perception that there is no impact to the product.
• Transcat offers consulting in this area to help your engineers and
quality personnel to understand how to get the most out of an
impact evaluation.
• How many OOT-NCR evaluations have you had in the past 5
years?

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Calibration Results: Impact on Your Process

NCR Review (continued):

• How many of these had a negative impact on your product?
• Answer one simple question: If the answer is zero or nearly zero,
then is this because you have such a well designed measurement
assurance program that it always prevents an impact to the
product?
• If it seems too good to be true, then you might consider the quality
of the evaluations being performed.
• Also, an external auditor might not believe you’ve designed a
system that is so good that you’ve never experienced a problem
with your measurements that affected product.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Risk Mitigation in Your Process

If you have discovered an OOT condition that may have

negatively influenced your decision about the
acceptance/rejection of product, consider this:
• If the OOT condition caused you to reject good product, then it is in
your best interest to determine how much this cost your company
in:
– scrapped product
– rework costs
– OOT-NCR evaluation costs

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Risk Mitigation in Your Process

If you have discovered an OOT condition that may have

negatively influenced your decision about the
acceptance/rejection of product, consider this:
• If the OOT condition caused you to accept bad product, then it is in
your best interest to determine how much this cost your company
in:
– consumer perception of a troublesome or faulty product
– liability suits for safety issues resulting in property damage,
medical costs, or fatalities
• Determine root cause to reduce or prevent these costs from
recurring.

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Measurement Assurance / Measurement Risk
 In the end, running your business is not only about international,
federal, customer, or internal requirements, policies, or procedures –
it’s about making a safe, reliable, superior product that fills a
need/desire in the market place and is profitable for your company.
 Measurement Assurance should be designed to help you minimize
risks in your decision making process about your product’s safety and
quality.
 Too often it is not fully implemented (all 10 categories), causing the
reliability of your Measurement Assurance Program to lose value and
become ineffective.
 If you’re going to put the effort and money into only some parts of this
program, it will likely cost you more elsewhere through rework, recall,
or consumer perceptions.
 Make your Measurement Assurance Program robust so that it works
for you to keep the cost and safety dogs at bay!

Better by Your Every Measure • 800.828.1470 • www.transcat.com

Thank You for Joining Us!

Questions?

800.828.1470
www.transcat.com