National Association of Testing Authorities, Australia

National Association of Testing Authorities

NATA PROFICIENCY TESTING REQUIREMENTS

Edita Grujic
Proficiency Testing and Technical Projects Coordinator, NATA

THE USE OF (C)REFERENCE MATERIALS IN

TESTING/CALIBRATION AND THE ROLE OF PROFICIENCY
TESTING
Mohan Sabaratnam
Program Manager
Reference Material Producers, NATA
www.nata.com.au Be Absolutely Assured
National Association of Testing Authorities, Australia

What
NATA - Assures Technical Standards across government and private sectors.

Who
NATA is passionate about working for the public good, having a strong belief
that we can make a difference through our core skills in technical assurance

Why
NATA is the Authority that provides assurance of technical standards through a
proven network of best-practice industry experts, for client's who demand
certainty and confidence in the delivery of their products and services

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

NATA offers services in:

Accreditation

Assessment

Training

Advisory Services

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

NATA and Proficiency Testing

Update:

Proficiency Testing field/program policies now available from the

NATA website as Technical Circulars with the field/program specific
information (except where policies are already defined e.g. NPAAC
requirements for Medical testing)

The NATA Policy Circular #32: Proficiency Testing Scheme Providers

Implementation of ISO/IEC 17043: 2010 has been publicised recently

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

Proficiency Testing
On the international level, ILAC has issued an update to the document
P9: ILAC Policy for Participation in Proficiency Testing Activities

Effective immediately

Sets requirements for, and includes guidance to accreditation bodies

ILAC P9
removes the previous concept of major sub-disciplines
Introduces the need for facilities to have a PT Plan in accordance
with NATAs requirements for PT coverage

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

Proficiency Testing

Revised requirements will be reflected in the updated NATA PT Policy

NATA will review at assessment facilities PT plans

Other requirements e.g. such as expectations regarding action taken

in response to poor performance in PT, any requirements set by
regulators or similar bodies etc. will continue unchanged

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

QUESTIONS FOR DISCUSSION

Open for discussion

THANK YOU!

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

OVERVIEW OF PRESENTATION
Importance of good measurements
Use of CRM/RM in testing/calibration and proficiency testing (PT)
Common Ground
Practical approaches to CHARACTERISATION
CORE Requirements for Accreditation of RMP
ISO Guide 34 - Management & Technical Coverage
Establishing traceability in the laboratory
ISO/IEC 17025 clauses related to PT & use of CRM/RM
Possible combinations of Reference Material Producers
Unavailability of (C)RM & Selection of a (C)RM
NATA RMP Field Updates
Summary and Conclusions
Useful websites

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

IMPORTANCE OF GOOD MEASUREMENTS

Measurement results must be fit for their intended use in order to
achieve:

Sustainable development and economic growth;

Environmental integrity of the planet;
Societal equity.

Confidence in measurement results can result in:

Minimizing Technical Barriers to Trade (TBT);

Sensible decisions based on, for example, food, health and
environmental analyses;
Cost effective spending on measurement infrastructure by
governments.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

IMPORTANCE OF GOOD MEASUREMENTS

Method Calibration
Validation

Traceability/
QA/QC Measurement
Uncertainty

Valid measurement

All measurements must be underpinned by sound metrological

practices method validation, calibration, QA/QC and
Traceability/Measurement Uncertainty

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

ROLE OF PROFICIENCY TESTING PROGRAMS & USE OF

(C)REFERENCE MATERIAL
Proficiency Testing - primary aim
Provide a quality assurance tool for individual laboratories to enable them to
compare their performance with similar laboratories, to take any necessary
remedial action, and to facilitate improvement.
A demonstration of competence by a laboratory during an accreditation assessment in
accordance with ISO/IEC 17025.

(Certified) Reference Material - primary aim

Establish proper traceability to primary units of measure (eg. SI units) when
testing and/or calibration is performed by using (C)RM in:
Equipment/instrument calibration;
Validation of measurement procedures;
Quality control;
Establishing proficiency testing samples, where required.

Note: All the above are essential for metrological underpinning of measurement
results
www.nata.com.au Be Absolutely Assured
 National Association of Testing Authorities, Australia

COMMON GROUND FOR ESTABLISHING PROFICIENCY

TESTING PROGRAMS & PRODUCTION OF
(C)REFERENCE MATERIAL

COMMONALITIES BETWEEN THE TWO PRODUCTION PROCESSES:

Sample stability;
Sample homogeneity;
Statistical analysis of measurement data;
Establishing and Reporting associated measurement uncertainty;
Use of SUBCONTRACTORS to establish assigned property;
A requirement for gaining and maintaining third party accreditation (ie.
NATA)

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

PRACTICAL APPROACHES TO CHARACTERISATION & PROPERTY

(VALUE) ASSIGNMENT ACCORDING TO ISO GUIDE 34 & 35

ISO Guide 34 & 35 recommends possible approaches for characterising a

candidate reference material:

Measurement by a single (primary) method in a single laboratory;

Measurement by two or more independent reference method in one laboratory;

Measurement by a network of laboratories using one or more methods of

demonstrable accuracy;

A method-specific approach giving only method-specific assessed property

values, using a network of laboratories.

Note: Reference materials established by the above modes have been known to
be used as proficiency testing samples.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

ACCREDITATION OF REFERENCE MATERIAL PRODUCERS (RMPs)

The core requirements documents are:

ISO Guide 34 (2009) Requirements for the Competence of Reference
Material Producers.
ISO Guide 31 (2000) Reference materials Content of certificates and
labels.
ISO/IEC 17025 (2005);
NATA RMP Field Application Document

Other related documents include:-

REMCO ISO Guides 30, 32, 33, 35 and 80 (in preparation)
APLAC TC 008-2010 Guidelines on the Approach to the
Assessment of Reference Material Producers and the Resulting Scope
of Accreditation
ILAC - G9 (2005) The Selection and Use of Reference Material
ILAC - G12 (2000) Guidelines for the Requirements for the
Competence of Reference Material Producers
www.nata.com.au Be Absolutely Assured
National Association of Testing Authorities, Australia

MANAGEMENT & TECHNICAL COVERAGE

OF (CERTIFIED) REFERENCE MATERIAL PRODUCTION

ISO Guide 34

ISO/IEC 17025
ISO 9000

ISO Guide 34
includes application of
ISO Guide 30, 31 & 35

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

ESTABLISHING (FIT FOR PURPOSE) TRACEABILITY IN THE

LABORATORY
Laboratories should be aware that in most cases, all stages of the
testing/calibration process shall need validation, and traceability to be
established.

Reference materials need to be used on an ongoing basis to

ensure integrity in the testing and calibration is maintained.

Laboratories are required to investigate the availability and

suitability of reference materials and calibrated artefacts for their
accredited procedures.

Laboratories are required to determine if the calibrated artefacts or

reference materials used in their methodologies provide a level of
traceability that is fit for purpose.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

ESTABLISHING (FIT FOR PURPOSE) TRACEABILITY IN THE

LABORATORY

Thus, competence of RMP organisations conducting the production or

calibration of reference materials/artefacts is a critical step towards obtaining
suitable well designed reference materials or artefacts.

Note 1: Accreditation under NATA requires all accredited testing and calibration laboratories to
comply with Clause 5.6, ISO/IEC 17025:2005. It is acceptable that not always
metrological realisation is achievable through established lines of traceability.

Note 2: The concept of traceability of measurement in fields such as chemical and biological
sciences is still under international debate so no clear pathway has been accepted.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

ISO/IEC 17025 (2005) CLAUSES RELATING TO

(C)REFERENCE MATERIALS
ISO/IEC 17025, clause 4.5 requires laboratories to evaluate suppliers; this
includes producers of reference materials and suppliers of calibration
services/items.

Materials purchased by the laboratory is determined by the quality of the

producers of these items and the distributor is only part of the supply chain
and in most cases only has custody of the material until it is delivered to the
user. This is reflected in ISO/IEC 17025 under clause 4.6 Purchasing services
and supplies

4.6.2 The laboratory shall ensure that purchased suppies and reagents and
consumable materials that affect the quality of test and/or calibrations are not
used until they have been inspected or otherwise verified as compying with
standard specifications or requirements defined in the methods for the tests
and/or calibrations concerned. These services and supplies used shall comply
with specified requirements. Records of action taken to check compliance shall
be maintained.

4.6.4 The laboratory shall evaluate suppliers of critical consumables, supplies and services
which affect the quality of testing and calibration, and shall maintain records of these
evaluations and list those approved.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

ISO/IEC 17025 (2005) CLAUSES RELATING TO

PROFICIENCY TESTING
5.9 Assuring the quality of test and calibration results
5.9.1 The laboratory shall have quality control procedures for monitoring the
validity of tests and calibration undertaken. The resulting data shall be
recorded in such a way that trends are detectable and, where practicable,
statistical techniques shall be applied to the reviewing of the results. The
monitoring shall be planned and reviewed and may include, but not limited, the
following:
(a) regular use of certified reference materials and/or internal quality control using secondary reference
materials;
(b) participation in interlaboratory comparision or proficiency-testing programmes;
(c) replicate tests or calibrations using the same or different methods;
(d) retesting or recalibration of retained items;
(e) correlation of results for different characteristics of an item;

NOTE: The selected methods should be appropriate for the type and volume of the
work undertaken.

5.9.2 Quality control data shall be analysed and, where they are found to be
outside pre-defined criteria, planned action shall be taken to correct the
problem and to prevent incorrect results from being reported.

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

POSSIBLE COMBINATIONS OF REFERENCE MATERIAL PRODUCERS (RMPs)

Note: The Reference Material Producer (ie The body which is subject to formal accreditation) is the organisation that takes
responsibility and authorises the values on the (C)RM certificate - not the actual material/product itself.
Stages and tasks of (Certified) Reference Material ISO Requirements = Tasks actually performed by each organisation
[(C)RM)] Production Document(s) Org 1-5 etc = Examples only

Org 1 Org 2 Org 3 Org 4 Org 5 etc

Material preparation ISO Guide 34 EC EC EC EC

Homogeneity & Stability assessment ISO Guide 34 EC EC EC

Characterization of Property values ISO Guide 34 EC EC EC

+ISO/IEC 17025

* Authorisation and Assignment of values on ISO Guide 34

(C)RM and issue of certificates +ISO/IEC 17025

Supply & distribution ISO Guide 34 EC EC EC EC

The RMP accreditation process assesses the competence for each task.
The RMP itself may perform one or several tasks, as shown above, but must have a quality system that covers all of the tasks
involved and the necessary expertise to competently evaluate all the inputs.

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

WHERE NO REFERENCE MATERIALS EXIST?

Then
The laboratory shall demonstrate that the alternatives used have:
Sufficient traceability;
Stability;
Homogeneity,
and Accuracy, such that the method and subsequent results can be deemed fit for purpose.

Mandatory use of a specified reference material may be a requirement for some sector schemes or in
support of regulatory activities.

There is overlap between ISO/IEC 17025 CALIBRATION accreditation and ISO Guide 34 Reference
Material Producer accreditation, in that organisations accredited to these standards can both provide
materials that can be used as a reference and to provide traceability for analytical methods.

However, it has been accepted that ISO/IEC 17025 is primarily used for characterisation of candidate
reference material whilst ISO Guide 34 encompasses both the characterisation and associated
production requirements for the production of a reference material and/or certified reference material.

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

CHOOSING A REFERENCE MATERIAL?

Testing/Calibration laboratories should consider the following points when selecting the
service/product they require:

Where the metrological properties of an artefact or instrument have a significant

effect on the result of a measurement then calibration is required. This must be
performed by a competent laboratory.
Some examples of materials that fall into this category are:
Masses used to calibrate balances, Thermometers used for monitoring the temperature of ovens,
incubators, Glass beads used for checking sieve pores, Alloy standards for conducting arc-spark
tests

Where the laboratory requires a homogenous and stable artefact or if the

artefact has the potential for heterogeneity and/or instability, then the testing or
calibration laboratory should purchase an artefact from a reference material
producer that can demonstrate that these requirements have been met. These
material characteristics can also be ascertained by a competent laboratory.
Some examples of materials that fall into this category are:
Certified reference cement material, Fatty acid reference material used for GC analysis, A solution
of anions or cations used to calibrate an ICP-MS, Soil with certified dioxin content used for the
validation of a dioxin method, Reference microorganisms cultures for identify and purity
characterisation

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

NATA RMP FIELD UPDATES

Accreditation granted to SEVENTEEN producers in the areas of organic reference

materials used in sports drugs, forensic, therapeutic, reference gases, ethanol
standards, grain (wheat & barley) and biological reference cultures;

Currently, NATA is the largest RMP accreditation body in the world;

APLAC RMP Technical Document (TC 008) completed. NATA had considerable
input;

APLAC RMP MRA signed in December 2007; Therefore all accredited NATA
RMP facilities can use the World Recognised ACCREDITATION logo

Conversion of accredited RMP facilities from ISO Guide 34 (2000)

to ISO Guide 34 (2009) is underway.

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

SUMMARY (1)
1. It is essential that (accredited) labs have access to good quality (C)RMs (which are fit
for their interned use).

2. There is process OVERLAPP with the production of reference materials and in

developing proficiency testing samples however the primary aim of both process is to
be a performance tool in one aspect and establish metrological traceability in another.

3. APLAC has established a MRA for this service (5 signatories NATA, Australia; CNAS,
China; IAJapan, Japan; ACLASS, USA; A2LA, USA.

4. Given the restricted number of RMPs in Australia, reference materials purchased from:
RMPs accredited by NATA;
RMPs accredited by signatories of the APLAC RMP MRA;
National Measurement Institute Australia;
other NMIs who are part of the CIPM MRA

are preferred, where available.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

SUMMARY (2)
5. The accreditation of RMPs and PT Providers is gaining further
momentum and it is hoped that both activities will be included under
the ILAC Arrangement in the near future. The laboratory community
and accreditation bodies are working closely in finalising this work.

6. These activities will provide further confidence in laboratory outputs

(i.e. measurement results) and thus provide socio-economic benefits to
the community at large.

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

USEFUL WEB-SITES
(REFERENCE MATERIALS)

Whilst no single source of information on the availability of reference materials currently exists,
laboratories are recommended to contact reference material producers and distributors to discuss
availability and suitability.

In addition, laboratories may find it useful to refer to the COMAR database for availability of reference
materials. COMAR is a non-commercial network of national and international organisations, which is
funded by BAM. The website address is www.comar.bam.de.

Other sources of information are:

NATA www.nata.com.au REMCO: www.iso.org/remco
APLAC www.aplac.org Springer (ACQUAL): http://link.springer.de
ILAC www.ilac.org Euromet: www.euromet.org
BIPM www.bipm.fr Eurolab: www.eurolab.org
CITAC www.citac.cc Eurachem: www.eurachem.org
EA www.european-accreditation.org

Other National Accreditation Body websites

www.nata.com.au Be Absolutely Assured

 National Association of Testing Authorities, Australia

THANK YOU FOR YOUR KIND ATTENTION

The scientist of today think deeply instead of clearly. One must be sane to think clearly, but
one can think deeply and be quite insane.

-Nicola Tesla-

Wikipedia

The International System of Units (SI) measuring magnetic field B (also referred to as the magnetic flux density and
magnetic induction) is the TESLA, was named in his honour at the Confrence Gnrale des Poids et Mesures, Paris,
1960.

www.nata.com.au Be Absolutely Assured

National Association of Testing Authorities, Australia

QUESTIONS FOR DISCUSSION

1) What has been your experiences in getting appropriate CRMs?

2) Have you tried using a PT sample as a reference material?

3) How do you deal with CRMs that are in a different matrix to that of the
sample?

4) In general, your experiences with NATA assessments and the composition

of assessment teams?

5) How do you select your CRM producer?

www.nata.com.au Be Absolutely Assured