REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING LABORATORIES

Jukka Hyrynen Heikki Kallio Anna-Maija Kosonen

NORDTEST
Tasks The tasks of Nordtest are to promote the safety of life, health, environment and material values and to encourage a free exchange to trade. The approach adopted by Nordtest to achieve its objectives is: to develop, adopt and recommend test methods and to promote the use of these by industry and the authorities and also in the standardisation work to obtain international recognition of test results and also the competence of the Nordic countries, for instance by quality assurance and verification of testing activity to endeavour that tests and approval of test results are made in a resource and cost effective manner to promote the technical testing infrastructure in the Nordic countries by means of research, development of competence and collaboratian, and to participate in the European and international development of testing and to promote Nordic interests,

Organisation The organisation consists of a board, a secretariat and nine technical groups. These groups are Acoustics and Noise, Building, Electronics, Environment, Fire, Mechanics, Polymers, VVS (Mechanical Building Services) and Quality Assurance. The work is directed by the board which comprises representatives of all the Nordic countries, The members are appointed by the government or appropriate department of the country concerned. The technical groups initiate and evaluate projects. The projects are often structured in such a way that they can be used as catalysist for development of the combined technical competence in the Nordic countries. At present, about 250 Nordic projects are being carried out in some 40 firms and institutions. The board as well as the technicai groups are assisted by the secretariat which is responsible for day to day activity. The secretariat is located at Esbo, Finland. Financial framework The cost of the Nordtest secretariat and a Iarge proportion of project activity is financed from the budget of the Nordic Council of Ministers. The grant for 1992 is approx. 2 millions ECU, The work of the board and the technical groups is financed by the participating organisations. Publications Register of 1300 test methods and technical reports Test methods Technical reports

NT TECHN REPORT 177 Approved 1992-04

Authors: Jukka Hyrynen Heikki Kallio Anna-Maija Kosonen Title (English): Title (Original): Abstract:

NORDTEST project number: 956-90-l Institution: VTT

REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING LABORATORIES

This paper is the final report of NORDTEST Project 956-90-l Referensmaterial vid kvalitetsskring av provningslaboratorier (Reference materials in quality assurance of testing laboratories). The scope of this project was to collect information for the users of reference materials in the Nordic countries and to propose recommendations for the use of reference materials

Technical Group: Quality Assurance ISSN: 0283-7234 Class (UDC): 620.1 Distributed by: NORDTEST Tekniikantie 12 FIN-02150 Espoo Finland Language: English Pages: 35

Key words: reference materials, quality assurance, testing laboratories Publication code:

CONTENTS 1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Certification of reference materials . . . . . . . . . . . . . . . . . . . . . . 1.3 Organizations within the CRM schemes . . . . . . . . . . . . . . . . . . 3 4 7 7

2 REFERENCES IN QUALITY AND TESTING STANDARDS . . . . . . . . . . . . . . 8 2.1 EN45001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 ISO/IEC Guide 25 : 1990(E) . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 IS0 9001 : 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 NAMAS accreditation standard Ml0 . . . . . . . . . . . . . . . . . . . . . 2.4.1 NAMAS information sheets . . . . . . . . . . . . . . . . 2.5 Testing standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Hardness test standards . . . . . . . . . . . . . . . . . . . 2.5.2 Impact test standards . . . . . . . . . . . . . . . . . . . . . 3 CHOOSING AND MAINTAINING REFERENCE MATERIALS . . . . . . . . . . . . 3.1 Choosing a reference material . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Working standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Household practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING LABORATORIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Indirect verification of test instruments . . . . . . . . . . . . . . . . . . . 4.1.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Controlling laboratory performance . . . . . . . . . . . . . . . . . . . . . . 4.2.1 X-R charts . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Value-time charts . . . . . . . . . . . . . . . . . . . . . . . 4.3 Using the control charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Interlaboratory tests in estimating laboratory performance and the uncertainty of testing . . . . . . . . . . . . . . . . . . . 9 10 10 11 11 12 13 13 14 15 15 16 17 17 18 19 20 21

5 SOME SOURCES OF CERTIFIED MATERIALS . . . . . . . . . . . . . . . . . . . . . 22 5.1 BCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5.2 NIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6 COMAR DATABASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS . . . . . . . . . . . 24 8 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 APPENDIX 1 Excerpts from BCR REFERENCE MATERIALS CATALOG APPENDIX 2 Example of a certificate of measurement APPENDIX 3 Producers of RMs and CRMs 2p. 2p. 1p.

-3-

1 INTRODUCTION High-quality quantitative information on the properties of materials is needed for the safe, efficient and reliable design and use of modern industrial products. Confidence in the materials and the material properties is being enhanced through standardized test methods and verified performance of laboratories; formal accreditation is chosen as an accepted path in Europe towards harmonized practices and results. Accredited laboratories are expected to perform tests consistently from day to day; furthermore a test carried out at different accredited laboratories according to a standardized method should produce identical results. Reference materials (RMs) are used in various fields of testing to provide users with accurate reference values and traceability for their measurements, to quantify and to calibrate methods and equipment and to monitor the performance of the laboratory. These tests can be complex, or the result is qualitative or is not readily traceable to national or international standards. A reference material is by definition a material or substance one or more properties of which are sufficiently well established to be used for calibration of an apparatus, assessment of a measurement method, or for assigning values to materials1. RMs are widely used within the field of analytical chemistry, where they form the basis for ensuring accuracy and for adhering to valid measurement systems. The use of reference materials is gaining momentum in other fields of testing, too. Certified reference materials (CRMs) are reference materials whose property values are certified by a technically valid procedure, accompanied by or traceable to a certificate or other documentation which is issued by a certifying body1. In chemical laboratories CRMs correspond to the reference standards of physical measurements. The use of CRMs has been encouraged and emphasized by the

definition, ISO GUIDE 35

-4-

calibration and traceability requirements in the international standards also for accreditation of testing laboratories.

1.1 Definitions There are a number of terms often used when dealing with quality assurance of testing as well as with calibration of equipment and instruments. The definitions below are collected from various standards to clarify further discussions in the paper. REFERENCE MATERIAL & CERTIFIED REFERENCE MATERIAL * see above WORKING STANDARD * Reasonably homogeneous, previously analyzed materials, pure compounds, solutions of pure elements etc. used instead of CRMs for frequent checks and in situations when cost or availability limit the use of CRMs.2 STANDARD REFERENCE MATERIAL (NIST Standard Reference Materials Catalogue 1990-91, NIST Special Publication 260) * NIST Standard Reference Material (SRM) is a certified reference material issued by NIST.

Distinction between RMs and working standards is somewhat vague; one way could be: RM is commercially available whereas a working standard is manufactured in-house.

-5-

REFERENCE STANDARD OF MEASUREMENT (NAMAS ACCREDIT ATION STANDARD M10) * Instrument or gauge used to calibrate measuring and test equipment. The term also applies to a material used as a reference standard. Note: The term standard reference material is also used in NAMAS standards for a certified reference material. CERTIFIED VALUE (ISO-GUIDE 30) * For a CRM, the value that appears in the certificate or other documentation accompanying the material, this value having been certified by a technically valid procedure. CONSENSUS VALUE (ISO-GUIDE 30) * For a reference material, the value of the quantity obtained by interlaboratory testing, or by agreement between appropriate bodies or experts. CALIBRATION (ISO GUIDE 25 : 1990) * The set of operations which establish, under specified conditions, the relationship between values indicated by a measuring instrument or measuring system, or values represented by a material measure, and the corresponding known values of a measurand. VERIFICATION (ISO GUIDE 25 : 1990) * Confirmation by examination and provision of evidence that specified requirements have been met. * NOTE In connection with the management of measuring equipment, verification provides a means for checking that the deviations between values indicated by a mea-

-6-

suring instrument and corresponding known values of a measured quantity are consistently smaller than the maximum allowable error defined in a standard, regulation or specification peculiar to the management of the measuring equipment. * The result of verification leads to a decision either to restore to service, or to perform adjustments, or to repair, or to downgrade, or to declare obsolete. In all cases it is required that a written trace of the verification performed be kept on the measuring instruments individual record. DIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN standards) * Method for checking the main functions of the machine. * NOTE This method shall be used: When the machine is installed or removed or repaired. When the result of the indirect verification is not satisfactory. INDIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN standards) * Method suitable for overall checking of the machine. The indirect verification method may be used on its own for periodic routine checking of the machine in service. * The indirect verification may be carried out by means of certified reference materials (standardized, calibrated blocks). VERIFICATION OF A REFERENCE MATERIAL (ISO standards) * All the operations carried out, normally by a technically competent third party, with the object of confirming the validity of the value(s) stated in a reference material certificate.

-7-

CALIBRATION OF STANDARDIZED BLOCKS (EN standards) * A specified method to be used to gain required traceability for the reference material used for the indirect verification of the testing machines.

1.2 Certification of reference materials Reference materials can be gaseous, liquid or solid. Certification of CRMs can be done according to three approaches: DEFINITIVE METHOD, certification is done by one highly qualified laboratory using the best possible methods to reach very high precision and very low systematic error. INDEPENDENT METHOD, usually done by one laboratory using two or more reliable independent methods of measurement, each of which has been shown to yield good results. The methods produce results which overlap and all results by all the methods must fall within allowable end-user uncertainty. INTERLABORATORY METHOD, a number (5 - 20) of laboratories carry out an interlaboratory test programme, and a statistical analysis of the results is carried out. The programmes often consist of two or more rounds of measurements to cut down the uncertainties due to different methods of handling, measurement and analysis of the samples.

1.3 Organizations within the CRM schemes In Europe the Community Bureau of Reference (BCR) is the largest establishment dealing with certified reference materials (see appendix 1). There are also other

-8-

establishments with more limited fields of interest. In the US the National Institute of Standards & Technology is federally responsible for the CRMs. The ISO Committee on reference materials (REMCO) is harmonizing the work on reference materials by organizing relevant research projects and by publishing ISO GUIDES on the subject. REMCO has prepared an international database on reference materials (COMAR, see chapter 6).

2 REFERENCES IN QUALITY AND TESTING STANDARDS The quality-related standards either recommend or require the use of reference materials to increase the reliability of testing activity. The following excerpts from a number of standards show the level of requirements.

2.1 EN 45001 The EN 45001 standard General criteria for the operation of testing laboratories describing the requirements for the accreditation of laboratories refers to reference materials as follows: * Reference materials shall where possible be traceable to national or international standard reference materials. (5.3.3. Equipment) * The Quality Manual shall contain at least: ... f) where appropriate, reference to proficiency testing, use of reference materials, etc.; (5.4.2. Quality System)

-9-

2.2 ISO/IEC Guide 25 : 1990(E) The ISO/IEC Guide 25 General requirements for the competence of calibration and testing laboratories describes the activities of both calibration and testing laboratories as follows: * The quality manual, and related quality documentation, shall state the laboratorys policies and operational procedures established in order to meet the requirements of this Guide, the quality manual and related quality documentation shall also contain ... n) reference to verification practices including interlaboratory comparisons, proficiency testing programmes, use of reference materials and internal quality control schemes. (5.2, Quality system, audit and review) * Reference materials shall, where possible, be traceable to national or international standards of measurements, or to national or international standard reference materials. (9.7, Measurement traceability and calibration)

2.3 ISO 9001 : 1987 The ISO 9001 standard Quality systems. Model for quality assurance in design/ development, production, installation and servicing describes the requirements for inspection, measuring and test equipment as follows: * The supplier shall control, calibrate and maintain inspection, measuring and test equipment, whether owned by the supplier, on loan, or provided by the purchaser, to demonstrate the conformance of product to the specified requirements. Equipment shall be used in a manner which

-10-

ensures that measurement uncertainty is known and is consistent with the required measurement capability. (4.11)

2.4 NAMAS accreditation standard Ml0 The NAMAS accreditation standard General Criteria of Competence for Calibration and Testing Laboratories describes the requirements for measurement traceability and calibration as follows: * Where the concept of traceability is not in practice, the Laboratory shall provide satisfactory evidence of correlation of calibration or test procedures, for example by participation in a suitable program of measurement audit or inter-laboratory comparison or through the regular use of standard reference materials. (7.5) * Where the reference standard of measurement is a reference material the Laboratory shall, wherever possible, obtain a certificate that provides evidence of the characterization of the material and evidence of traceability to national or international standards of measurement, or national or international reference materials. (7.7)

2.4.1 NAMAS information sheets NAMAS has published various information sheets as supportive media for laboratories applying for or maintaining accreditation. These information sheets give requirements, for example, for test performance, machine verification, use of reference materials and also give sources of accredited verification and test blocks. (NAMAS Information Sheets NIS 13 TRACEABILITY, HARDNESS MEASUREMENTS)

-11-

2.5 Testing standards Many testing standards referring to calibration or traceability do require reference materials either as primary or secondary means of traceability. Examples of such methods are measurement of surface hardness and impact testing of metallic materials. These are chosen below as examples of test methods. ISO or EN standards on mechanical testing usually have separate standards for the verification of testing machines. For example hardness and impact testing machines shall or may be verified by means of standardized blocks (=RMs). This method is so-called indirect verification (see definitions).

2.5.1 Hardness test standards ISO/EN New EN standards will incorporate corresponding ISO standards and replace the existing European national standards. These standards contain both direct and indirect verification methods of testing machines. There are also separate standards for calibration of standardized blocks to be used for indirect verification of hardness testing machines. Direct verification (see definitions) of testing machines involves -verification of test force, -verification of the indenter and -verification of the measuring device. The hardness values given by the testing machine do not depend only on the dimensions of the indenter, but also on the surface roughness and the seating of the indenter. For this reason an indirect verification using RMs is considered necessary. The indirect verification can be performed with two procedures. According to the first procedure each hardness scale (Rockwell), different

-12-

hardness ranges and test forces or ball sizes (Brinell) will be verified periodically. The other possibility is to verify prior to the test only one hardness value that corresponds approximately to that of the tests to be performed. ASTM In ASTM standards instructions for a specific hardness testing method, verification of machine and calibration of hardness blocks are in one standard. According to ASTM standards the direct verification shall be used for new and rebuilt machines. The indirect method using standardized test blocks may be used in routine testing to assure the operator that a machine is operating properly.

2.5.2 Impact test standards According to prEN 10 045 Part 2, the main method for verification of impact test machines is the indirect verification using the Charpy V reference test pieces, whose breaking energies are known. The dimensional requirements are more accurate than the requirement for standard test specimens. The test pieces shall be national test pieces traceable to the BCR test pieces or certified BCR test pieces. The direct method shall be used, when the machine is being installed or repaired or if the indirect method gives an incorrect or unacceptable result. Contrary to the EN standard, the main method in the ISO 442 standard is the direct verification. The indirect method is not even an alternative to complete direct verification. The reference test pieces are unnotched and their thickness is smaller than the standard specimens. The same material shall be tested in parallel in a reference machine. According to the ASTM E 23-88 standard, Charpy impact machines shall be verified by testing with standardized specimens. If these produce values outside the tolerances, those parts subjected to wear shall be inspected and if necessary repaired or replaced. ASTM E 127l-88 gives very detailed instructions for manu-

-13-

facturing and qualifying Charpy verification specimens. Certified reference material is delivered by the National Institute of Standards and Technology. It must be noted that this material can be used only in machines made according to ASTM E 23. Certain dimensions in ISO and EN standards differ from those in ASTM.

3 CHOOSING AND MAINTAINING REFERENCE MATERIALS The ISO-GUIDE 35 Certification of reference materials - General and statistical principles describes the requirements for establishing certified reference materials. Much importance is given to confirming homogeneity of the materials and to the statistical methods to define the uncertainties. Certification can be established by a definite method, by interlaboratory testing or through a metrological approach. The ISO-GUIDE 31 Contents of certificates of reference materials describes the requirements for the certificates. There are unfortunately a number of commercially available reference materials whose certificates do not clearly state the level of confidence, level of certification or the uncertainties. The users of CRMs have to seek confirmation on the quality of reference materials before purchase.

3.1 Choosing a reference material The user of a reference material must decide what properties are relevant to the measurement process. The level (of property value) of the CRM should correspond with the level of the intended measurement. The matrix should be close to the matrix of the material to be subjected to the measurement process (carbon in low-alloy steel or carbon in stainless steel). The form should correspond with that of the intended samples (solid, liquid, gas...). The quantity of the CRM should be

-14-

sufficient; the need to obtain additional CRM during the experimental programme should be avoided and batches of CRMs should partly overlap in use for continuity. Stability of the CRMs should be ensured or changing of the properties should be recorded in the certificate. The uncertainty of the certified value should be acceptable when compared to requirements (legal limits, trade agreements and commercial requirements, previous experience in the laboratory). Appendix 4 shows an example of a BCR reference material.

3.2 Working standards Often the availability of certified reference material is limited, the use of the reference material is extensive or the price of the certified reference material is high. In these cases the commercial certified reference materials cannot be used in everyday practice by the laboratories. The laboratories can and are also encouraged to produce in-house working standards. Homogeneity of these working standard reference materials should be verified with sufficient accuracy through in-house test series; the level of the parameters of interest have to be calibrated against valid certified reference materials. Uncertainty (95% confidence limits, for example) should be estimated for the working standard. Generally a well-prepared and evaluated working standard with traceability to CRMs is of higher value to the quality assurance of a testing laboratory than a commercial reference material with limited confidence and questionable certificate.

-15-

3.3 Household practices Reference materials have to be treated as highly vulnerable instruments. The CRMs have to be stored in a manner which will insure that none of the properties of the CRMs alter more than the normal deterioration stated in the certificates, if evident. Records on all uses of CRMs should be kept and all unqualified uses of CRMs have to be discouraged. Clear procedures should be maintained on how each reference material is to be used and what records kept. CRMs, RMs or working standards should be used in a continual manner to ensure quality and traceability of testing in a laboratory as described in the following chapter.

4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING LABORATORIES Two major uses of reference materials in a testing laboratory can be identified. One use is the indirect verification and calibration of test instruments, while the other is the continual monitoring of laboratory performance.3

The NIST catalogue gives the following reasons for use of reference materials: * To help develop accurate methods of analysis (reference methods); * To calibrate measurement systems used to: (a) facilitate exchange of goods, (b) institute quality control, (c) determine performance characteristics, or (d) measure a property at the state-of-the-art limit; and * To assure the long-term adequacy and integrity of measurement quality assurance programmes.

-16-

4.1 Indirect verification of test instruments As mentioned above, verification of an instrument is understood as confirmation by examination and provision of evidence that the instrument meets the specified requirements set for it. Verification provides a means for checking that the deviations between values indicated by a measuring instrument and the corresponding known values of a measured quantity are consistently smaller than the maximum allowable error defined in a standard, regulation or specification characteristic of the measuring equipment. Indirect verification usually consists of an overall checking of the working conditions of the test instrument as well as carrying out the normal test procedures using samples of known level(s) of measured parameter(s). The results from the tests are compared with the values given for the verification samples and conclusions are drawn based on the deviations (differences of test results and corresponding true value) and the requirements given for the instrument. The result of verification leads to a decision either to restore to service, or to perform adjustments, or to repair, or to downgrade, or to declare obsolete. In all, cases it is required that a written trace of the verification performed shall be kept on the measuring instruments individual record. Often the standards that define indirect verification of test instruments require the use of certified reference materials (CRMs) as samples with known levels of the parameters of interest. A reduced level of reliability can be allowed for the reference material; this depends on the requirements set for the instruments, on the level of uncertainty of the certified reference material (traceable, verified, calibrated material) and on the performance of the instrument. The laboratory can manufacture its own working standard reference materials or use reference materials without a certificate as long as the laboratory has verified and documented the production and the

-17-

traceability of the working standard. ISO-GUIDE 33 encourages laboratories to study possibilities to supplement CRMs with working standards when the required CRMs have a limited availability and relatively high cost.

4.1.1 Calibration Calibration is often (but not necessarily) an integral part of verification of a test instrument. Calibration defines the relationship between values indicated by a measuring instrument or measuring system, or values represented by a material measure, and the corresponding known values of a measurand. The known value here can be given through CRM or RM as above and through instruments whose calibration is traceable to national and international standards.

4.2 Controlling laboratory performance Controlling the laboratory performance should be understood as a philosophy, a system and a set of techniques for controlling and improving testing processes. In a sense the process is related to the statistical process control (SPC) (Juran 1974) generally used in the production and service industries. In SPC the normal production is evaluated in an organized manner by measuring critical attributes of the production afterwards. Much of the testing is destructive or testing affects the attributes measured; often adjustment, verification or calibration of the instruments with reference material (or instruments) is required prior to testing. The possibility to re-test the samples is thus very unlikely and valuable information is received on performance of the test process by recording the measurements (of known values of attributes) done prior to the test.

-18-

The process described above is very open and has to be included in the normal testing procedures. Another way to receive material for performance control is to include blind samples in the testing process. The open process has the possibility of biased or specially good practice in testing as the CRMs are naturally identified; blind testing can be difficult to organize and in time samples of known values of attributes are identified by the personnel. Often it is advisable to incorporate not only testing but also sample preparation in the blind testing programmes. An integral part of successful control is good planning of the sampling and testing plan along with the number of tests, levels and intervals. Control charts measure the process over time. There are many ways to draw up these charts; below two choices are described.

4.2.1 X - R charts The conventional SPC control chart is constructed as a pair of graphs showing both mean and range of measurements. Sample size for SPC charts is usually between 2 and 10. The means and control limits (three standard deviations) are calculated (Fig 1). The next step is to determine whether the process is under control or not. A simple test sets conditions for out-of control process as follows: * any points above the upper control limit or below the lower control limit in either graph or * eight points in a row above (or below) the centreline in either graph. This test tells only if the process is under control, not if the measurements are within required tolerances. The tolerance limits have to be added to the graphs to justify evaluation on whether the results are within specifications.

-19-

X-bar

8 6 4 2 0 R 2.5 |

LCL

UCL

Fig. 1

X-R chart of a process showing both upper and lower control limits set at three standard deviations from the average value lines. Sample lot size is 5 per data point.

4.2.2 Value-time charts A simpler and probably more used method within testing is recording individual measurements of the attributes of known value i.e. testing CRMs. Fig. 2 shows a hypothetical chart of this kind. A test is done each morning using a CRM of known level and uncertainty according to procedures and standards. The graph shows the uncertainty of the CRM and the (in-house) specification limits for the tests.

-20-

4.3 Using the control charts Fig. 2 a) shows a value-time chart with measurements at equal intervals and sample size 1, while limits of allowed deviations are also included. Fig. 2 b) shows increasing scatter of results; this phenomenon can be due to loss of calibration/adjustment or even wear of the instruments. All in all, the verification interval of the equipment is clearly overdue.
VALUE 12 10 8 6 4 2 0 0 VALUE
12 10 8 6 4 2 12 10 8 6 4 2 0
0

VALUE 12 10 8 6 4 2 0 20 40 60 80 100 0 VALUE 20 40 60 80

.

100

0
20 40 60 80 100

20

40

60

80

100

Fig. 2

Value-time charts showing control measurement data. Explanations of the figures in the text

Figure 2 c) shows a sudden increase in scatter in the results, which could be due to a mechanical fault in the instrument. Further analysis of the data in figure 2 d) shows that when the measurements by two operators are separated, the increase in scatter is probably due to the test being conducted by an operator less skilful than the initial operator. A process has to be under control before the quality of the results can be managed. Timely recording of testing performance and the process charts provide

-21-

simple practical means of finding the level of control of the operations, the quality of testing and pinpointing the forthcoming problems possibly causing quality nonconformance.

4.4 Interlaboratory tests in estimating laboratory performance and the uncertainty of testing Interlaboratory tests can be used to study the reliability of test methods (ISO 5725), performance of testing laboratories (proficiency testing) and to estimate the uncertainty of laboratory performance. Interlaboratory tests need to have a relatively high number of participants to yield statistically valid results. Thus such test programmes can be arranged only at relatively long intervals. Proficiency testing and estimation of uncertainty of test performance can be carried out between the interlaboratory test programmes through the use of materials whose performance (level of parameters, repeatability, reproducibility, long-term stability etc) has been established in statistically valid comparisons (interlaboratory tests). Such materials can, as mentioned above, be certified reference materials. Thus CRMs can be used in laboratories to estimate the development of uncertainty of laboratory performance. The repeatability value (r) defines the value below which the absolute difference between two single test results obtained under identical (repeatability) conditions may be expected to lie with a probability of 95%, whereas reproducibility value (R) gives the equivalent limit value when using the same method and test material but in different conditions (time of test, equipment, personnel, laboratory)4. The within-laboratory long-term reproducibility value can be used as a conservative estimate for uncertainty of laboratory performance when using the test method.
4

definition, ISO 5725

-22-

5 SOME SOURCES OF CERTIFIED MATERIALS There is several sources to get certified reference materials. Some of the best known are the Community Bureau of References, BCR, the National Institute of Standards and Technology, NIST, Laboratory of the Government Chemist, LGC and Analytical Quality Control Services, AQCS, which is related with International Atomic Energy Agency, IAEA.

5.1 BCR The role of the Community Bureau of References (BCR) programme of the European Community is to solve the major technical difficulties which arise in the measurements done in the implementation of Community Directives or in applying standardized methods of testing. BCR operates through collaborative programmes (often interlaboratory test programmes). The programmes have multiple stages with the aim of harmonizing the performance of the participating laboratories. Large numbers of laboratories carry out a particular test or measurement, and the inherent results with a high level of confidence are generally used to produce reference materials (CRMs). By autumn 1991 the BCR programme had produced in approximately 550 projects over 300 CRMs, and the CRMs are used in over 2500 laboratories. For the mechanical tests of metals BCR can supply a reference material for Charpy impact testing and is planning to certify a CRM for tensile testing (NIMONIC 75, CRM 425, which is currently available as a creep testing reference material). If a laboratory has a specific problem due to lack of reference materials it should contact BCR and initiate a collaborative programme on the topic.

-23-

5.2 NIST The National Institute of Standards and Technology (NIST) offers for sale primarily NIST Standard Reference Materials (SRMs). These are well-characterized materials produced in quantity to improve measurement science. SRMs are certified for specific chemical or physical properties. In their certificates NIST reports the results of the characterization and indicate the intended use of material. NIST has the function to develop, produce, and distribute SRMs that provide a basis for comparison of measurements on materials, and that aid in the control of production processes. To perform this function, the Office of Standard Reference Materials evaluates the requirement of science, industry, and government for carefully characterized reference materials, and their production and distribution.

6 COMAR DATABASE The Reference Material Committee (REMCO) of ISO adapted and started jointly to develop the French reference material database originated by Service de Matriaux de Rference at the Laboratoire National dEssais (LNE). The original goal was to provide answers to the question Where can I buy a CRM with the following properties...? without the need to consult all the catalogue s of the many suppliers. (Klich 1991) Code of Reference Materials COMAR has been developed by three contractors; LNE (France) as the main contractor and LGC (UK, replaced NPL 1988) and BAM (Germany) as associate contractors. COMAR consists of over 3500 CRMs from 50 producers; the database contains * name and general description of the material, * name and address of the producer, * form of the material and * certified properties.

-24-

The COMAR database is delivered on floppy disks and a standard IBM-compatible (286 processor or better, min 30Mb hard disk) computer can run the database. The user interface is rather easy to learn through a step-by-step menu system though there seem to be still a number of intrinsic bugs and anomalities. The database is quite comprehensive and thus searching for relevant reference materials can be tedious. Good knowledge and experience of the database and also the tests performed and CRMs needed is necessary to efficiently use the database. The database is now reaching the level of international commercial utilization. Currently the contractors are unfortunately not able to quote uniform prices and terms for delivery and updating the database. Hopefully an agreement will be reached in the REMCO meeting in May 1992.

7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS In order to get information on the general knowledge of reference materials and on use of the RMs in testing laboratories (destructive testing of metals) a questionnaire was sent to 82 testing laboratories in the Nordic countries. Questions on the use of testing standards and reference materials for 12 different test methods was asked. Answers were received from 16 laboratories. The distribution of the test methods and the reference materials used in these laboratories is presented table 1.

-25-

Table 1 The test methods used in the laboratories

Test

Number of laboratories 14 13 16 13 5 12 4 13 4 2 7 7

Number of labs using RMs 9 9 9 2 4 -

Brine1l hardness testing Rockwell hardness testing Vickers hardness testing Tensile testing Compression testing Bend testing Flattening testing Charpy impact testing Drop weight testing (DWT) Drop weight tear testing (DWTT) Fatigue testing Fracture mechanical testing

According to the answers, reference materials are quite widely used in hardness testing and most of reference materials had some certification; there were 7 suppliers. Most of the certificates did not fully correspond with the ISO GUIDE 311981 Contents of certificates of reference materials. Some certificates referred to national standards (DIN, BSI) and national requirements and were thus mainly intended for local use; certificates were not very clear in general. The most popular (and generally also the most trusted) hardness reference blocks come from Staatliches Materialprfungsamt Nordrhein-Westfalen (MPA-NRW), whose certificates give good information on the reference blocks and their use. Calibration (and certification) of these blocks is according to DIN 51303. The British and the French systems have accredited (or recognized) suppliers of reference blocks and some of the certificates (most of the new certificates) refer to the accreditation. Many laboratories used reference blocks with no certificates; some RMs were rather old, dating from the 1970s, so the use of the blocks had obviously not been very frequent.

-26-

One laboratory manufactures in-house reference materials (working standards) for tensile and Charpy impact testing (RMs according to ISO 442) and supplies one participating laboratory with tensile test reference material. There were a number of accredited laboratories from different Nordic countries among the respondents. Some of these laboratories did not use reference materials or used uncertified RMs; one laboratory stated that their equipment is traceably verified once yearly according to standard verification methods and thus no reference material is needed for quality assurance of testing. These answers definitely show that the accreditation requirements between the Nordic countries vary significantly as in some countries the use of CRMs in indirect verification and between-verification control of equipment is compulsory. The questionnaire showed that the significance of the use of reference materials is not very well understood or even identified as a part of quality assurance activity in a testing laboratory. Availability of reference materials and of general information on reference materials seems very limited.

8 CONCLUSIONS The reference materials are an integral part of the quality assurance of testing laboratories. Test performance between laboratories can be harmonized through use of certified reference materials, and individual laboratories can continually follow up their own performance and check the instruments between verifications and calibrations. Reference materials are used also to calibrate and tune instruments which are difficult or impossible to calibrate with direct methods. RMs are used as a transfer standard between a high-quality instrument and a working instrument.

-27-

Recommendation: The importance of attention to quality and traceability of measurements of testing laboratories cannot be stressed enough. Use of reference materials in periodic checks, verifications and calibrations is an essential part of quality assurance. Laboratories should judge for themselves how to organize the process; accreditation bodies should stress the significance of RMs and if possible publish supportive information.

Related standards There is a definite incompatibility between quality standards (EN 45001, ISO GUIDE 25), testing standards and the quality of reference materials and partly also the application of EN 45001 by the national accreditation bodies. Recommendation: Test methods should be harmonized with the quality standards and also the available reference materials (requirements in standards have to be related to physical possibilities and available uncertainty).

Availabilitv of reference materials The number and quality of reference materials are increasing, although information on them has not been spread so well. The active role of BCR has had a positive effect on the field; bringing laboratories and RM producers (Appendix 5) together in collaborative programmes will increase knowledge and interest in the related questions. All CRMs should be produced in a way which would allow for international use and distribution, preferably through BCR in Europe. Recommendation: International distribution of CRMs is encouraged; COMAR, for example, should be updated and used actively.

-28-

Certification documents

Some of the reference materials available are still inadequately documented uncertainties are not given and it is difficult to know whether the material has been adequately studied and the RM produced in a suitable manner. This uncertainty has a negative effect on the willingness of laboratories to use reference materials. In some countries RMs are produced by accredited or national calibration laboratories according to national standards. Information on the certification/calibration methods is generally available nationally but not internationally. Recommendation: The producers of RMs and CRMs should be encouraged to publish certificates according to ISO GUIDE 31 to help users (laboratories) evaluate and compare possible CRMs.

COMAR database The COMAR database is an effective and efficient way to find relevant certified reference materials. The database should be at least nationally available (in France 30 Minitel subscribers are currently connected to COMAR). There are two major concepts of organizing the use of COMAR nationally; on one hand a national body (accreditation body, national technical information centre etc) could centrally be responsible for giving out information on reference materials, on the other hand national laboratories would know better their own technical field and could thus better serve testing laboratories. A national solution is, however, necessary to spread the information to all laboratories needing information on reference materials.

-29-

REFERENCES

ASTM E 23-88. Standard test methods for notched bar impact testing for metallic materials. ASTM E 127l-88. Standard practice for qualifying Charpy verification specimens of heat-treated steel EN 45001:1989, General criteria for the operation of testing laboratories ISO 442-1965. Verification of pendulum impact testing machines for testing steels. ISO 9001:1987, Quality systems. Model for quality assurance in design/development, production, installation and servicing ISO/IEC GUIDE 25:1990 (E) General requirements for the competence of calibration and testing laboratories ISO GUIDE 30:1981 (E), Terms and definitions used in connection with reference materials ISO GUIDE 31:1981 (E), Contents of certificates of reference materials ISO GUIDE 33:1989 (E), Uses of certified reference materials ISO GUIDE 35:1989 (E), Certification of reference materials - General and statistical principles Juran, J.M. 1974. Quality Control Handbook. 3rd ed. New York, McGraw-Hill Book Company. 1780 p.

-30-

Klich, H. 1991. COMAR Database for Certified Reference Materials. Berlin, BAM. 4p. Klich, H. Personal communication and COMAR demonstration, 31.1.1992. Berlin, BAM. NAMAS Accreditation standard M10, General Criteria of Competence for Calibration and Testing Laboratories. 1989. 24p. NAMAS Information Sheet NIS 13, Traceability, hardness measurements. 1987. 5p. NIST Standard Reference Materials Catalogue 1990-91, NIST Special Publication 260, U.S. DEPARTMENT OF COMMERCE, National Institute of Standards and Technology prEN 10 045 Part 2:1990, Metallic materials - Charpy impact test - Part 2: Verification of the testing machine (pendulum impact) Quality Management in the Laboratory, Espoo, Finland, 24. - 25.9.1991, National Association of Testing Authorities, NATA Australia in conjunction with the Technical Research Centre of Finland, VTT.

Excerpts from BCR REFERENCE MATERIALS CATALOG Appendix 1 1(2)

INTRODUCTION
For some ten years the Community Bureau of Reference (BCR), a department of the Commission of the European Communities, has provided laboratories with a means of carrying out collaborative measurement studies in Physical Measurements and Chemical Analysis. To understand the role of the BCR it is important to remember that common written standards (Norms), although essential to achieve harmonization of measurements, are not sufficient by themselves to avoid discrepancies of results. The principle of this contribution is to bring together some of the most expert laboratories of preferably all the Member States for intercomparison exercises. Results are discussed with all participants. Discrepancies are given particular attention to identify their causes. Subsequently, additional measurements are made to verify that the causes of errors can be eliminated. When necessary several successive intercomparisons are done to study the effect of the different parameters.

Certification - Uncertainties - Uses When the major sources of errors are eliminated and agreement is reached on a value which can be considered as the most accurate in the current state of the art, it is essential to provide a means of maintaining the consistency between laboratories and of disseminating the accurate value to other laboratories. The instruments or pieces of equipment used to transfer values are called transfer standards in Applied Metrology and reference materials in Analytical Chemistry. The Community Bureau of Reference undertakes the establishment of transfer standards and reference materials which can be used by any laboratory which wishes to verify its methods and calibrate its instruments. The uncertainty is by definition the estimate of the range of values within which the true value is most likely to be. The uncertainties given in this catalogue take into consideration the variations between laboratories and between methods and care is taken not to underestimate uncertainties. They are given at a confidence level of 95%. A certified reference material can be used by a laboratory to verify if the method used gives results in agreement with the certified values. If the result is different the user is immediately warned that he has to look for errors in his method and to find their causes. For calibration the reference material is used to assign a value to the indication of his instrument. Reference materials are used mainly for: - calibration and verification of measurement processes in routine analytical conditions; - internal quality control and quality assurance schemes; - verification of the correct application of standardized methods; - development and validation of new methods of measurements. The BCR programme welcomes any suggestion from industrial organisations, official laboratories, research establishments or universities, concerning measurements of practical importance that need to be improved or for which there is a need for reference materials. Further information on the BCR is available on request.

Report Each sample is usually delivered with a certificate and a report which contains all the data which formed the basis for the certification. It also gives details on the methods of measurements that were used, recommendations on the best methods to use, and advice for the use of the material.

Excerpts from BCR REFERENCE MATERIALS CATALOG Appendix 1 2(2)

Procedure for purchase of BCR reference materials

The certified reference materials which are made available by the BCR were established in response to a need to help laboratories to improve their measurements. Except when batches of reference materials are very limited, the BCR makes no restriction in their availability for these specific uses as described in its catalogue. The BCR cannot supply quantities of these materials for any other types of uses. 1. Purchase orders for reference materials and requests for information should be addressed to:

COMMUNITY BUREAU OF REFERENCE (BCR) COMMISSION OF THE EUROPEAN COMMUNITIES 200, Rue de la loi B-1049 BRUSSELS
or by telexing: BCR -BRUSSELS 21877 COMEU B All purchase orders should give: CRM numbers indicated in the catalogue, name of the material and number of units required. 2. Delivery time: 3-4 weeks maximum after receipt of the purchase order by the BCR. 3. Shipment: prices include handling and normal postage (surface mail, air mail). If airfreight is requested, it has to be paid by the customer. 4. Terms of payment: the payments shall be made preferably by bank transfer following instructions to be found in the debit note (or invoice) that the customer will receive after delivery of the samples. Methods of payments such as credit letter are not acceptable. Prices of reference materials are shown in the price list attached. Payments are expected within 30 days of receipt of the debit note.

Appendix 2 l(2)

COMMISSION OF THE EUROPEAN COMMUNITIES

COMMUNITY BUREAU OF REFERENCE - BCR

Ns. 27 I to 56E

CERTIFIED REFERENCE MATERIAL

CERTIFICATE OF MEASUREMENT

Batch E

CRM 016
IMPACT TOUGHNESS OF V-NOTCH CHARPY SPECIMENS
(Energy level 120 J Nominal)

Impact energy at 20 C According ASTM E 23-82 According ISO

Cenified value (1) 124.8 J 121.5 J

Uncertainty (2)

5.0J
4.7J

(1) This is the mean impact energy for the 5 specimens delivered in one set (2) The uncertainty includes mainly the effect of the variation between samples

DESCRIPTION OF THE REFERENCE MATERIAL A unit comprises 5 specimens which must be broken successively in the machine to verify. The specimens are packed in protective oil and cleaning must done as recommended.

Appendix 2 2(2)

INSTRUCTTONS FOR USE

Specimens should be kept in their original packing until they have to be used. Special attention is drawn to cleaning, for which vigorous methods should be avoided (e.g. brushes or ultrasonic). The following is recommended :
1. Wipe excess oil with cellulose paper 2. Immerse specimens in a clean bath of degreasing solvent for about 5 minutes 3. Wipe the specimens with cellulose paper 4. Immerse the specimens in a bath of fresh absolute alcohol 5. Wipe with paper and allow to dry in still air 6. Keep at room temperature for 24 hrs prior to testing PARTICIPATING LABORATORIES

National Physical Laboratory, Teddington (UK) Material Testing Laboratory, Watertown (USA) Cogne, Aosta (IT) Franco Tosi Industriale, Legnano (IT) Cermet, Bologna (IT) Equipos Nucleares SA, Santander (ES) Bundesanstalt fr Materialforschung und-prfung, Berlin (DE) Staatliches Materialprfungsamt, Dormund (DE) Ugine, Isbergues Research Centre (FR) Laboratoire National dEssais, Paris (FR) Aubert et Duval, Gennevilliers (FR) Aubert et Duval, Les Ancizes (FR) Centre de Recherches et dEtudes dArcueil (FR) Sollac, Dunkerque (FR) GTS Industries, Dunkerque (FR)
METHODS USED

Charpy Testing Machine according to ASTM E23-82 Charpy Testing Machine according to ISO (R 442)
LEGAL NOTICE

This document was prepared under the sponsorship of the Commission of the European Communities. Neither the Commission of the European Communities nor any person acting on its behalf makes any warranty or representation, express or implied, that the use of any information, material, apparatus, method or process disclosed in this document does not infringe privately owned rights ; or assumes any liability with respect to the use of, or for damages resulting from the use of, any information, material, apparatus, methods or process disclosed in this document. CERTIFICATION REPORT The work done to certify this reference material is described in a report which is available to users free of charge upon request.
Provisional address : rue de la Loi 200, B-1049 Brussels - Phone (32 2) 235.50.14 - Fax (32 2) 235.80.72

Appendix 3 l(1)

Producers of RMs and CRMs: - Acco Industries Inc. (ACCO) - The American Oil Chemists Society (AOCS) - Analytical Quality Control Services (AQCS) - Centre Mcanique Chimie Matriaux (CMCM) - Community Bureau of Reference (BCR) - Eidgenssische Materialprfungs- und Forschungsanstalt (EMPA) - Engineering & Scientific Equipment Ltd. (ESE) - Forschungs- und Materialprfungsanstalt Baden-Wrttemberg (FMPA) - Foundrax Engineering Products Ltd - Georg Reichter, Eslinger/Neckar - Laboratoire National dEsaais (LNE) - Laboratory of the Government Chemist (LGC) - National Institute for Environmental Studies (NIES) - National Institute of Standard and Technology (NIST) - National Research Centre for Certified Reference Materials (NRCCRM) - National Research Council Canada (NRC) - Shimadzu Kyoto - Staatliches Materialprfungsamt Nordhein-Westfalen (MPA NRW) - Werk - Yamato Scientific

TECHNICAL REPORTS FROM THE PROGRAMME GROUP FOR QUALITY ASSURANCE

082 Larsen, B., NORDTEST Guidelines for development of Quality Manuals for testing laboratories. Helsinki 1987. Nordtest, NT Techn Report 082. 60 p. (in English and Danish) Fahln, P., Madsen, E., Johansen, M.M., Rautiainen, L., Siitonen, V., Herje, J. & Sandberg, P.I., Guidelines for development of Quality Manuals in the field of building physics, heating & ventilation and adjacent technical areas. Helsinki 1987. Nordtest, NT Techn Report 084. 25 p. Hyrynen, J., Draft NORDTEST guidelines for interlaboratory test comparison, Espoo 1988. Nordtest, NT Techn Report 104. 95 p. Fremstad, D., Lundgren, B., Olsen. H. & Saarela, K., Guidelines for development of Quality Manuals in the field of chemical analysis. Espoo 1988. Nordtest, NT Techn Report 105. 32 p. Haglund, M., A check list concerning the cost/benefit of laboratory accreditation. Espoo 1989. Nordtest, NT Techn Report 107. 18 p. (in Swedish) Haglund, M., Check-list concerning the advantages, cost and disadvantages of laboratory accreditation. Espoo 1990, NT Techn Report 107 Engl, Edition 2. 9 p. Hyrynen, J., Interlaboratory testing. Espoo 1989. Nordtest, NT Techn Report 118. 11 p. Quality assurance in testing laboratories, Espoo 1989. Nordtest, NT Techn Report 119. 156 p. (in Dan/Nor/Swed/Engl) Quality assurance in testing activities. Espoo 1990. Nordtest, NT Techn Report 131. 261 p. (in Dan/Nor/Swed/Engl) Larsen, J.B., An examination of the costs in establishment and maintenance of an accreditation and the including quality control system. Espoo 1991. Nordtest, NT Techn Report 143. 106 p. Ploug, H.H., Nordtest practical guidelines for round robin tests. Espoo 1991. Nordtest, NT Techn Report 144. 28 p. Lindskov Hansen, S., Guidelines for the development of software to be used in test and measuring laboratories. Espoo 1991. Nordtest, NT Techn Report 145. 46 p. Ohlon, R., About procedures for internal quality audits of testing laboratories. Espoo 1991. Nordtest, NT Techn Report 146. 39 p. Andersson, H., An introductory study of calibration in testing laboratories. Espoo 1991. Nordtest, NT Techn Report 147. 30 p. nk, S. & Sillanp, J., Requirements on the personnel and organization of testing laboratories. Espoo 1991. Nordtest, NT Techn Report 148. 70 p. (in Swedish) Forstn, J., A view on the assessment of the technical competence of testing laboratories. Espoo 1991. Nordtest, NT Techn Report 149. 46 p. Sillanp, J., Requirements of ISO 9002 for testing and inspection. Espoo 1992. Nordtest, NT Techn Report 164. 42 p. Baade, S., Interpretation of EN 45 001 for fire testing laboratories, Espoo 1992. Nordtest, NT Techn Report 165. 25 p. Hyrynen, J., Kallio, H. & Kosonen, A.-M., Reference materials in quality assurance of testing laboratories. Espoo 1992. Nordtest, NT Techn Report 177. 35 p. nk, S. Qualification system for the personnel of testing laboratories. Espoo 1992. Nordtest, NT Techn Report 178. 17 p. Ohlon, R., Comparison of standards with requirements on calibration and testing laboratories, Espoo 1992. Nordtest, NT Techn Report 179. 71 p.

084

104 105 107 107:E 118 119 131 143 144 145 146 147

148 149 164

165 177 178 179

nordtest
Mailing Address: Tekniikantie 12 FIN-02150 Espoo, Finland. Phone +358 9 455 4600, fax +358 9 455 4272 www.nordtest.org