Dynamic Calibration and

Validation Testing
The world leader in serving science

Why is Dynamic Calibration Required

Factory calibration calibrates each detector so it is free of interelement correlations and determines the factors in the A3 model
that enhances the performance of Thermo Gamma-Metrics coal
analyzers.
But the factory can not exactly match the field conditions;
therefore fine tuning of the calibration is required in the field.
Sources used in the field are different from what was used in the factory
Coal stream is continuous through the unit in the field while it is a finite
length (equal to the calibration standard length) in the factory.
The range of ash and sulfur are narrower in the field therefore the analyzer
can be tuned to be more accurate over the narrower range.
The analyzer must be calibrated to the reference sample system.

Precision and Accuracy and Catch 22

Neither
Precise
nor
Accurate

Precise
but not
Accurat
e

Bias Adjustment
Needed

Precise
and
Accurate

The Catch 22 is we dont know what the location of the bulls eye is. We are all
measuring it with a number of systems each with their variances. Typically an
analyzer customer thinks his sample system and lab have no error and the
analysis results from the sample system/lab is the bulls eye for an analyzer.

The objective
During the field calibration and validation process, the objective
is to calibrate the analyzer to match the results of the
customers reference system, usually his sample system and
lab.
The analyzer measures the quality of all the coal that passes
through it.
The lab analyzes a very small subset of the material on the belt
to determine the quality of a large lot (usually one hour) of coal
flow.
It is critical that the sample analyzed in the lab is representative
of the coal on the belt seen by the analyzer.
It is critical that the lab prepare and analyze samples with precise
adherence to standard procedures,. with well maintained
equipment and well trained people.

Sample collection
Collection of each gross sample is controlled by ASTM
Standards D2234 (Collection of a Gross Sample of Coal) and
D2013 (Preparing Coal Samples for Analysis) in the USA.
D2234 covers collection of a gross sample without any crushing.
D2013 covers the reduction and division of the gross sample
collected according to D2234 with crushing.

To calibrate an analyzer generally requires 30 gross samples

covering a range of coal quality.
At least 20 gross samples are required to validate the analyzer
calibration.

Minute-by-minute sulfur and ash data for a unit train

loading.
LoadingfromHighSulfurPile

DozerPushingfrom
HighAshPile

D2234defineshowmanyincrementsare
requiredtoaccuratelyrepresentthequalityof
thistrain.

The highlights of ASTM D 2234 collection of a gross

sample.
Number of increment samples required to make up a single gross
sample?
For coal with consistent quality (clean coal from a single seam) 15 increments
up to a 1000 ton lot size.
For inconsistent coal (raw coal, partial wash coal, any blend coal) 35
increments up to a 1000 ton lot size.
For lot sizes greater than 1000 tons, the number of increments required is;

N 2 N1

total lot size (tons )

1000 tons

N1=minimumincrementsshownabove
N2=numberofincrementsrequiredforlotsize
7

Example of number of increments required

Loading a unit train
Blended Coal dozed from different stacking tube piles; N 1 = 35
The unit train capacity is 10,000 tons
Therefore

10,000
N 2 35
110 .68 111
1000
111 individual increments are needed to represent the coal on the belt.

Impact of number of increments on the repeatability and how

well the sample represents the total lot.

Asthesegraphsshow,itiscriticalthatenoughincrementsaretakentoaccurately
representthequalityofthelotofcoal.Iftoofewincrementsaretaken,therandom
errorinthesamplecollectionprocesswillnegativelyimpactthecalibrationofthe
analyzer.
9

The highlights of ASTM D 2234 collection of a gross

sample, continued.
When a sample increment is taken the sampling device must have an
opening at least 2-1/2 to 3 times the top size of the coal, but never
smaller than 1.25-inches. This applies to stop belt sampling, falling
stream cutters or sweep samplers. Falling material must strike the
sample device at right angles to the opening.
In addition the cutter speed is specified (18inches/second) and the
minimum amount of sample that has to be taken, based on the material
top size is also specified.
Top size

5/8 inch
16-mm

2 inch
50-mm

6 inch
150-mm

2-lbs
1-kg

6-lbs
3- kg

15-lbs
7-kg

Minimum increment weight

10

The highlights of ASTM D 2234 collection of a gross

sample, continued.
If a very large sample increment is collected in the primary
sample, it is often required that the primary sample be reduced in
size with secondary sampling before a crushing step is
undertaken.
At least six secondary increments are required for each primary
increment.
All the rules concerning minimum increment weight, cutter speed
and cutter opening apply to this secondary sampler.

Once the sample has passed through a crusher, ASTM D 2013

defines the sampling procedures.

11

ASTM D 2013 Preparing coal samples for analysis

D 2013 covers both the manual and the mechanical reduction
and devision of samples collected according to D 2234.
Applies to sample reduction after crushing in the sample system
Applies to the lab manual reduction of the sample collected by a
sample system

At least 60 increments are required from the sample taken by D

2234. Good practice recommends that at least one tertiary cut
be taken for every primary cut if this is a larger number than 60.
The minimum sample size required is determined by how fine the
coal is crushed and whether the coal is 100% clean coal from
one seam or is it a variable coal quality (raw coal or blends).

12

ASTM D 2013 Preparing coal samples for analysis,

continued
PreparationofLaboratorySamples

Crush to 95% passing

through sieve

Divide to a minimum weight of, Grams

Group A

Group B

No.4 (4.75 mm)

2000

4000

No. 8 (2.36 mm)

500

1000

No. 20 (850 m)

250

500

No. 60 (250 m)

50

50

Tableshowstheminimumweightofthefinalsampledepending
onthematerialsizefollowingcrushing.
GroupA=coalsthathavebeencleanedinallsizes
GroupB=Allothercoalsincludingunknown
13

ASTM D 2013 Preparing coal samples for analysis,

continued
Good practice in a sample system
Cutter opening after crushing 1- inch minimum opening
Maximum cutter speed 18 inches/sec.
Final sample weight collected by the sample system about 40
pounds.

ASTM D 2013 manual sample reduction procedures must be

rigorously followed in the lab sample preparation process to
insure the sample is not biased or contaminated in the process.
Errors made in sample collection or preparation will invalidate a
sample.

14

Dynamic Calibration of the Analyzer

Dynamic calibration of the analyzer involves calibrating the analyzer so
it matches the reference system (the sample system and the lab).
The Moisture, Sulfur, Ash Oxides and the Ash are calibrated by plotting
the analyzer results against the reference system.

15

Dynamic Calibration of the Analyzer

As the X-Y plot of the sulfur calibration data showed, PGNAA
responses are linear; therefore, the field calibration of the
analyzer will be the equation of the linear regression fitting the
analyzer raw results to the reference system results.
Before the linear regression is performed, the data should be
screened for outliers (data that does not match the trend seen
with the rest of the data).
Outliers need to be addressed before the analyzer is calibrated.

16

Handling Outliers in the Data

Errors can happen in the collection of samples, problems in the lab, improperly
recording values, etc. therefore it is not unexpected to see outliers in a data set.
If an outlier is seen, an effort needs to be made to determine is there an
attributable reason. In the data below the sample weight for all samples except
the outlier were 40-pounds 4. The sample weight of the outlier was 8-pounds.
Something must have happened to the sample system. Therefore this point was
discarded and not used in the calibration.

Outlierdatapointdiscarded
fromthedatasetbecauseit
wasdeterminedthecollected
sampleweightwastoosmall
toberepresentativeofthe
tonssampled.

17

Dynamic calibration

Thedynamiccalibrationiscompletedbyapplyingthegain(inthiscase0.8108)and
offset(+0.0642)determinedtotheanalyzercalibration.Thisisdonefortheashand
moisturetoo,completingthedynamiccalibrationoftheanalyzer.
18

Validation Test
Once the analyzer has been dynamically calibrated the validation test
can be performed.
The validation test ideally consists of the collection of 30 samples,
collected and analyzed in exactly the same manner as the dynamic
calibration samples.
As the normal guarantees are only for ash, sulfur and often moisture, the
ash oxides only need to be run if the customer is specifically interested in
them.

The Root Mean Square Deviation (RMSD) of the analyzer to the lab
data is calculated.
n

RMSD

analyzer

lab

i
i
i 1

n=numberofdatasets

19

RMSD example
Analyzer Ash

Lab Ash

Difference

Difference2

8.25

8.40

-0.15

0.0225

7.88

8.20

-0.32

0.1024

9.01

8.73

0.28

0.0784

8.65

8.82

-0.17

0.0289

7.50

7.22

0.28

0.0784

8.02

7.71

0.31

0.0961

Sum of differences
squared =

Number of samples =

RMSD =

20

0.4067

0.26

Acceptance of Analyzer Performance

The analytical precision of the analyzer is guaranteed by Thermo.
RMSD measures the combined error of the analyzer and the sample
system/lab reference system.
If the assumption is made that the analyzer and the sample system/lab
have approximately the same precision (an assumption that has been
proven to be often true). Then the precision of the analyzer can be
estimated as:

RMSD
2

21

Acceptance continued
If the measured RMSD of the analyzer is equal to or less than
the precision guarantee, the analyzer has passed the
acceptance test.
If the RMSD is larger than the guarantee, the analyzer precision
may still satisfy the guarantee.
If the lab and analyzer precision assumed to be equal, an RMSD of
0.707 would indicate an analyzer precision of 0.50.

If the precision of the analyzer is in question, then ASTM D 6543

Evaluation of Measurements Made by On-Line Coal Analyzers,
details the procedure to conduct a Grubbs Estimator
measurement of the precision of the analyzer.

22

Grubbs Estimator
Requires dual ASTM sampling of the coal stream at two different
locations.
Requires the collection of 60 samples from each sample location
Requires the samples to be processed completely by independent labs.
The result is an estimate of the precision of the three analysis systems,
the coal analyzer, sample collection/lab one and sample collection/lab
two.
The Grubbs estimate of the precision of the analyzer is what is
compared to the guarantee performance.
Thermos guarantee is a one sigma guarantee (68% of the results will
fall within plus or minus the guarantee value), and sukbject to statistical
significance testing.

23

Precision of Sample Collection

How well does the sample collected represent the true quality of
the lot?
What is the precision of the sample collected by the sample
system?
For the example in slide 9 the measured ash had a precision of 0.60 and the sulfur
precision was 0.036 at the ASTM recommended increment.
This number will be less for coal streams with less variability and more if the variability is
greater.

The sample collected by the sample system goes back to the lab
and is air dried, crushed and manually reduced to 500 grams that
goes into the lab.
The lab then runs very small subsets of the prepared sample in a
number of instruments.

24

Reproducibility Lab to Lab

Repeatability is the ability of a single lab to duplicate the same
results when repeating the same analysis of the same sample.
Reproducibility is how well the results from two different labs
should compare for the same sample.
As the coal analyzer is one lab we should look at the
reproducibility of results on analysis system to another 95% of
the time (2 sigma).
D 3173 Moisture Reproducibility - 0.3% for coals with less than 5%
moisture and 0.5% for coals with more than 5% moisture.
D 3174 Ash Reproducibility 0.32% on a dry basis
D 4239 Sulfur Reproducibility 0.02 + 0.09(average sulfur between
labs)

25