Update on FDA Center for Devices and

Radiological Health Investigation of

Quantitative CT through Anthropomorphic
Phantom Studies
Nicholas Petrick1, Marios A. Gavrielides1,
Rongping Zeng1, Lisa M. Kinnard2, Kyle J. Myers1

1
Division of Imaging and Applied Math, OSEL/CDRH/FDA
2
US Army Medical Research and Materials Command
Outline

• Update on FDA CT phantom collection

• Reference Standard (truthing)
• Latest Analysis
• Summary

N Petrick PCF Presentation 2

May 14, 2010
Assessing Quantitative
Measurements
• Simulation studies
• In silico phantoms

• Physical phantom studies

• Anthropomorphic phantoms
• Realistic imaging physics
• Ability for repeat measurements

• Clinical studies
• Coffee break experiment
• Repeat scans on same patient
N Petrick PCF Presentation 3
May 14, 2010
Quantify Sources of
Estimation Error
• Understand how specific parameters
influence bias/variance of measurements
• Development of standards for imaging protocols &
software tools
• Speed adoption of quantitative imaging in drug
trials and clinical practice

• Systematic data collection

• Allows for measuring both bias and variance
N Petrick PCF Presentation 4
May 14, 2010
FDA CT Phantom Study

Data Collection
Gavrielides, SPIE 2010

5
FDA Phantom Experiment
• Goal
• To quantify sources of measurement error in nodule change
assessment and when possible, minimize their effects

• Method
• Well-controlled anthropomorphic phantom studies
• Measure effect sizes
• Identify influential imaging parameters
• Determine smallest measurable change

• Funding
• FDA: Critical Path
• NIH: NCI, NIBIB
N Petrick PCF Presentation 6
May 14, 2010
Factors Influencing Quantitative
Accuracy and Precision
• Scanner characteristics

• Acquisition/reconstruction parameters

• Nodule characteristics

• Analysis software

• Patient biology/genetics

• Clinician
N Petrick PCF Presentation 7
May 14, 2010
Anthropomorphic Thorax
Phantom

N Petrick PCF Presentation 8

May 14, 2010
Nodules

lobulated elliptical spiculated spherical irregular

N Petrick PCF Presentation 9

May 14, 2010
Nodules

HU Range
[-709, +41]

Courtesy of Binsheng Zhao, Columbia University

Nodules
• Shapes
• Spherical, elliptical,
lobulated, spiculated,
irregular
• Sizes
• 5-40 mm
• Densities
• [-800, +100 HU]
• Locations
• Attached
• Unattached

N Petrick PCF Presentation 11

May 14, 2010
Data Collection
Layout 1: 5-10mm Spheres
8 mm
8 mm

• Philips 16: 480 sets

• Siemens 64:
10 mm
• Total: 480 sets
10 mm

5 mm
5 mm

-800 HU -630 HU

N Petrick PCF Presentation 12

May 14, 2010
Data Collection
Layout 2: 5-10mm Spheres/Irreg.
8 mm 8 mm
• Philips 16: 720 sets
• Siemens 64:
• Total: 1200 sets
10 mm 12 mm

10 mm
5 mm

-800 HU -300, +20, +100 HU

N Petrick PCF Presentation 13

May 14, 2010
 Data Collection
Layout 3: 10-20mm Aspherics
10 mm 10 mm

• Philips 16: 720 sets

20 mm 20 mm • Siemens 64: 720
B
sets
10 mm A
10 mm • Total: 2640 sets
A
A
20 mm 20 mm • Siemens 64
A
A • Collected
10 mm curtsey of
20 mm
A 10 mm
? Washington
A
20 mm University
B

+100 HU -630 HU

N Petrick PCF Presentation 14

May 14, 2010
Data Collection
Layout 4: 5-40mm Spheres
8 mm 8 mm

• Philips 16:
• Siemens 64: 720
20 mm 10 mm sets
20 mm
• Total: 3360 sets
10 mm

• Siemens 64
• Collected
40 mm 5 mm 40 mm
5 mm
curtsey of
Washington
University

+100 HU -630 HU

N Petrick PCF Presentation 15

May 14, 2010
Data Collection
Layout 5: 40mm Spiculated
• Philips 16:
• Siemens 64: 360
sets
40 mm
40 mm • Total: 3720 sets

• Siemens 64
• Collected
curtsey of
Washington
University

+100 HU -630 HU

N Petrick PCF Presentation 16

May 14, 2010
Data Collection
Layout 6: 5-40 mm Spheres
A
20 mm 8 mm
• Philips 16: 360 sets
• Siemens 64:
B • Total: 4080 sets
20 mm 20 mm
10 mm

40 mm

5 mm

40 mm

-630, +100 HU -10 HU

N Petrick PCF Presentation 17

May 14, 2010
Data Collection
Layout 7: 5-20 mm Aspherics
10 mm
5 mm

• Philips 16: 720 sets

20 mm • Siemens 64:
8 mm
• Total: 4140 sets
10 mm
5 mm

20 mm
5 mm

10 mm
8 mm
8 mm 20 mm

-10 HU -10 HU

N Petrick PCF Presentation 18

May 14, 2010
Data Collection (2009)
Layout 8: 5-10 mm Spheres
8 mm 8 mm
• Philips 16: 720 sets
• Siemens 64:
• Total: 4860 sets

10 mm 10 mm

5 mm 5 mm

-800. -10 HU -630, +100 HU

N Petrick PCF Presentation 19

May 14, 2010
Data Collection (2009)
Layout 9: 5-8 mm Aspherics
8 mm
5 mm

• Philips 16: 720 sets

8 mm 5 mm • Siemens 64:
• Total: 5580 sets
8 mm
5 mm

5 mm 5 mm

8 mm
8 mm
8 mm
5 mm

+100 HU -630 HU

N Petrick PCF Presentation 20

May 14, 2010
Data Collection
General Properties
Attached Nodules
Layout 1-9

Exposure (120 KVp) 25 mAs 100 mAs 200 mAs

Standard High
Pitch
1.2 0.9

x10
Collimation 16x0.75 mm 16x1.5 mm

Slice Thickness 0.8 mm 1.5 mm 3.0 mm 5.0 mm

Reconstruction Kernel Detail Medium Recon

21
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 22

May 14, 2010
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 23

May 14, 2010
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 24

May 14, 2010
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 25

May 14, 2010
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 26

May 14, 2010
Public Access
(https://imaging.nci.nih.gov)

N Petrick PCF Presentation 27

May 14, 2010
CT of Thorax Phantom

0.8 mm slices
16x0.75
100 mAs

Case 1114, 100 mAs, 16x0.75, 0.8 mm Slices

N Petrick PCF Presentation 28

May 14, 2010
Current Collection
• Nodule
• Spiculated
• 20 mm, -630 HU

• Acquisition
• 200 mAs
• 1.2 pitch
• 0.8 mm slices
• BF40 kernel
N Petrick PCF Presentation 29
May 14, 2010
Future Collection
• Nodule
• Mixed density
• 10mm, -10HU outer
• 5mm -630 HU core

• Acquisition
• 200 mAs
• 1.2 pitch
• 0.8 mm slices
• BF60 kernel
N Petrick PCF Presentation 30
May 14, 2010
Discussion
• >5000 scans completed
• Public access through National Biomedical
Imaging Archive
• https://imaging.nci.nih.gov
• Additional releases every 4-6 weeks

• Current focus on
• Completing solid nodule collection
• Heterogeneous nodules

N Petrick PCF Presentation 31

May 14, 2010
FDA CT Phantom Study

Reference Standard
Kinnard, SPIE 2010

32
Study Purpose
• Investigate methods to estimate true
volume & shape of nodules
• How accurate is truthing process?

• Truth measurement methods

• Weight-density
• Micro CT
• Water displacement
N Petrick PCF Presentation 33
May 14, 2010
Why worry about truth?
• Standardization of quantitative imaging
• Accurate measurements of variance (and bias)
• Need truth to estimate bias

• Why bias matters

• Different acquisition parameters may have
different biases
• Different biases among scans has direct impact on
precision of change assessment

N Petrick PCF Presentation 34

May 14, 2010
Why micro CT?

• Micro CT provides
• Very high resolution imaging

• Absolute volume
• Shape information
• Useful for understanding how software perform
with complicated shapes

N Petrick PCF Presentation 35

May 14, 2010
 Methods

• 26 synthetic nodules
• Shapes
• spherical, elliptical,
lobulated, spiculated
• Sizes (nominal diameter)
• 5, 8, 10, 20, 40 mm
• Densities
• -630, +100 HU

N Petrick PCF Presentation 36

May 14, 2010
Nodule Volume
• Weight-density
• Density
• By manufacturers
• Weight
• Precision scale
• 0.1 mg tolerance

• Micro CT
• Scanco CT 40
• Imtek MicroCAT II

• Measurements courtesy
• NIST
• Washington University
40 mm lobulated nodule in MicroCAT II

N Petrick PCF Presentation 37

May 14, 2010
Micro CT Parameters
• Mesh size
• Consistent
through
(10,10,10)

• Chose (1,1,1)
• Highest
precision

N Petrick PCF Presentation 38

May 14, 2010
Micro CT Processing
Parameters
• Segmentation threshold
• Upper/lower grayscale threshold
• Upper threshold constant
• Varied lower value
• Selection strategy
• Segmentations consistent with perceived edge
• 3D-rendered volume consistent with weight-
density

N Petrick PCF Presentation 39

May 14, 2010
Comparison of Volume Estimates

• Calculated percentage difference (PD):

 VμCT  VWD 
• PD  100    (%)
 VWD 

• Estimates
• VWD Weight-density estimate
• VCT Micro CT estimate

N Petrick PCF Presentation 40

May 14, 2010
Results

• For all nodule shapes and sizes

• PD: -630 HU nodules
• Mean: -11.9%
• Range: [-21.74%, -1.48%]

• PD: +100 HU nodules

• Mean: 1.7%
• Range: [-0.88%, 2.99%]

N Petrick PCF Presentation 41

May 14, 2010
Spherical Nodules

N Petrick PCF Presentation 42

May 14, 2010
Elliptical Nodules

N Petrick PCF Presentation 43

May 14, 2010
Lobulated Nodules

N Petrick PCF Presentation 44

May 14, 2010
Spiculated Nodules

N Petrick PCF Presentation 45

May 14, 2010
Segmentation Example

Spiculated, -630 HU, 40 mm

N Petrick PCF Presentation 46

May 14, 2010
3D Renderings

Spiculated Nodule Lobulated Nodule

N Petrick PCF Presentation 47

May 14, 2010
Discussion

• Segmentation appears consistent with

visual borders

• Blurred boundary
• Imaging & reconstruction process
• May lead to underestimates in volume

N Petrick PCF Presentation 48

May 14, 2010
Discussion
• Difference between micro CT & weight-density
• Increases for lower density nodules
• -630 HU nodules
• Increases for complicated shapes
• Lobulated, spiculated

• Weight-density may underestimate true volume

• Air pockets
• Earlier experiment showed air pockets did not contribute to
significant volume differences

N Petrick PCF Presentation 49

May 14, 2010
FDA CT Phantom Study

Latest Analysis

50
FDA Volume Estimator

• Purpose
• To develop a low bias/variance estimator
• Improves measurement of CT effects by
reducing software effect

• Matched filter volume estimator

• Sum of absolute differences (SAD) metric

N Petrick PCF Presentation 51

May 14, 2010
Attached Spheres
0.8mm, Attached, SAD

20

• Legend

Normalized bias
10

0
• + -630HU -10

• О -10HU -20
5 8 10
• Δ +100HU Nodule Size (mm)
3mm, Attached, SAD

20
Normalized bias

10
• Acquisition 0

• 200 mAs -10

-20

• 1.2 pitch 5 8
Nodule Size (mm)
10

• BF60 kernel
N Petrick PCF Presentation 52
May 14, 2010
Non-attached Spheres
0.8mm, Non-attached, SAD

20

• Legend

Normalized bias
10

0
• + -630HU -10

• О -10HU -20
5 8 10
• Δ +100HU Nodule Size (mm)
3mm, Non-attached, SAD

20
Normalized bias

10
• Acquisition 0

• 200 mAs -10

-20

• 1.2 pitch 5 8
Nodule Size (mm)
10

• BF60 kernel
N Petrick PCF Presentation 53
May 14, 2010
Attached Elliptical
• Legend
• О -10HU

• Acquisition
• 200 mAs
• 1.2 pitch
• 0.8 mm slices
• BF60 kernel

N Petrick PCF Presentation 54

May 14, 2010
Discussion
• Spherical nodules
• Little difference between attached/non-attached
spheres in bias or variance
• Elliptical nodules
• Bias
• ~2% for 8 & 10 mm ellipsoids (0.8 mm slices)
• -27% for smallest 5 mm ellipsoids

• Bias is mainly impacted by nodule size

• Relative to slice thickness
• Variance small
N Petrick PCF Presentation 55
May 14, 2010
Summary
• FDA data collection progressing
• Over 5000 data sets
• Public access: https://imaging.nci.nih.gov

• References standard
• Micro CT provide both volume and shape information
• Volume estimates consistent with weight-density for high
densities nodules and simple shapes

• Data analysis (volume estimation)

• Bias is mainly affected by nodule size
• Variance small for matched filter estimator
N Petrick PCF Presentation 56
May 14, 2010
Acknowledgments
• Phantom scans on the Siemens 64 were conducted at the Center for
Clinical Imaging Research in the Mallinckrodt Institute of Radiology
(MIR) at the Washington University School of Medicine in St. Louis,
MO. The authors would like to thank Bruce Whiting of MIR for his
assistance

• Micro CT scans were obtained from both NIST and Center for Clinical
Imaging Research in the Mallinckrodt Institute of Radiology (MIR) at the
Washington University School of Medicine in St. Louis, MO. The
authors would like to thank Sheng Lin-Gibson from NIST and Richard
Laforest and Anne Stroncek from MIR for their assistance

• Mention of commercial products should not be construed as either an

actual or implied endorsement of such products by D-HHS

N Petrick PCF Presentation 57

May 14, 2010