Escolar Documentos
Profissional Documentos
Cultura Documentos
Development Center
______________________________________________________________
Class 1
Title
Application of Six Sigma Quality Tools to High Throughput and Combinatorial Materials
Development
Author(s)
J.N. Cawse
R. Wroczynski
Component
Report
Number
2001CRD055
Date
April 2001
Number
of Pages
12
Class
Key Words
Phone
(518)387-6095
8*833-6095
Six Sigma, combinatorial, high throughput, screening, Design for Six Sigma, quality,
experimentation, chemistry, materials development
As our ability to generate large numbers of experiments has accelerated, it has become more
important to ensure that the quality of each process step and individual sample is as high as possible.
There are too many samples for each data point to be checked individually, and too many process steps
for human oversight. To achieve a reasonable level of quality in the information from a high throughput
experimental system, the principles of modern Six Sigma manufacturing must be applied. The quality of
the end product is then ensured by the quality of each individual process step and the robustness of final
quality to variation in the process steps. Design for Six Sigma tools are particularly appropriate in this
effort.
Introduction
Background
Number of Reactions
Over the past ten years, the new research technology called Combinatorial Chemistry or High
Throughput Screening has seen exponential growth. This technology a set of techniques for creating a
multiplicity of compounds and then testing them for activity has been widely adopted in the
pharmaceutical industry over the past few years. Virtually every major drug manufacturer is now using
these techniques as the cornerstone of their research and development program. In the pharmaceutical
industry, libraries of 1000 to 1,000,000 distinct compounds are routinely created and tested for biological
activity. This is now practical because of the
convergence of low-cost computer systems,
25,000
reliable robotic systems, sophisticated molecular
modeling, statistical experimental strategies, and
database software tools.
20,000
Combinatorial Reactor
Conventional Reactors
15,000
10,000
5,000
0
7
8 8
8 9 9 9
7 8 8 8
t-9 ec-9 b-9 pr-9 n-9 g-9 ct-9 ec-9 b-9 pr-9 n-9
c
u
e
u
O D F A J A O D Fe A Ju
Critical to Quality (CTQ). Any product or service characteristic that satisfies a key customer
requirement or process requirement.
Defect opportunities. Every operation and every subsystem component is considered to be
a potential source of one or more defects.
Defects per million opportunities (DPMO). As actual or statistically probable defects are
found, they are divided by the opportunities. The ratio is multiplied by 1,000,000 to
accentuate the importance of even small numbers of defects.
Unit. An appropriate subdivision of the total effort is defined as a unit. In the present case,
a single 96-well sample array was chosen.
Defects per Unit (DPU). To calculate the total number of potential defects in the unit, each
DPMO is multiplied by the number of times it can occur in a single unit. The results are
totaled to give the DPU.
Z. The standard metric for measuring process capability. ZLT is based on the overall
standard deviation and average output. ZST represents the process capability with special
cause variation removed.
chemicals in making up the stock solutions for a run. An error in any weighing will affect 96 samples,
however, so any weighing defect will result in 96 sample defects. Therefore this step has 96 x 5 = 480
opportunities for defects. We assume that any human operation has 0.6% (6000 dpmo) chances of being
defective4, so the total number of defects potentially arising from this process are 480 x 6000 x 10-6 = 2.88
defects/array. Other sources add to this as shown in Figure 2, which is a highly abbreviated total from a
more complex system. It is easy to see that the number of defects/array can quickly become quite large.
Process Step
Make Stocks
Make Samples
React Samples
Analyze Samples
Source
Chem. Supplier
Technician
Robot
Heatup
Temp. Control
Cooldown
Gas Chromatograph
GC
GC
Defect
offspec chemicals
weigh chemicals
poor mixing
Time
Temp.
Time
Peak Identification
Endpoint Detection
Peak Integration
dpmo # times
1000
480
6000
480
3000
96
1500
96
5000
96
1500
96
100
96
10000
96
500
96
DPU
0.48
2.88
0.29
0.15
0.48
0.15
0.01
0.96
0.05
Design
Optimize
Validate
Quality by Design
Identify
End User
GE Business
Manufacturing
Operation
Final customer for
catalyst, not process
GE Business
Pilot Plant Team
Interface of
development with
end user
GE Corporate Research
and Development Catalyst
Development Team
Validation of combinatorial
development process and
laboratory scale test of
candidates.
GE Corporate Research
and Development
Combinatorial Team
Development of process for
catalyst development and
generation of catalyst leads
for further investigation.
Customers
Manufacturing
Manager
Manufacturing
Engineer
Customers
Process Engineer
Pilot Plant
Chemist
Customers
Catalyst Chemist
Development Engineer
Customers
Combinatorial Chemist
Microreactor Engineer
Data specialist
House
#2
Product
Deliverables
House
#3
Material
Process
Variables
HTS Variables
(HOW s)
CombiChem System
(WHAT s)
Customer
Needs
HTS System
Characteristics
(HOW s)
House
#1
Catalyst,
Materials, Process
(HOW s)
Product Deliverables
(WHAT s)
Customer Wants
(WHAT s)
Business Product
Deliverables
(HOW s)
House
#4
High
Throughput
Screening
System
Figure 5. The structure of the Quality Function Deployment Houses used for CTQ flowdown in high throughput
screening experimentation.
Variable Costs
Reaction Residuals
Resin Manufacturability
Total
Capping Level
Total
Resin Hydrostability
Resin Color
Manufacturing Throughput
Investment Costs
Total Cost
House 2
External Customer
Expectation
Importance
Quality Product
Importance
House 1
Quality Product
285
156
Total Cost
92
21
Business Product
Deliverables
49 41 41 17
86 78 56 54 54 54 54 40 30 26 22
Total
Figure 6. First and Second House QFD relating customer CTQs with measurable product and process deliverables.
Database System
Chemical
Reaction
Preparation
Analytical
Preparation
Chemical
Analysis
Data
Analysis
Total
230
582
m m
112
Reproducibility
m m m
380
Scaleability
m m
156
Total
N2 Flow Rate
Catalyst Concentration
Catalyst Delivery
Preparation Time
Reaction Time
Temp Profile
Order of Addition
m m
Temp. Uniformity
m m
Outlier ID
Solvent Choice
Monomer Reactivity
Catalyst Choice
Sample Tracking
Sampling Rates
Post-reaction Reaction
Diessolution Process
Slurry Delivery
Monomer Purity
Temp. Accuracy
Reagent Volatility
Accuracy
Overall Variability
Monomer Stability
CombiChem Factors
Monomer Ratio
Importance
HTS Variables
126 72 70 70 58 58 54 54 54 54 54 54 54 54 54 42 42 42 40 38 34 30 28 28 28 28 24 24 24 24 24 20
Preparation
Reaction
Work Up
Chemical
Analysis
Data
Analysis
Figure 7. A Fourth House QFD connecting the materials, catalyst, or process for desired changes.
screening (HTS) system (the combinatorial factory). Conversely, once the HTS system is in place, its
statistical capability to identify new leads can be related back (flowup) to the ability to reliably deliver
the Customer CTQs.
The next key step in utilizing the QFD is to focus on the design of the HTS system. This is an iterative
process linking the Customer CTQs with the chemical analysis system and the reactor design
specifications. Simply stated, both the analysis and the reaction system have to be sufficiently capable to
allow identification of leads or hits above the composite noise of the systems. However, the ability to
perform certain high throughput analyses is a function of the type of reaction and reactor employed and,
similarly, the reaction system can be configured to more easily allow rapid in-line or off-line analysis. This
iterative process occurs early in the HTS design process and fully utilizes the interactive matrix of
variables and responses delineated by the QFD process.
To fully understand the interaction of the analysis and reaction systems, a complete process map is
constructed for the steps involved in running the HTS factory. Figure 8 depicts a generalized process map
including the design and operation stages of a HTS project. These generalized process steps can be subdivided into more detailed actions or steps. Figure 9 highlights an example of detailed steps that
correspond to Prepare Arrays in Figure 8. The detailed process steps that are unnecessary or unwieldy
can be changed or eliminated. The remaining steps in the process map can then be linked to the variables
in the HTS house of the QFD. Process steps related to those elements of the HTS house which have the
greatest impact on the Customer CTQ's are then the focus of significant quality evaluation and refinement
to reduce sources of variability.
Factory
Design
Inputs
Chemical
System
Inputs
Ideas,
Chemicals
Process Design
Determine
Design Required
Construct
Prototype
Analytical System
Design
Decide on System
to Study
Prework
Process
Purchase
Chemicals
Setup, Process,
and Evaluate
Prepare
Arrays
Data
Analysis
System
Database
Validate Safety
and Operation
Evaluate and
Validate
Method
Plan Combinatorial
Strategy for
Exploration
Synthesis
Safety,
Solubility, and
Compatibility Tests
Define Stock
Solutions and
Coat Mixtures
Factory
Production
Output
Candidates
for Scale-up
Sample
Analysis
Process
Data
Analysis
Inferential
Engine
Figure 8. A generalized process map for design and operation of a High Throughput Experimental System.
Weigh
reagents for
stocks
Dissolve
reagents for
stocks
Weigh
catalyst
for stock
Add
solvent
for stock
Charge
catalysts to
HTS array
Remove
solvent
Secure tips
to 96 well
replicator
Charge reagents
to array with
replicator
Transport
array
to oven
Remove
solvent
Secure
pipette
tips
Charge initiator
to array
with replicator
Transport
array to
vac oven
Remove
solvent
Preparation
Chemical
Reaction
Analytical
Preparation
Chemical
Analysis
Data
Analysis
Database System
Specifications
Unlike a conventional factory, a high throughput experimental system has only information as its product,
so the specifications are on the quality of that information. Therefore, all specifications are expressed in
terms of variance (or, more conventionally, standard deviation). Specifications are not internally
determined; they are externally agreed upon by the customer and the process team, and should flow
logically from the measurements of the critical customer parameters (CTQs). In our catalyst
development case, the critical parameters were accurate and rapid identification of catalyst leads. In
addition, the desired catalyst activity was more than double the current level. From that, we agreed that a
25% increase in activity would represent a useful lead, and that we desired a 95% probability of correctly
detecting that lead.
CTQs
Accurate lead identification
Rapid lead identification
>100% increase in catalyst activity
Specification
95% probability of correctly detecting
a 25% increase in catalyst activity
This specification is on the whole system: the major program of the Design effort is to flow that
specification down to the process subsystems, then flow the variances of the subsystems back up to the
whole system. In short, the variance of the whole system is a function of the variances of the subsystems
(Figure 10).
Whole System
Preparation
Chemical
Reaction
Analytical
Preparation
Chemical
Analysis
Data
Analysis
Database System
Figure 10. The variance of the whole system is a function of the variances of the subsystems.
This flow up of variance can be done in two ways: by defects accounting or experimentally using nested
DOEs. The approaches are complementary and both were used in this case. The purpose of nested
experimentation is
to get a measurement of the total process variance, so it can be compared to the variance based
specification to see if the experimentation has a reasonable chance of success, and if so, to estimate
the required number of samples
to identify the most important sources of experimental variance so that they may be reduced with Six
Sigma MAIC (Measure Analyze Improve Control)7 projects.
Nested experiments should capture the variance resulting from normal operation of the process.
The purpose of defect accounting is to capture discrete sources of error not caught in the nested
experiment. This can include missed points, erroneously entered information, contaminated or mislabeled
chemicals, and so on. Defect accounting should capture the variance resulting from all abnormal events
in the process. These also become targets of MAIC projects.
Nested DOEs
As a set of samples moves through the high throughput factory, it undergoes a series of processes, each
of which contributes a certain amount of variance to the total. This leads to a situation where there are
multiple experimental units,8 and the variances from each unit must be calculated from a nested statistical
analysis.
Figure 11 shows a fairly typical process in a combinatorial factory. Four subprocesses are shown: making
stocks, making an array of samples from the stocks, reacting the array, and analyzing the samples. Each
has its own sources of variance. Careful planning and analysis of the data is necessary to ensure that each
subprocess is assigned its appropriate degrees of freedom and variance.
Determination of the effect of the variances of each step of the process requires a hierarchical design.
This initial step in variance determination should be a simple hierarchical design in which a single
representative system is replicated many times, with all of the subprocesses represented in the replication
(Figure 12). That will give an estimate of the overall variance of the system and the variance of the initial
subprocesses. The statistical analysis of a balanced system like the one shown is standard and is covered
in most texts 9 and statistical software packages.10 These texts and software often do not, however,
mention that the uncertainty in the estimates of variance at the upper levels of the hierarchy can be very
large!11
Process Step
Factors
Make Stocks
Make Array
Sample factors:
Amounts added
Total volume
React Array
Analyze Array
Reactor factors:
Temperature
Reaction tim e
Pressure
Gas composition
Analysis factors:
Peak identification settings
Integration settings
Replicate Stocks
Replicate Reaction Runs
Replicate Samples
Replicate Analyses
Defect Accounting
In defect accounting, the entire system and each subsystem is process mapped in detail. Each step is
examined by the Six Sigma Black Belt and the process expert for potential sources of defects. These can
be classified by type:
Parts. The quality level of components or raw materials (such as chemicals)
Process. The quality level of each unit operation (e.g. weighing, fluid transfer, or reaction).
Performance. The overall quality of subsystem performance vs specifications when all elements are
working correctly but normal variance is present.
Software. The quality of any software specifically written for the system.
The outcome of the process is illustrated in Figure 2, which shows a small part of a high throughput
screening system. Even with this small fraction of the overall opportunities for defects, we predicted 5.6
DPU.
Outcome of the Variance Flowup
The results of the nested experiments and defect accounting are combined to give an overall estimate of
the system quality. They are also used to pinpoint the largest sources of variance. Once this is done, we
can apply Six Sigma techniques to reducing that variance. Continued nested experiments and defect
accounting will allow us to track improvements in system quality until the system has reached a practical
quality level.
USL
Short Term
Long Term
ZST 1.8
ZST 27
ZLT 0.6
ZLT 12
.80
3.85
3.90
3.95
4.00
4.05
4.10
3.93
Short Term
Long Term
3.98
10
Control: Careful choice of the internal standards allows diagnosis of potential deterioration in
chromatographic behavior by tracking their relative areas using control charts.
Mix Ratios
Mixture Sample
Figure 16. The variation in composition of six repetitive measurements taken on 58 samples after mixing by aspiration.
LSL
USL
ZST 7.7
Short Term
Long Term
ZLT 7.5
0.90
1.00
Mix Ratio
11
Temp C
Validate
The most important validation for a HTS project is confirmation that the screening results predict
outcomes on the laboratory and pilot scale. For the catalyst project, 94% of the leads identified in high
throughput screening were confirmed at the laboratory (150 ml reactor) scale.
A second catalyst project had two important CTQs: high catalyst activity and low side-product formation.
In traditional studies, these two responses went hand-in-hand. That is, highly active catalysts gave high
amounts of side-product. Over 300 catalyst compositions were evaluated by HTS with an aggressive 24%
12
50
45
Frequency
35
30
25
20
15
Current Catalyst
40
10
5
Conclusion
References:
(1) Liu, D. R.; Schultz, P. G. Generating New Molecular Function: A Lesson From Nature. Angew. Chem. Int. Ed
1999, 38, 36-54.
(2) Symyx Technologies.http://www.symyx.com, (April 3)
(3) Harry, M. J. The Nature of Six Sigma Quality; Motorola University Press: Schaumburg, IL, 1988.
(4) The quality of routine manual operations (e.g . restaurant bills, prescription writing) tends to cluster at 4
Sigma or 6000 DPMO: Harry, M. J. The Vision of Six Sigma: A roadmap for Breakthrough; Sigma Publishing
Company: Phoenix, AZ, 1994, p19.26.
(5) Juran, J. M. In Jurans Quality Control Handbook; 4th ed.; Juran, J. M., Ed.; McGraw-Hill: New York, 1988; pp
6.4ff.
(6) Quality Function Deployment: 3-Day Workshop; American Supplier Institute: Dearborn, MI, 1997.
(7) Breyfogle, F. W. Implementing Six Sigma; John Wiley and Sons: New York, 1999.
(8) Milliken, G. A.; Johnson, D. E. Analysis of Messy Data; Van Nostrand Reinhold: New York, 1984.
(9) Montgomery, D. C. Design and Analysis of Experiments; John Wiley & Sons: New York, 1984.
(10) 12 ed.; Minitab, Inc, 1999.
(11) Hendrix, C.: S. Charleston, WV, 1997.
(12) The Temperature Handbook; Omega Engineering, Inc., 1995.
13
Distribution List
Corporate Research and Development
Research Circle
Niskayuna, NY 12309
M. Brennan
D. Buckley
J. Carnahan
J. Cawse
G. Chambers
B. Chisholm
K. Cueman
D. Dietrich
N. Doganaksoy
T. Early
V. Eddy-Helenek
W. Flanagan
J. Flock
J. Hallgren
C. Hansen
B. Johnson
T. Jordan
R. Kilmer
T. Leib
J. Lemmon
R. Mattheyses
R. May
C. Molaison
GE Plastics
5th and Avery Streets
Parkersburg, WV 26101
A. Berzinis
GE Capital
260 Long Ridge Road
Stamford, CT 06927
W. Morris
D. Olson
B. Pomeroy
R. Potyrailo
E. Pressman
T. Repoff
C. Sabourin
A. Schmidt
R. Shaffer
K. Shalyaev
O. Siclovan
J. Silva
G. Soloveichik
J. Spivack
C. Stanard
J. Teetsov
J. Webb
J. Wetzel
D. Whisenhunt
B. Williams
R. Wroczynski
GE Plastics
Highway 69 South
Mt. Vernon, IN 47620
S. Cooper
J. DeRudder
D. Whalen
GE Plastics
1 Plastics Avenue
Pittsfield, MA 01201
N. Izadi-Khalili
S. Clarke
2001CRD055
J.N. Cawse
R. Wroczynski
2001CRD055
April 2001