20 E P

CM

14 rog
J M ram
D
The Journal of Molecular Diagnostics, Vol. 16, No. 3, May 2014

jmd.amjpathol.org

PERSPECTIVES
Methods-Based Prociency Testing in Molecular Genetic
Pathology
Iris Schrijver,*y Nazneen Aziz,z Lawrence J. Jennings,x{ Carolyn Sue Richards,k Karl V. Voelkerding,**yy and Karen E. Weckzzxx

From the Departments of Pathology* and Pediatrics,y Stanford University Medical Center, Stanford, California; the Transformation Program Ofce,z College
of American Pathologists, Northeld, Illinois; the Department of Pathology and Laboratory Medicine,x Ann & Robert H. Lurie Childrens Hospital of
Chicago, Chicago, Illinois; the Department of Pathology and Laboratory Medicine,{ Northwestern University Feinberg School of Medicine, Chicago, Illinois;
the Department of Molecular and Medical Genetics,k Oregon Health & Science University, Portland, Oregon; the Department of Pathology,** University
of Utah School of Medicine, Salt Lake City, Utah; the ARUP Laboratories Institute for Clinical and Experimental Pathology,yy Salt Lake City, Utah;
and the Departments of Pathology and Laboratory Medicinezz and Genetics,xx University of North Carolina, Chapel Hill, North Carolina

CME Accreditation Statement: This activity (JMD 2014 CME Program in Molecular Diagnostics) has been planned and implemented in accordance with the Essential Areas and
policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Society for Clinical Pathology (ASCP) and the
American Society for Investigative Pathology (ASIP). ASCP is accredited by the ACCME to provide continuing medical education for physicians.

The ASCP designates this journal-based CME activity (JMD 2014 CME Program in Molecular Diagnostics) for a maximum of 48 AMA PRA Category 1 Credit(s)
. Physicians
should only claim credit commensurate with the extent of their participation in the activity.

CME Disclosures: The authors of this article and the planning committee members and staff have no relevant nancial relationships with commercial interests to disclose.

What Is Methods-Based Prociency Testing? analyte-specic testing in a practice environment of large-scale

Prociency testing (PT) is intended to be an external measure
of clinical laboratory quality.1 In the United States, it is a
requirement of accreditation by the Centers for Medicare and
Medicaid Services. It is part of a quality assurance program to
verify the accuracy and reliability of laboratory testing. Lab-
oratories in the United States are certied under the Clinical
Laboratory Improvement Amendments and accredited by
professional organizations with deemed status, such as the
College of American Pathologists (CAP). Participation in
external quality assessment (EQA) may be through CAP PT
programs or through another prociency testing provider
accepted by the Clinical Laboratory Improvement Amend-
ments. For those analytes that do not have EQA surveys
available, laboratories must implement an alternative EQA
(PT) assessment procedure. Appropriate alternative perfor-
mance assessment procedures may include split sample anal-
ysis with other laboratories, or, if that is not available,
assessment of split samples with an established in-house
method and previously assayed material, which are run and
interpreted by laboratory personnel who do not have access to
the prior results.2 With the increase in diagnostic sequencing,
however, the list of genes for which CAP PT is not available is
rapidly growing, and it is virtually impossible to continue
sequencing of patients DNA.
Methods-based prociency testing (MBPT) is a subset of
overall PT and refers to an EQA approach that is based on
method, rather than based on each individual analyte tested.
MBPT is already well established for several pathology sub-
specialty areas, and the CAP offers a variety of PT products that
are wholly methods based, for example, in cytogenetics, ow
cytometry, and immunohistochemistry. Until recently, how-
ever, PT in the area of molecular testing has been entirely an-
alyte specic. The CAP supports an MBPT approach for
clinical molecular genetic testing, and the Centers for
Medicare and Medicaid Services agrees with the concept of
MBPT in molecular diagnostics, taking into account the cost
and difculty in obtaining materials for the many different
possible mutations in any given gene. Thus, the concept
of MBPT complies with federal laboratory regulations.
Although MBPT in molecular diagnostics is currently
Accepted for publication February 11, 2014.
Disclosures: None declared.
Current address of N.A., Phoenix Childrens Hospital, Phoenix, AZ.
Address correspondence to Iris Schrijver, M.D., Department of Pathol-
ogy, L235, Stanford University Medical Center, 300 Pasteur Dr., Stanford,
CA 94305. E-mail: ischrijver@stanfordmed.org

Copyright 2014 American Society for Investigative Pathology

and the Association for Molecular Pathology.
Published by Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jmoldx.2014.02.002
Schrijver et al
limited to inherited genetic conditions, in the future, mo-
lecular cancer testing will probably include MBPT as well.
Methods-based approaches are not limited to PT, but they
also affect other aspects of developing and implementing
laboratory testing. Several published guidelines and exam-
ples describe the steps involved in validating a molecular test,
but they focus on analyte-specic tests.3e11 It is important to
note that these same validation principles can also apply to
methods-based testing. In fact, several molecular tests may be
better approached through method-based validation rather
than through analyte-specic validation, particularly when it
may be impractical or even impossible to obtain positive
samples of all targeted mutations. All validations should
address the same critical question: given the intended use, is
the test t for its purpose? To answer this question, labora-
tories must assess the potential sources of error and must
provide documented evidence that the test will consistently
meet the predetermined analytical requirements. For example,
validation of a new DNA extraction method may only require
documentation that the method yields DNA of sufcient
quantity, purity, and integrity for anticipated downstream ap-
plications. More complex methods, however, may also be
amenable to method-based validation, at least for those
processes that have been standardized. Multiplex ligation-
dependent probe amplication is an example of a method that
involves several steps, but these steps have been standardized
across probe sets. After validating the analytical performance of
the test system and verifying equivalent performance across
multiple probe sets, the validation of a new probe set may
be limited to actual probe validation.12 Similarly, aspects of
chromosomal microarray testing, Sanger sequencing, and next-
generation sequencing (NGS) are amenable to method-based
validation. The initial validation of the test system may be
quite involved, but the addition of another targeted genomic
region or gene sequence may only require probe or primer
validation and a determination of the normal reference range.
What Are the Advantages and Limitations of
MBPT?
The advantages of MBPT are in part practical, because it is
logistically challenging to develop separate prociency chal-
lenges for each of the approximately 20,000 human genes that
may be analyzed. More importantly, a widespread MBPT
program in which all laboratories conducting nucleic acid
sequencing may participate provides better comparisons of
performance among laboratories to efciently and thoroughly
test the important aspects of analysis and interpretation that are
central to laboratory performance. For example, the ability of
laboratories to detect different types of mutations, including
single nucleotide variants, insertions and deletions, and muta-
tions in GC-rich genomic regions and to recognize technical
reasons for false positive or false negative results (such as allele
dropout) can systematically be tested by an MBPT program.
MBPT testing can assess prociency much more efciently and
284
effectively for thousands of individual genes than a disease-
specic PT approach, in which the PT for each gene would
incorporate numerous challenge events. In addition, MBPT is
not analyte specic; therefore laboratories performing rare tests
can participate in an EQA program, provided they are using
comparable methodologies. By participating, the performance
of laboratories performing rare genetic tests can be evaluated
and even graded. MBPT for ultra-rare disorders (<2000 cases
prevalence) has already been recommended by the American
College of Medical Genetics and Genomics,13 but, in light of
the paradigm shift that results from widespread NGS-based
genomic testing, the CAP extends this option to less rare con-
ditions, as well. Of note, MBPT is only acceptable if provided
by an external PT provider, as opposed to an internal scheme
designed by individual laboratories.
Although there are clear advantages to the integration of
MBPT, laboratories are also required to participate in analyte-
specic PT, if available, because specic aspects of test per-
formance and interpretation can be optimally addressed by PT
designed for the gene(s) or disease process being analyzed.
Such aspects include the assessment of competency about the
identication and interpretation of specic mutations in a given
gene, as opposed to a more general nomenclature-specic
competency. Thus, laboratories should subscribe to all formal
assay/analyte-specic PT available for the assays they perform,
because this evaluates gene- and mutation-specic competency.
MBPT also has limitations compared with analyte-based PT.
Traditional PT challenges have attempted to replicate clinical
samples to assess performance of the entire test system. MBPT,
by contrast, is focused on a step or process within the test
system and may fail to assess other error-prone steps. For
example, MBPT may be focused on the analytical step rather
than preanalytical steps (eg, specimen processing and nucleic
acid extraction) or the postanalytical steps (eg, interpretation
and reporting). However, these challenges are not exclusive to
MBPT. Even traditional PT samples sacrice authenticity to
provide samples that are stable, traceable, and homogenous, or
which target rare, well-characterized mutations. Therefore,
most PT samples, whether traditional or method-based, are
variably articial and cannot assess every single individual step
in a process. Nevertheless, PT samples provide an external
measure of quality and therefore are a critically important part
of a laboratorys quality assurance program. The mentioned
limitations are considered acceptable, provided the laboratory
monitors error-prone steps in the test system in other ways
(eg, quality controls, monitoring prevalence of mutations in a
population, report review, etc), and these processes are
incorporated in CAP molecular pathology checklists and
reviewed during on-site peer-reviewed inspections.
Why Is the Time Right for Implementing
MBPT in Molecular Genetic Pathology?
The rationale for implementing MBPT for molecular
testing addresses several key aspects of sequencing-based
jmd.amjpathol.org - The Journal of Molecular Diagnostics

technologies that are unique compared with other types of
molecular testing: i) many clinical laboratories are currently
performing genomic sequencing analysis for a variety of
multigene panels by using similar (Sanger or NGS-based)
methods, even though the specic gene(s) tested may be
different; ii) important aspects of the technical performance
and annotation of sequence variants are key to laboratory
prociency and can be tested systematically by MBPT but
may be agnostic to the specic gene(s) being tested; iii) the
ability of a laboratory to design, validate, perform, and
interpret certain types of genomic sequencing analysis is not
wholly dependent on the specic gene(s) or disease process
being tested but rather on their ability to conduct the type of
sequencing analysis in its totality; and iv) an MBPT program
can be efciently designed to test the ability of laboratories to
analyze the wide spectrum and types of results that are crucial
to procient laboratory performance in genomic sequencing
analysis.
In 2010, a methods-based Sequencing Educational
Challenge (SEC) survey was launched by the Biochemical
and Molecular Genetics Committee of the CAP and the
American College of Medical Genetics and Genomics. Of
note, this was preceded (by years) by European laboratories
MBPT that involved many countries.14 Subsequently these
SEC surveys have been developed and evaluated by the
Biochemical and Molecular Genetics Committee. In the
initial 2 years, participating laboratories interpreted elec-
tronic sequence traces in a variety of disclosed genes for
which these laboratories did not necessarily offer testing.
With the use of the information provided with these dry
challenges, laboratories applied the same methods for
analysis and interpretation as they would for those genes
they routinely sequence. The purpose of the CAP dry
sequencing challenges is to test laboratory performance by
MBPT, to assess analytical capability, as well as the clinical
interpretation component for the diversity of sequencing
results that diagnostic laboratories may encounter when
testing by Sanger sequencing methods. Laboratories
participating in the SEC survey (http://www.cap.org/apps/
docs/prociency_testing/2014_surveys_catalog.pdf, last
accessed January 2, 2014) each received a compact disk that
included three sequence data les that comprise the PT
challenge; three normal sequence data les; additional in-
formation that includes the gene name, the genomic, and the
coding RefSeq numbers; a predicted protein translation of
the sequence; a Mutation Surveyor (SoftGenetics, State
College, PA) sequence analysis le; web-based references
to the gene-specic databases and single nucleotide poly-
morphism databases (National Center for Biotechnology
Information Entrez SNP Database; http://www.ncbi.nlm.nih.
gov/snp?db, last accessed February 6, 2013); and references
to Human Genome Variation Society nomenclature rules
(http://www.hgvs.org/mutnomen, last accessed February 6,
2013).15 The laboratories were directed to use standard
Human Genome Variation Society nomenclature to report
identied sequence changes. In addition, laboratories were
The Journal of Molecular Diagnostics - jmd.amjpathol.org
Methods-Based Prociency Testing
asked to include all variants identied, to indicate zygosity
for each, and to provide a basic interpretation for each
variant. Interpretations include the categories of pathogenic,
benign, or of unknown clinical signicance, according to the
guideline of the American College of Medical Genetics and
Genomics Interpretation of Sequence Variants.16 After 3
years of the SEC survey,17 it is clear that MBPT for mo-
lecular genetic testing can be successfully developed and
implemented and that the feedback to laboratories from this
survey has resulted in improved and more consistent inter-
pretation of sequence variants.
The original CAP PT program for sequencing was
expanded in 2013 to include a wet lab MBPT challenge
(SEC-1), which was designed to test not only analytical and
clinical interpretation but also the technical component of
Sanger sequencing. This survey includes three DNA spec-
imens and primers for sequencing, in addition to the mate-
rials listed above that are supplied in the dry survey.
In SEC-1 the laboratories are asked to both generate the
sequence and to interpret results for nucleotide change,
zygosity, predicted protein change, and predicted pathoge-
nicity. Participation in the wet survey is encouraged for
laboratories that perform sequencing in house, whereas
laboratories that send out the sequencing but interpret the
subsequently received data are encouraged to subscribe to
the dry survey. The Biochemical and Molecular Genetics
Committee will evaluate the performance of laboratories
participating in this survey, going forward.
Because MBPT is a reasonable and logical approach in the
era of genomic medicine, CAP has recently initiated the
development of a pilot PT for a methods-based NGS PT
product in which a healthy persons genome, referred to as the
CAP genome, has been extensively sequenced by the three
commonly used commercial NGS platforms. Laboratories will
be evaluated for their prociency to correctly call single
nucleotide variants, insertions, deletions, and nonvariant nu-
cleotides in this genome. These MBPT pilot studies address
multiple key aspects of the NGS workow, such as the indi-
vidual steps of sequence read generation, sequence mapping,
alignment, variant calling, and annotation of pathogenicity.
They are expected to be completed and evaluated in early 2014.
In Europe, the European Molecular Genetics Quality
Network and the UK National External Quality Assessment
Scheme for molecular genetics recently launched a pilot MBPT
for NGS. Thirty laboratories were selected and received a well-
characterized genome. Individual laboratories were advised to
run their smallest panel or their largest gene. This scheme was
specically designed to assess the technical ability of labo-
ratories that perform NGS-based diagnostic testing, focusing
on the sequence quality rather than assessing prociency of
accurate variant calls. The NGS EQA scheme has been pre-
sented at several conferences but has not yet been published.
Currently, the scheme report is available only to European
Molecular Genetics Quality Network NGS PT participants (S.
Patton, personal communication, Director, European Molec-
ular Genetics Quality Network).
285
Schrijver et al
With the rapid adoption of sequencing-based clinical testing,
the number of disease-causing variants in almost every area of
medicine to which molecular diagnostic testing is applied is
growing and is projected to expand. In genome analysis, the
number of variants per person is approximately three million.
For exomes, that number approximates 15 to 20,000.18 With
any NGS test, a large number of novel variants will be
discovered, some of which will be disease causing. For
inherited diseases our earlier understanding of a single gene
mutation for a single disease has evolved and concludes that
there are i) relatively few common disease-causing variants in a
single gene (eg, broblast growth factor receptor 3, which is
associated with, for example, achondroplasia); ii) multiple rare
disease-causing variants in a single gene (eg, predisposition to
breast cancer: breast cancer 1 or 2, early onset); iii) a few
common disease-causing variants in multiple genes associated
with a single disease (eg, cardiomyopathy and some of the
complex diseases such as type 2 diabetes); and iv) multiple rare
disease-causing variants in multiple genes associated with a
single disease (some common complex diseases such as
hypertension).
Therefore, disease-causing mutations will outnumber the
handful of genetic mutations that are currently typically
tested in clinical laboratories. The MBPT approach assesses
the expertise of laboratories to correctly perform a test
method and to accurately identify the types of variants
encountered during genomic analysis (single nucleotide
variants, insertions, deletions, duplications, inversions, ho-
mopolymers, repeated sequence, and chromosomal trans-
locations), rather than developing PT for each and every
known disease gene and its associated mutations. The
ability of the operator to correctly call the presence/absence
of different categories of variants provides a broad assess-
ment and helps determine whether the laboratory is able to
handle the subtleties related to identifying different types of
genetic mutations by using a specic assay method.
Development of MBPT for NGS
The total process of generating a diagnostic result by using
NGS is comprised of a wet bench workow to generate a
sequencing library and to sequence the library. There also
is a dry bench workow, which consists of analysis of
sequence reads and subsequent interpretation of variants
obtained. This multistep process is technically complex and
is being used for the diagnostic evaluation of multigene
panels, exomes, and genomes for both germline and somatic
variation. Clinical laboratories with experience in high-
complexity molecular testing are generally well suited for
implementing the wet bench methods and for operating
NGS instrumentation. In contrast, the analysis of NGS read
data sets is a new subject for clinical laboratories.19
In this environment, clinical laboratories may take several
approaches in the adoption of NGS. These approaches
include i) laboratories perform the total process within their
286
own setting; ii) laboratories process samples and generate
sequence data but then outsource the data analysis and
interpretation components; iii) laboratories outsource the wet
laboratory component but develop the informatics pipeline in
house; and iv) laboratories and software companies specialize
in providing data analysis and interpretation. In some sce-
narios, laboratories that outsource data analysis review the
analyzed data returned to them to arrive at a diagnostic
interpretation. These different approaches were considered
when the CAP NGS Working Group developed
NGS-specic accreditation requirements. Thus, the NGS
Working Group decided to develop separate accreditation
requirements for the wet bench (inclusive of sequencing) and
the bioinformatics data analysis components. This served the
pragmatic function of accommodating laboratories that pur-
sue different approaches with respect to implementing NGS.
By extension, EQA testing for NGS should also incor-
porate diverse implementation approaches. To address this,
the CAP NGS Working Group is developing an MBPT
challenge that will address the total process as described
above. Importantly, it should be noted that the different
approaches to the adoption of NGS (as listed above) pose a
challenge when the wet and the dry bench components are
performed at different addresses. In the United States, PT/
EQA referral is a violation of federal law, and under the
current rule it may not be possible to test the entire NGS
process under a single EQA challenge. Given that the
computing often does not occur at a physical address but
rather in the cloud, it will be interesting to follow how the
interpretation of this rule evolves in the future.
Separate dry NGS PT challenges are being discussed that
will focus on the bioinformatics component of the work-
ow. The dry challenges could be comprised of sequence
data sets (eg, FASTQ, BAM les) derived from a genome,
exome, or gene panel analysis spiked with in silico sequence
reads that contain articially created variants, for example,
single nucleotide variants or insertions and deletions. Such
PT will interrogate a laboratorys ability to properly align
and map sequence data and to identify and annotate vari-
ants. In silico dry challenges are particularly useful in mo-
lecular oncology testing or other mixed genotype sample
testing, including mosaicism and mitochrondrial hetero-
plasmy, in which the limit of detection of the assay is critical
for identifying low-frequency alleles. By admixing synthetic
sequence reads that contain mutations of interest at varying
ratios with a set of true sequence reads it will be possible to
assess a laboratorys analytical bioinformatics aptitude to
identify low-frequency mutations, which will be encoun-
tered in clinical patient samples. Other advantages of dry
bench testing are related to logistic issues associated with
creating wet bench PT samples for cancer. An ideal sample
would be a genomic DNA which harbors multiple cancer
mutations all in the same isogenic background. Such options
are being explored by some commercial vendors whereby
several different mutations are knocked-in by site-directed
mutagenesis within isogenic cell lines, and the DNA from
jmd.amjpathol.org - The Journal of Molecular Diagnostics

these cell lines, each harboring a different mutation, is
mixed. This is an area of active research and development.
Another area for NGS PT will be to assess the ability of
laboratories to distinguish disease-causing variants from
variants that are benign. Numerous variants are found in
all persons with any type of NGS testing (gene panels or
exome- or genome-level analysis); therefore laboratories
must evaluate all variants found in clinically relevant genes
and need to report on variants that may be causative. Bioin-
formatics algorithms reduce the variant list to a set of those
variants that are interpreted as causal or potentially causal
with respect to patient phenotype. Algorithms are used in a
series of ltering steps such as variant population frequency,
in silico tools for predicting protein function change, and
clinical variant database cross-referencing for previously
reported pathogenicity. In addition, laboratories may need to
identify and report on variants not directly associated with
patient phenotype, but present as incidental ndings,
depending on the laboratorys policies for analyzing inci-
dental ndings. Moreover, with the recent discovery that
apparently healthy persons harbor approximately 100 or
more deleterious mutations that cause loss of function within
protein coding regions of genes, the interpretive task is now
even more difcult than previously thought. In the era of
genomic medicine, interpretation of variants has been iden-
tied as one of the critical bottlenecks for translating
sequence information to clinical practice. Therefore, in the
future, PT challenges are needed to assess the laboratorys
ability to analyze and interpret known and novel variants.
In conclusion, in the rapidly evolving eld of molecular
pathology with an exponential increase in data generation,
MBPT is a rational extension of traditional EQA. MBPT is
one helpful tool to assess prociency of molecular diagnostic
laboratories that provide sequence-based testing and can be
used in conjunction with analyte-specic testing to ensure
optimal preanalytic, analytic, and postanalytic laboratory
performance. We consider MBPT to be the most efcient,
thorough, practical, and cost-effective method to measure
laboratory prociency in genome-based sequencing analyses.
References
1. Kalman L, Lubin IM, Barker S, du Sart D, Elles R, Grody WW,
Pazzagli M, Richards S, Schrijver I, Zehnbauer B: Current landscape
and new paradigms of prociency testing for molecular genetics. Arch
Pathol Lab Med 2013, 137:983e988
2. CLSI: Assessment of Laboratory Tests When Prociency Testing Is
Not Available; Approved Guideline, ed 2. CLSI document GP29eA2:
Wayne, PA: Clinical and Laboratory Standards Institute; 2008
3. Mattocks CJ, Morris MA, Matthijs G, Swinnen E, Corveleyn A,
Dequeker E, Mller CR, Pratt V, Wallace A; EuroGentest Validation
Group: A standardized framework for the validation and verication
of clinical molecular genetic tests. Eur J Hum Genet 2010, 18:1276e1288
4. Halling KC, Schrijver I, Persons DL: Test verication and validation
for molecular diagnostic assays. Arch Pathol Lab Med 2012, 136:
11e13
5. Jennings L, Van Deerlin VM, Gulley ML; College of American Pa-
thologists Molecular Pathology Resource Committee: Recommended
The Journal of Molecular Diagnostics - jmd.amjpathol.org
Methods-Based Prociency Testing
principles and practices for validating clinical molecular pathology
tests. Arch Pathol Lab Med 2009, 133:743e755
6. Jennings LJ, Smith FA, Halling KC, Persons DL, Kamel-Reid S;
Molecular Oncology Resource Committee of the College of American
Pathologists: Design and analytic validation of BCR-ABL1 quantita-
tive reverse transcription polymerase chain reaction assay for moni-
toring minimal residual disease. Arch Pathol Lab Med 2012, 136:
33e40
7. Kamel-Reid S, Zhang T, Persons DL, Nikiforova MN, Halling KC;
Molecular Oncology Resource Committee of the College of American
Pathologists: Validation of KRAS testing for anti-EGFR therapeutic
decisions for patients with metastatic colorectal carcinoma. Arch
Pathol Lab Med 2012, 136:26e32
8. Lacbawan FL, Weck KE, Kant JA, Feldman GL, Schrijver I; Bio-
logical and Molecular Genetic Resource Committee of the College of
American Pathologists: Verication of performance specications of a
molecular test: cystic brosis carrier testing using the Luminex liquid
bead array. Arch Pathol Lab Med 2012, 136:14e19
9. Pont-Kingdon G, Gedge F, Wooderchak-Donahue W, Schrijver I,
Weck KE, Kant JA, Oglesbee D, Bayrak-Toydemir P, Lyon E;
Biochemical and Molecular Genetic Resource Committee of the Col-
lege of American Pathologists: Design and analytical validation of
clinical DNA sequencing assays. Arch Pathol Lab Med 2012, 136:
41e46
10. Saxe DF, Persons DL, Wolff DJ, Theil KS; Cytogenetics Resource
Committee of the College of American Pathologists: Validation of
uorescence in situ hybridization using an analyte-specic reagent for
detection of abnormalities involving the mixed lineage leukemia gene.
Arch Pathol Lab Med 2012, 136:47e52
11. Schlaberg R, Mitchell MJ, Taggart EW, She RC; Microbiology
Resource Committee of the College of American Pathologists: Veri-
cation of performance specications for a US Food and Drug
Administration-approved molecular microbiology test: Clostridium
difcile cytotoxin B using the Becton, Dickinson and Company
GeneOhm Cdiff assay. Arch Pathol Lab Med 2012, 136:20e25
12. Jennings LJ, Yu M, Fitzpatrick C, Smith FA: Validation of multiplex
ligation-dependent probe amplication for conrmation of array
comparative genomic hybridization. Diagn Mol Pathol 2011, 20:
166e174
13. Maddalena A, Bale S, Das S, Grody W, Richards S; ACMG Labora-
tory Quality Assurance Committee: Technical standards and guide-
lines: molecular genetic testing for ultra-rare disorders. Genet Med
2005, 7:571e583
14. Ahmad-Nejad P, Dorn-Beineke A, Pfeiffer U, Brade J,
Geilenkeuser WJ, Ramsden S, Pazzagli M, Meumaier M: Methodo-
logic European external quality assurance for DNA sequencing: the
EQUALseq program. Clin Chem 2006, 52:716e727
15. den Dunnen JT, Antonarakis SE: Mutation nomenclature extensions
and suggestions to describe complex mutations: a discussion. Hum
Mutat 2000, 15:7e12
16. Richards CS, Bale S, Bellissimo DB, Das S, Grody WW, Hegde MR,
Lyon E, Ward BE; Molecular Subcommittee of the ACMG Laboratory
Quality Assurance Committee: ACMG recommendations for standards
for interpretation and reporting of sequence variations: Revisions 2007.
Genet Med 2008, 10:294e300
17. Richards CS, Palomaki GE, Lacbawan FL, Lyon E, Feldman GL;
CAP/ACMG Biochemical and Molecular Genetics Resource Com-
mittee: Three-year experience of a CAP/ACMG methods-based
external prociency testing program for laboratories offering DNA
sequencing for rare inherited disorders. Genet Med 2014, 16:25e32
18. Stitziel NO, Kiezun A, Sunyaev S: Computational and statistical ap-
proaches to analyzing variants identied by exome sequencing.
Genome Biol 2011, 12:227
19. Coonrod EM, Durtschi JD, Margraf RL, Voelkerding KV: Developing
genome and exome sequencing for candidate gene identication in
inherited disorders: an integrated technical and bioinformatics
approach. Arch Pathol Lab Med 2013, 137:415e433
287