Persons using assistive technology might not be able to fully access information in this file. For assistance, please send e-mail to: mmwrq@cdc.gov. Type 508 Accommodation and the title of the report in the subject line of e-mail.
Appendix C
Conducting Studies To Evaluate Chlamydia
trachomatis and Neisseria gonorrhoeae Tests, Including Studies Required by the Regulations of
the Clinical Laboratory Improvement Amendments of 1988 for Verifying
or Establishing Test Performance Characteristics
When a test is a candidate to replace an established test, the sensitivities and specificities of the candidate and established
tests should be compared. Recommendations for the conduct of test evaluation studies have been reviewed elsewhere
(C-1--C-4). The imperfect sensitivity and specificity of reference standards complicate evaluation of
C. trachomatis and N. gonorrhoeae
tests. Discrepant analysis has been used universally to address this problem in the past. However, this approach has been criticized
because of the likelihood of producing biased estimates of test characteristics
(C-5--C-8). Key to advancing the evaluation of
C. trachomatis and N. gonorrhoeae tests is the introduction of study designs that compare candidate and established tests in the same study by
using an independent reference standard or by using approaches that do not require designation of a reference standard. Investigation
of such designs for evaluation of C.
trachomatis tests has begun
(C-6,C-9--C-15).
The Clinical Laboratory Improvement Amendments of 1988 (CLIA) requires that a clinical laboratory verify or establish
the performance of a test to be used in the medical care of persons when it is introduced into that laboratory
(C-16). Verification of performance under CLIA applies to the introduction of tests that are Food and Drug Administration (FDA)-cleared, and the
CLIA
requirement is that the laboratory demonstrate that test performance (i.e., accuracy, precision, reportable range of results,
and appropriate normal values) in that laboratory be equivalent to the performance claimed by the manufacturer in the
FDA-cleared package insert. Establishing performance under CLIA applies to tests or modifications of tests that have not been FDA-cleared for
a particular use. When a modification or use of a test has not been FDA-cleared, CLIA requires that the laboratory establish
the performance characteristics of the test for that use. In addition to the performance specifications noted previously,
establishing performance must include determinations of analytical sensitivity and specificity, to include interfering substances.
Such verification or establishment studies are relatively straightforward when the candidate test performs similarly to the
established test. In such cases, the laboratory can verify or establish performance by using the established test as the reference standard
if discrepancies between the candidate and established test are uncommon. If the candidate test performs differently from
the established test, the laboratory is faced with the economic and logistical challenge of verifying or establishing the performance of
the candidate test by introducing an independent reference standard. If the laboratory lacks an independent reference standard,
a reference laboratory with an independent reference standard might be enlisted
to 1) blindly test a suitable sample of verification
or establishment study specimens with the reference standard tests or 2) supply a panel of specimens previously characterized by
the reference laboratory. The reader is referred to the American Society for Microbiology CUMITECH series for detailed
guidance regarding conduct of verification and establishment studies
(C-17).
References
C-1. Irwig L, Tosteson ANA, Gatsonis C, et al. Guidelines for
meta-analyses evaluating diagnostic tests. Ann Intern Med 1994;120:667--76.
C-2. Noe DA. Evaluation of the medical utility of classification studies [Chapter 4]. In: Noe DA, Rock RC, Grochow LB, Kahn SB, Perlin E,
eds. Laboratory medicine: the selection and interpretation of clinical
laboratory studies. Baltimore, MD: Williams & Wilkins, 1994:44--52.
C-3. Noe DA. Diagnostic classification [Chapter 3]. In: Noe DA, Rock RC, Grochow LB, Kahn SB, Perlin E, eds. Laboratory medicine: the
selection and interpretation of clinical laboratory studies. Baltimore, MD: Williams & Wilkins, 1994:27--43.
C-4. Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research: getting better but still not good.
JAMA 1995;274:645--51.
C-5. Hadgu A. Bias in the evaluation of DNA-amplification tests for detecting
Chlamydia trachomatis. Statist Med 1997;16:1391--9.
C-6. Miller WC. Bias in discrepant analysis: when two wrongs don't make a right. J Clin Epidemiol 1998;51:219--31.
C-7. Lipman HB, Astles JR. Quantifying the bias associated with use of discrepant analysis. Clin Chem 1998;44:108--15.
C-8. McAdam AJ. Discrepant analysis: how can we test a test? J Clin Microbiol 2000;38:2027--9.
C-9. Alonzo TA, Pepe MS. Using a combination of reference tests to assess
the accuracy of a new diagnostic test. Statist Med 1999;18:2987--3003.
C-10. Miller WC. Can we do better than discrepant analysis for new diagnostic test evaluation? [Editorial response]. Clin Infect Dis 1998;27:1186--93.
C-11. Johnson RE, Green TA, Schachter J, et al. Evaluation of nucleic acid amplification tests as reference tests for
Chlamydia trachomatis infections in asymptomatic men. J Clin Microbiol 2000;38:4382--6.
C-12. Hadgu A, Qu Y. Biomedical application of latent class models with random effects. Appl Statist 1998;47(Pt 4):603--16.
C-13. Qu Y, Hadgu A. Model for evaluating sensitivity and specificity for correlated diagnostic tests in efficacy studies with an imperfect
reference test. Journal of the American Statistical Association 1998;93:920--8.
C-14. Cheng H, Macaluso M, Vermund SH, Hook EW III. Relative accuracy of nucleic acid amplification tests and culture in detecting
Chlamydia in asymptomatic men. J Clin Microbiol 2001;39:3927--37.
C-15. Black CM, Marrazzo J, Johnson RE, et al. Head-to-head multicenter comparison of DNA probe and nucleic acid amplification tests for
Chlamydia trachomatis in women with use of an improved reference standard. J Clin Microbiol (in press).
C-16. Code of Federal Regulations. Title 42, Chapter IV, part 493---Laboratory requirements.
C-17. Elder BL, Hansen SA, Kellogg JA, Marsik FJ, Zabransky RJ. Verification and validation of procedures in the clinical microbiology laboratory
[31]. In: McCurdy BW, ed. CUMITECH: cumulative techniques and procedures in clinical microbiology. Washington, DC: American Society
for Microbiology, 1997.
Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of
Health and Human Services.References to non-CDC sites on the Internet are
provided as a service to MMWR readers and do not constitute or imply
endorsement of these organizations or their programs by CDC or the U.S.
Department of Health and Human Services. CDC is not responsible for the content
of pages found at these sites. URL addresses listed in MMWR were current as of
the date of publication.
Disclaimer
All MMWR HTML versions of articles are electronic conversions from ASCII text
into HTML. This conversion may have resulted in character translation or format errors in the HTML version.
Users should not rely on this HTML document, but are referred to the electronic PDF version and/or
the original MMWR paper copy for the official text, figures, and tables.
An original paper copy of this issue can be obtained from the Superintendent of Documents,
U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800.
Contact GPO for current prices.
**Questions or messages regarding errors in formatting should be addressed to
mmwrq@cdc.gov.
