Test Method Validation and Verification

Page 1 of 9

Excerpts from CAP All Common Checklist, August 21, 2017 release

Inspector Instructions:

● Policies and procedures for the introduction of new tests, methods, or instruments

● Sampling of assay validation and verification studies with emphasis on tests introduced in the past two

years, especially high volume tests and tests with the highest risk to patients

● Sampling of patient reports for laboratory-developed assays

● Which laboratory tests or instruments have been implemented in the past two years, particularly those that

are not FDA-cleared/approved?

● Do you follow the manufacturer's instructions exactly for all FDA-cleared/approved diagnostic kits or

devices?

● How does your laboratory validate or verify assay performance prior to test implementation?

● How does your laboratory verify or establish reference intervals?

● How does your laboratory validate clinical claims made by the laboratory about LDTs?

● Select at least one validation or verification study performed for each type of instrument or method

introduced during the past two years.

● In addition, select assays for evaluation if recurrent problems have been identified in proficiency testing

results, quality control, competency assessment, or physician complaints regardless of how long the

assay has been in place.

● Review validation or verification records to confirm that appropriate studies were performed using an

adequate number of cases, and data were reviewed. If the data showed discordances or unacceptable

variations, investigate how they were resolved. If a study was not performed or is missing required

components, cite the appropriate related requirement(s) (e.g. COM.40300, COM.40350, COM.40400).

● Confirm that there is a written assessment of each component (accuracy, precision, interferences, etc.) of

the validation or verification studies with laboratory director (or qualified designee) approval prior to the

initiation of clinical testing. If the study assessment was not signed by the laboratory director or designee,

cite COM.40000.

**NEW** 08/17/2016

COM.30980 Waived Test Implementation and Approval Phase II

For each waived test, the laboratory follows manufacturer's instructions for the

introduction of the instrument or device and there are records that the test(s) is approved

for use by the laboratory director, or designee meeting CAP director qualifications, prior

to use in patient testing.

NOTE: Waived testing must be performed following the manufacturer's instructions as written.

If the laboratory modifies a waived test, the checklist requirements for high complexity testing

apply, including the requirements for validation of the method performance specifications.

The laboratory director's signature on the written test procedure may be used to show approval of

the test for use in patient testing.

Evidence of Compliance:

✓ Records of test approval

**REVISED** 08/21/2017

COM.40000 Method Validation and Verification Approval - Nonwaived Tests Phase II

For each nonwaived test, there is an evaluation of the test method validation or

verification study (accuracy, precision, etc.) signed by the laboratory director, or

designee meeting CAP director qualifications, prior to use in patient testing to confirm the

acceptability of the data and approve each nonwaived test for clinical use.

NOTE: This checklist requirement is applicable only to nonwaived tests implemented after June

Test Method Validation and Verification

Page 2 of 9

15, 2009; however, all nonwaived tests must have records of completed analytical validation or

verification, regardless of their implementation date.

If multiple identical instruments or devices are in use, there must be records showing that the

method performance specifications have been separately verified for each test and instrument or

device. The evaluation and data must clearly support the decision to approve the test for clinical

use.

The evaluation must include: 1) a written assessment of each component of the validation or

verification study, including the acceptability of the data; 2) a signed approval statement, such

as, "I have reviewed the verification (or validation) data for accuracy, precision, reportable range,

and reference interval studies (insert other components, as required) for the (insert instrument/

test name), and the performance of the method is considered acceptable for patient testing." If

data include discordant results, there must be a record of the discordance and investigation of

any impact on the approval of the test for clinical use.

If a validation or verification study (accuracy, precision, reportable range, etc.) was not performed

or is missing required components, the appropriate, related checklist requirements must also be

cited (e.g. COM.40300, COM.40350, COM.40400).

For an FDA-cleared/approved test, an evaluation of the verification data must address analytical

performance specifications, including analytical accuracy, precision, interferences, and reportable

range, as applicable.

In addition, for modified FDA-cleared/approved tests or LDTs, the evaluation must address

analytical sensitivity, analytical specificity, and any other parameter that is considered important,

to assure the analytical performance of a test (e.g. specimen stability, reagent stability, linearity,

carryover, and cross-contamination, etc. as appropriate and applicable).

If the laboratory makes clinical claims about its tests, the evaluation must address the validation

of these claims.

See the Method Performance Specifications section for details concerning validation and verification.

Evidence of Compliance:

✓ Evaluation of validation and verification studies with review and approval

Method Performance Specifications - Nonwaived tests

NOTE: This subsection on METHOD PERFORMANCE SPECIFICATIONS does not apply to waived

tests.

ANALYTICAL VALIDATION/VERIFICATION

Laboratories are required to perform analytical validation or verification of each nonwaived test, method, or

instrument system before use in patient testing, regardless of when it was first introduced by the laboratory,

including instruments of the same make and model and temporary replacement (loaner) instruments. There

is no exception for analytical validation or verification of tests introduced prior to a specific date. The

laboratory must have data for the validation or verification of the applicable method performance

specifications and retain the records as long as the method is in use and for at least two years after

discontinuation.

The method performance specifications (e.g. accuracy, precision, reportable range) must be validated or

verified in the location in which patient testing will be performed. If an instrument is moved, the laboratory is

responsible for determining that the method performance specifications are not affected by the relocation

process or any changes due to the new environment (e.g. refer to the manufacturer's manual regarding

critical requirements, such as set-up limitations, environmental conditions, etc.). The laboratory must follow

manufacturer's instructions for instrument set up, maintenance, and system verification. Separate

Test Method Validation and Verification

Page 3 of 9

requirements for verifying the performance of instruments and equipment to confirm that they function

according to expectations for the intended use and within the defined tolerance limits are found in the

Instrument and Equipment Maintenance and Function Checks section (COM.30550, COM.30600). Not all

method performance specifications apply to qualitative tests. For qualitative tests, the laboratory must verify

or establish the method performance specifications that are applicable and clinically relevant.

LABORATORIES SUBJECT TO US REGULATIONS:

● For unmodified FDA-cleared or approved tests, the laboratory may use information from manufacturers, or

published literature, but the laboratory must verify such outside information on accuracy, precision and

reportable range.

● For modified FDA-cleared or approved tests and laboratory-developed tests (LDTs), the laboratory must

establish accuracy, precision, analytical sensitivity, interferences, analytical specificity, and reportable range,

as applicable; data on interferences may be obtained from manufacturers or published literature, as

applicable.

LABORATORIES NOT SUBJECT TO US REGULATIONS:

● The laboratory must verify or establish analytical accuracy, precision, analytical sensitivity, analytical

specificity (interfering substances) and reportable range for each test. Laboratories may use information from

manufacturers, published literature, or studies performed in other laboratories, but should verify such outside

information, whenever practical.

LABORATORY-DEVELOPED TESTS:

For the purposes of interpreting the checklist requirements, a laboratory-developed test (LDT) is defined as

follows: A test used in patient management that has both of the following features:

1. The test is performed by the clinical laboratory in which the test was developed wholly or in part; AND

2. The test is neither FDA-cleared nor FDA-approved.

EMERGENCY USE AUTHORIZATION (EUA)

For laboratories subject to US regulations, an emergency use authorization (EUA) is the legal mechanism

used by the FDA to allow the use of an unapproved medical product (e.g. diagnostic device) or an

unapproved use of an approved medical product during an emergency to diagnose, treat, or prevent a

serious or life threatening disease condition caused by a chemical, biological, radiological, or nuclear agent

(CBRN).

A laboratory that uses an EUA assay may not be able to establish accuracy, precision, analytical sensitivity,

interferences, analytical specificity, and reportable range studies. Laboratories using an EUA assay must

follow the assay or test system's protocol without modification and document the alternative mechanism

employed to ensure accurate test results.

COM.40100 Intermittent Testing Phase II

When a test is put back into production, the following requirements must be met:

1. PT or alternative assessment performed within 30 days prior to restarting patient testing

2. Method performance specifications verified, as applicable, within 30 days prior to restarting patient

testing

3. Competency assessed for analysts within 12 months prior to restarting patient testing

NOTE: This requirement applies to tests that are taken out of production for a time (for example, seasonal

testing for influenza). A test is considered to be taken out of production when (1) patient testing is not offered

AND (2) PT or alternative assessment, as applicable, is suspended. It does not apply to situations where a

proficiency testing challenge is not performed due to a temporary, short-term situation, such as a reagent

back order or an instrument breakdown. In those situations, the laboratory must perform alternative

assessment for that testing event.

The laboratory should have written procedures for putting intermittent tests into production.

For tests for which PT is required by CAP, if a PT challenge is not offered during the 30-day

Test Method Validation and Verification

Page 4 of 9

period prior to restarting patient testing, the laboratory may perform an alternative assessment

of the test. The laboratory must participate in the next scheduled PT event, if the Laboratory

Accreditation Program requires external PT for that analyte.

COM.40200 LDT and Modified FDA-cleared/approved Test List Phase I

The laboratory maintains a list of laboratory-developed tests (LDTs) and modified FDA

cleared/approved tests implemented by the laboratory.

NOTE: The list must include tests developed in-house, and for laboratories subject to US

regulations, tests using analyte-specific reagents (ASRs), and FDA-cleared/approved tests that

have been modified by the laboratory. A form is available on the CAP website that may be used for

maintaining this list and can be downloaded from the CAP website (http://www.cap.org) through e-LAB

Solutions Suite.

COM.40250 Manufacturer's Instructions Phase II

The laboratory follows manufacturer's instructions or provides validation records if the

test has been modified.

NOTE: Following manufacturer's instructions includes performing quality control, calibration, calibration

verification, and related functions as applicable to the scope of testing. Reagents, fluids, and disposable

materials supplied by the laboratory must meet the specifications in the instructions.

If the laboratory modifies manufacturer's instructions, the test is no longer an FDA-cleared/approved test,

and the modification(s) must be validated by the laboratory. Changes in the specimen type or collection

device are examples of common modifications (see "modification of manufacturer's instructions" in the

definition of Terms). Additional requirements for validation/verification may be found in the discipline-specific

checklists.

For waived and moderately complex tests, if manufacturer instructions are modified,

requirements for high complexity testing apply.

Evidence of Compliance:

✓ Validation records of established performance specifications (accuracy, precision, analytical

sensitivity, analytical specificity, interferences, reference interval(s), and reportable range) of

any test that has been modified.

REFERENCES

1) Department of Health and Human Services, Centers for Medicare and Medicaid Services. Clinical laboratory improvement

amendments of 1988; final rule. Fed Register. 2003(Jan 24): [42CFR493.1252]

**REVISED** 08/21/2017

COM.40300 Accuracy Phase II

The laboratory verifies or establishes analytical accuracy for each test using a sufficient

number of characterized samples.

NOTE: Accuracy is verified or established by comparing results to a definitive or reference

method, or an established comparative method. Use of matrix-appropriate reference

materials, patient specimens (altered or unaltered), or other commutable materials with known

concentrations or activities may be used to verify or establish accuracy. The use of routine quality

control materials or calibrators that were used to calibrate the method is not appropriate.

Evidence of Compliance:

✓ Written procedure for determining method performance characteristics, including accuracy

AND

✓ Records of verification or establishment of accuracy for each test

Test Method Validation and Verification

Page 5 of 9

**NEW** 08/21/2017

COM.40310 Precision Phase II

The laboratory verifies or establishes analytical precision for each test using a sufficient

number of characterized samples with repeated analysis.

NOTE: Precision is verified or established by repeat measurement of samples at varying

concentrations or activities within-run and between-run over a period of time.

Evidence of Compliance:

✓ Written procedure for determining method performance characteristics, including precision

AND

✓ Records of verification or establishment of precision for each test

COM.40350 Accuracy - Modified FDA-cleared/approved and LDTs Phase II

For modified FDA-cleared/approved and laboratory-developed tests (LDTs), validation of

analytical accuracy includes testing with an appropriate number of samples.

NOTE: An appropriate number of samples is defined as the following:

● For quantitative tests, a minimum of 20 samples with analyte concentrations distributed across the

analytical measurement range should be used. Proportionate mixtures of samples may be used to

supplement the study population.

● For qualitative tests, a minimum of 20 samples, including positive, negative, and low positive samples with

concentrations near the lower level of detection should be used; equivocal samples should not be used.

● For certain methods that test multiple analytes (e.g. next-generation sequencing, FISH, HPLC, GC-MS,

MALDI-TOF, etc.), analytic accuracy may be established for each method (not necessarily each analyte), as

appropriate.

If the laboratory uses fewer samples, the laboratory director must record the criteria used to determine the

appropriateness of the sample size. In many cases, a validation study with more samples is desirable.

For modified FDA-cleared/approved tests and LDTs in use prior to July 31, 2016, for which limited validation

studies are recorded, ongoing data supporting acceptable test performance may be used to meet the above

minimum sample requirement, unless the laboratory director has recorded the criteria used to determine the

acceptability of a smaller sample size. Examples of such ongoing data include records of proficiency testing,

alternative performance assessment, and quality control.

This checklist requirement does not apply to LDTs that employ the following methods:

● Manual microscopy (e.g. histopathologic and cytologic interpretation, microscopic

examination of blood or body fluids, Gram stains)

● Conventional microbiologic cultures and disc/broth/tube susceptibility studies

Evidence of Compliance:

✓ Written procedure for determining method performance characteristics, including accuracy

AND

✓ Records of establishment of analytical accuracy for each test

COM.40400 Analytical Sensitivity - Modified FDA-cleared/approved and LDTs Phase II

For modified FDA-cleared/approved tests or laboratory-developed tests (LDTs), the

laboratory establishes the analytical sensitivity (lower detection limit) of each assay, as

applicable.

Evidence of Compliance:

✓ Written procedure for determining method performance characteristics, including analytical

sensitivity AND

✓ Records of establishment of analytical sensitivity for each assay

Test Method Validation and Verification

Page 6 of 9

COM.40450 Analytical Specificity - Modified FDA-cleared/approved and LDTs Phase II

For modified FDA-cleared/approved tests or laboratory-developed tests (LDTs), the results

of each validation study include a sufficient number of samples to establish the test's analytical

specificity.

NOTE: The analytical specificity refers to the ability of a test or procedure to correctly identify or quantify an

entity in the presence of interfering or cross-reactive substances that might be expected to be present.

Laboratories are encouraged to review the published literature for guidance and provided confidence

intervals to estimated analytical specificity.

Evidence of Compliance:

✓ Records of validation studies and published references used to establish analytical specificity

COM.40500 Analytical Interferences Phase II

The laboratory understands the analytical interferences for each test, and has an

appropriate plan of action when they are present.

NOTE: Interfering substances may pose a significant problem to the clinical laboratory and healthcare

providers who may be misled by laboratory results that do not reflect patient clinical status. The laboratory

must be aware of common interferences by performing studies or referencing studies performed elsewhere

(such as by the instrument-reagent manufacturer).

Evidence of Compliance:

✓ Written procedure for determining method performance characteristics, including analytical interferences

AND

✓ Document listing known interferences for each test and plan of action when they are present

**REVISED** 08/21/2017

COM.40600 Reportable Range Phase II

The reportable range is verified or established for each analytical procedure before

implementation.

NOTE: The reportable range of an assay is the range of values that the laboratory reports for that assay. The

analytical measurement range (AMR) is the range of analyte values that a method can directly measure on

the specimen without any dilution or concentration. In some cases, clinically relevant limits may be narrower

than the analytical measurement range, and the clinically relevant limit is used as the limit for the reportable

range.

Expanded definitions and details of the AMR are provided in some of the section-specific checklists (e.g.

Chemistry). Verification of the AMR may not apply to certain assays (for example, in immunology and

coagulation).

Evidence of Compliance:

✓ Written policy for determining method performance characteristics, including reportable range

AND

✓ Records of verification or establishment of reportable ranges for each test

COM.40610 Calibration and Quality Control Procedures - Modified FDA-cleared/

approved and LDTs Phase II

For laboratory-developed tests and modified FDA-cleared/approved tests, the laboratory defines

written procedures for calibration and quality control based on the studies performed to evaluate the

method performance specifications.

NOTE: The procedures must define the frequency, number, and concentration of calibrators and controls to

be used.

Test Method Validation and Verification

Page 7 of 9

REFERENCES

1) Department of Health and Human Services, Centers for Medicare and Medicaid Services. Clinical laboratory improvement

amendments of

**REVISED** 08/17/2016

COM.40620 Body Fluid Analysis Phase II

Methods for body fluid analysis have been validated or verified and metrics for interpretation have

been established.

NOTE: This requirement applies directly to body fluid testing that the laboratory offers as a routine, orderable

test. If the test is routinely performed on the fluid, there must be a written procedure. The requirement

COM.40000 for a method validation or verification approval applies.

Method performance specifications for blood specimens may be used for body fluids if the laboratory can

reasonably exclude the existence of matrix interferences affecting the latter either by reference in the

procedure manual to published literature or by evaluation for interferences due to matrix effects by

performing an appropriate study (e.g. a dilution study using admixtures of samples, spiking samples, further

dilution). Alternative performance assessment is required (COM.01500) and may be performed using clinical

assessment by chart review.

The reference intervals must be defined and reported with the results, unless the concentration of the analyte

is reported in comparison to its concentration in a contemporaneously collected blood specimen. If the result

is to be interpreted by comparison to the patient's blood, serum, or plasma, such results must be

accompanied by an appropriate comment such as, "The reference interval(s) and other method performance

specifications are unavailable for this body fluid. Comparison of this result with the concentration in the

blood, serum, or plasma is recommended." Reference interval citations from the manufacturer's insert or

published literature citations may be used to determine the reference interval (COM.50000). However,

reference intervals have not been published for many body fluid analytes, and obtaining normal fluids to

establish reference intervals may not be feasible.

A request for a test on a body fluid specimen that is not listed on the laboratory's test menu that requires

clearance by the section director or designee is considered a clinically unique specimen, rather than a

routine, orderable test. Typically, these specimens are submitted due to an unusual clinical concern in a

specific patient or situation (e.g. pathologic states where the analyte is not normally found in the fluid type)

and it may not be possible to establish a comparative metric. In such cases, the result must be accompanied

by a comment such as, "The reference interval(s) and other method performance specifications have not

been established for this body fluid. The test result must be integrated into the clinical context for

interpretation."

Evidence of Compliance:

✓ Records of validation or verification studies with review and approval AND

✓ Records of reference interval study OR records of verification of manufacturer's stated

interval or published literature OR other methods approved by the laboratory/section director

**REVISED** 08/17/2016

COM.40630 LDT Reporting Phase I

Reports for laboratory-developed tests (LDTs) contain a statement that the assay was

developed by the laboratory.

NOTE: This requirement does not apply to traditional methods, such as manual microscopy, conventional

microbiologic cultures, conventional cytogenetics, and manual hematology and immunology tests.

Requirements for reports are given in the Results Reporting sections of the checklists.

Laboratories subject to US regulations often include an LDT disclaimer as follows: "This test was developed

and its performance characteristics determined by <insert laboratory/company name>. It has not been

Test Method Validation and Verification

Page 8 of 9

cleared or approved by the FDA. The laboratory is regulated under CLIA as qualified to perform high-

complexity testing. This test is used for clinical purposes. It should not be regarded as investigational or for

research."

The report should also contain a brief description of the method and any performance characteristics needed

for clinical use, unless the information is readily available to the clinician in an equivalent form (e.g. test

catalog).

COM.40640 Clinical Claims Validation Phase II

All clinical claims made by the laboratory are validated for the following types of tests:

● Laboratory-developed tests (LDTs)

● FDA-cleared/approved tests for which the laboratory makes a clinical claim(s) not included in

manufacturer instructions

NOTE: Clinical claims include statements about a test's diagnostic sensitivity and specificity, ability to predict

the risk of a disease or condition, clinical usefulness, or cost-effectiveness. Clinical claims may be found on

the test report or in other information distributed by the laboratory (websites, test catalogues, newsletters,

memoranda, advertisements, etc.).

Laboratories are not required to make clinical claims about a test, but any claims made by the laboratory

must be validated.

In order to adequately support a claim about diagnostic sensitivity and specificity and/or ability to predict risk

of a disease or condition, the laboratory must perform a clinical validation study, unless the clinical validity of

the test is documented in peer-reviewed literature or textbooks. The clinical validation study must include at

least 20 samples and must include both positive and negative samples. If the laboratory uses fewer samples,

the laboratory director must record the criteria used to determine the appropriateness of the sample size.

Evidence of Compliance:

✓ Records of clinical studies performed by the laboratory OR peer-reviewed literature that reasonably

substantiates all claims made by the laboratory about a test

COM.40700 Method Performance Specifications Availability Phase II

For current test methods, the laboratory makes the following available to clients and the

inspection team upon request:

● Summary of the analytical performance specifications for each method, validated or verified by the

laboratory to include analytical accuracy, precision, analytical sensitivity, analytical specificity (test method

interferences), reference interval, and reportable range, as applicable; and

● Supporting data for clinical performance claims, if applicable, validated or verified by the laboratory or

obtained from peer-reviewed literature.

NOTE: For the purposes of providing this information to clients, it may be presented in a summary format

referring to the supporting data, statistics, and published studies, as appropriate.

Clients include healthcare entities, other laboratories, and licensed independent practitioners.

This requirement does not apply to patients or their authorized representatives.

The laboratory may require clients to treat the data as confidential and not to use such

proprietary information for its own test development or share such data with any other party

except as required by law. The CAP inspection team is instructed to treat all such data as

confidential and to review them solely for accreditation purposes.

REFERENCES

1) Department of Health and Human Services, Centers for Medicare and Medicaid Services. Clinical laboratory improvement

amendments of 1988; final rule. Fed Register. 2003(Jan 24):7163 [42CFR493.1291(e)]

Test Method Validation and Verification

Page 9 of 9

COM.40800 Analytical Methodology Changes Phase II

If the laboratory changes its analytical methodology so that test results or their

interpretations may be SIGNIFICANTLY different, the change is explained to clients.

NOTE: This requirement can be accomplished in any of several different ways, depending on local

circumstances. Some methods include directed mailings, laboratory newsletters or part of the test report

itself.

Evidence of Compliance:

✓ Records such as directed mailings, laboratory newsletters or comment on the patient report advising of the

change

REFERENCES

1) Department of Health and Human Services, Centers for Medicare and Medicaid Services. Clinical laboratory improvement

amendments of 1988; final rule. Fed Register. 2003(Jan 24):7163 [42CFR493.1291(e)]

REFERENCE INTERVALS

COM.50000 Reference Intervals Phase II

The laboratory verifies or establishes its reference intervals.

NOTE: Reference intervals are important to allow a clinician to assess patient results against an appropriate

population. The reference intervals must be established or verified for each analyte and specimen source

(e.g. blood, urine, cerebrospinal fluid), when appropriate. For example, a reference interval can be verified by

testing samples from 20 healthy representative individuals; if no more than two results fall outside the

proposed reference interval, that interval can be considered verified for the population studied (refer to CLSI

guideline EP28-A3C, reference below).

If a formal reference interval study is not possible or practical, then the laboratory should carefully evaluate

the use of published data for its own reference intervals, and retain records of this evaluation. For many

analytes (e.g. therapeutic drugs, cholesterol, and CSF total protein), literature references or a manufacturer's

package insert information may be appropriate.

Evidence of Compliance:

✓ Record of reference interval study OR records of verification of manufacturer's stated interval when

reference interval study is not practical (e.g. unavailable normal population) OR other methods approved by

the laboratory/section director

COM.50100 Reference Interval Evaluation Phase II

The laboratory evaluates the appropriateness of its reference intervals and takes corrective action if

necessary.

NOTE: Criteria for evaluation of reference intervals include:

1. Introduction of a new analyte to the test repertoire

2. Change of analytic methodology

3. Change in patient population

If it is determined that the range is no longer appropriate for the patient population, corrective action must be

taken.

Evidence of Compliance:

✓ Records of evaluation and corrective action, if indicated