Home >> ALL ISSUES >> 2013 Issues >> Steep climb to suitable reference standards

Steep climb to suitable reference standards

image_pdfCreate PDF

William Check, PhD

February 2013—It’s a long way from ancient Greek philosophers to modern-day clinical laboratory directors. Yet both types of scholars have one thing in common: the pursuit of truth. Socrates and his disciples thought of truth as correspondence to an objective universal ideal in the mind. Today’s clinical laboratory scientists need a more concrete standard against which to measure their results, leading to the continuing search for suitable reference materials to be used in method development, test validation, internal QC, assay calibration, and proficiency testing.

For a laboratory test to produce true results reliably, reference materials are required that are well-characterized, homogeneous, stable, traceable, and commutable, Lawrence J. Jennings, MD, PhD, D(ABHI), noted. He was speaking at a plenary session on the lack of laboratory reference materials at the Association for Molecular Pathology 2012 Annual Meeting on Genomic Medicine. The talks of the plenary’s three speakers were intended to be complementary, he told CAP TODAY. “I spent most of my time talking about the challenges we face [in PT for molecular oncology testing] and justifying the approach to move away from tissue. David [Barton, PhD] and Lisa [Kalman, PhD] spent most of their time talking about the work they do characterizing [genetic] reference materials and what is available.”

Dr. Jennings proceeded from the premise that proficiency testing surveys (also known as external quality assurance, or EQA) should be graded. The CAP’s Molecular Oncology Committee, of which Dr. Jennings is chair, has made that a goal. Grading requires determining truth, and that means reference standards. However, “we often don’t have suitable reference materials,” said Dr. Jennings, director of molecular pathology and of HLA and immunogenetics and assistant professor of pathology at Northwestern University Feinberg School of Medicine and Lurie Children’s Hospital of Chicago. As a result, grading may be done by using referee laboratories or by consensus—preferably 80 percent—of reporting labs. “These are not always the best options,” he said at the meeting.

Later, in an interview, he elaborated. “Over the last several years we have used tissue [as reference material for PT]. Molecular pathologists like to use tissue because it is compatible with how pathologists work. We can identify the tumor and do macroscopic dissection to enrich for tumor.” Now the Molecular Oncology Committee would like to move away from tissue and toward cell lines for solid tumors, using a different cell line for each biomarker analyte—BRAF, EGFR, KRAS, and others. “This trend started several years ago with evaluating hematological malignancies for minimal residual disease,” he says. Initially leukemia cells from patient samples were used, but there was difficulty getting sufficient amounts of high-quality material, so they went to cell lines. “We are doing the same with solid tumors now,” he says. “But it has been difficult to get pathologists to accept it. People like to see that tissue.”

In the AMP session, Dr. Jennings gave examples of the kinds of problems that can arise from using tissue as reference material. In CAP Survey 2010B, testing for KRAS in adenocarcinoma of the colon, two-thirds of the laboratories reported a positive result and one-third reported negative. This discrepancy appeared among committee members as well. Sampling error in sending out the tumor specimens was ruled out. All blocks were from the same patient. However, it was found that the tumor sample was heterogeneous, even within the primary tumor. Data show that eight percent of primary and 31 percent of metastatic colon cancers are heterogeneous for KRAS (Baldus SE, et al. Clin Cancer Res. 2010;16:790–799). “So this was likely to happen again,” Dr. Jennings told CAP TODAY. “How could we prevent this?” Testing all parts of all blocks for mutations would be “undoable,” he says. “This experience helped convince people to move to something more reliable, better-characterized, and homogeneous.”

A second example concerned heterogeneity of allelic burden within tissue. A specimen of adenocarcinoma of the lung was sent to 82 laboratories to test for EGFR. Ninety percent reported an Exon 21 mutation; all used real-time PCR or pyrosequencing. The 10 percent reporting negative results all used Sanger sequencing, which has a lower sensitivity. Was this a problem of limit of detection? To investigate, the committee looked at cellularity. Percentage of neoplastic cells in the lesional area reported by Survey participants showed a broad distribution. “Their numbers were all across the board,” he says. The same was true in two KRAS Surveys.

Next they sent photos of H&E-stained tissue from a fairly differentiated adenocarcinoma. They asked, What is the percentage of neoplastic cells in this image? Answers ranged from 30 percent to 90 percent. Similar variability was seen in eight more photo challenges. “Even with a photo people were all over the board,” Dr. Jennings says. “We cannot trust a pathologist to report back the cellularity and thus allele burden. So we can’t assume all labs are getting the same tumor burden. We can’t get past the fact that we don’t know what we sent them.”

A third problem is poor or inconsistent quality. For instance, a large lymphoma specimen was so degraded that participant laboratories said they couldn’t analyze it.

Poorly characterized material can be a problem, such as a sample lacking a lesion. One tissue was positive by FISH break-apart fusion probe but negative by rt-PCR. In this sample, gene rearrangement did not yield expression of the fusion transcript.

Other obstacles are lack of samples with rare mutations, lack of sufficient material, and use of samples with single analytes (single mutations). “Labs are doing more multianalyte testing these days,” Dr. Jennings notes, adding, “How can we evaluate all those analytes in one sample?”

The lack of well-characterized reference materials is detrimental not only to PT programs but also to validation during assay development. “We are all struggling to find homogeneous, well-characterized material for validation,” he says, speaking as a director of a small molecular lab. “For large molecular laboratories such as the Cleveland Clinic or Mayo, that is not a problem. For us to get sufficient numbers of well-characterized specimens, we rely a lot on directors of larger labs. That is not the best way to do it.”

To pursue the goal of cell lines as reference materials for molecular oncology, Dr. Jennings has worked with Lisa Kalman, PhD, of the Centers for Disease Control and Prevention. “For the past eight years or so, Dr. Kalman’s group has focused on reference materials for heritable conditions,” he says. “They have been very successful in generating panels of materials for such things as Duchenne muscular dystrophy and fragile X.” Dr. Jennings attempted in 2011 to get the group to work on molecular oncology. While the Coriell Institute for Medical Research has cell lines for many heritable conditions, there is no repository of cell lines for molecular oncology analytes. Dr. Jennings and Dr. Kalman are working with the National Cancer Institute, American Type Culture Collection, and Coriell to see if they can establish such a repository. “I would like things to move much quicker than they have been,” he says.

“As we move away from tissue specimens,” he continues, “I am very cognizant that cell lines do not address all aspects of pathology practice, such as microdissection or extraction and amplification of DNA from FFPE [formalin-fixed paraffin-embedded] tissue.” One way around this is to include method-based challenges that are not tumor specific or mutation specific, such as a cellularity challenge and an FFPE challenge.

Dr. Jennings has an additional concern. “As we move to multianalyte testing, we may obligate people to do multianalyte challenges. They might not want or desire that. Should we have single or multiple Surveys? One argument is that we should allow people only to pay for what they want.” Participants in molecular oncology proficiency testing are now being surveyed to find out how many laboratories are testing for more than one analyte. “We know that a large majority are already doing at least two of three biomarkers,” Dr. Jennings says. “There would be a tremendous advantage to combining several analytes into a single Survey.”

Sharing the European perspective on reference materials for genetic testing was David E. Barton, PhD, chief scientist and associate professor at the National Centre for Medical Genetics, Our Lady’s Children’s Hospital, Dublin. He spoke about the work of EuroGenTest (EGT), which is a consortium of 35 partners whose goal is harmonization, validation, and standardization in genetic testing across Europe through training, EQA, and control materials. EGT sent a survey in June 2010 to 910 labs in 32 countries. Responses came from 291 (32 percent) labs. One hundred ninety-eight labs reported using genetic reference materials—samples of defined genotypes obtained from external sources. Of these 198, the vast majority, 69 percent, obtained reference materials from colleagues, while 46 percent used EQA/PT materials, 40 percent used certified reference materials, and 30 percent used cell lines.

“Even in what we consider a highly developed genetic testing environment, which we have been doing for 20 years or more, still there is this almost casual exchange of materials between colleagues that seems more suited to a research environment than a regulated clinical testing environment,” Dr. Barton told CAP TODAY. “Partly this is because there just aren’t formally designated reference materials available for much of genetic testing.”

Reference materials are most often used for test validation, internal QC, method development, and assay calibration, the survey showed. Dr. Barton believes that commonly available reference materials are most often used for daily run controls. “Certified reference materials, the top-level material available from NIST and WHO, tend to be quite expensive,” he says. “I expect labs would use those only occasionally to calibrate assays.” Dr. Barton uses reference materials across the board in his laboratory. “We’re a national centre,” he points out, “and I’ve spent many years advocating their use.”

Another group with which Dr. Barton works is the European Molecular Quality Network, or EMQN. “We are the biggest provider of EQA or PT for genetic disorders in the world,” he says. A unique feature of EMQN is that it assesses not just genotyping accuracy but also the interpretation of genotype. “We set clinical cases for every QA,” Dr. Barton says. “Our evaluators assess whether the genotype is correct plus whether the interpretation is correctly given for that case.” Last July Dr. Barton spoke to the CAP’s Next-Generation Sequencing Working Group, which is working on standards development, proficiency testing, and other NGS-related issues. He emphasized accuracy of interpretation. “We think that’s fundamentally important,” he told CAP TODAY. EMQN offers proficiency tests for 25 hereditary disorders, such as familial breast cancer, hereditary deafness, and fragile X syndrome.

CAP TODAY
X