Kevin B. O’Reilly
September 2014—Newly reported survey data that show widely varying international practices on managing critical values may demonstrate the need for a new guideline—already in development—to help laboratories formulate evidence-based policies.
The new data from European labs were presented during a session at the American Association for Clinical Chemistry’s Annual Meeting and Clinical Expo in Chicago (“Critical Result Management Practices: Global Perspectives and Recommendations for Best Practices”). The session also provided a preview of a forthcoming draft guideline from the Clinical and Laboratory Standards Institute that represents the organization’s first formal attempt to advise laboratories around the world on critical values reporting.
The survey of 871 participating European labs was conducted by the Task and Finish Group on Critical Results of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM), in partnership with the Australasian Association of Clinical Biochemists. While single-country surveys of critical result practices in Europe have been conducted previously, this work marks the first time that labs in many—30—different countries answered the same set of questions about how they operate in this sphere so essential to patient safety.
“It’s rather convincing that critical result management practices are really greatly varied. This means both within and between countries,” Eva Ajzner, MD, PhD, said at the session. Dr. Ajzner, from the Teaching Hospital of the University of Debrecen Medical and Health Science Center in Hungary, is chair of the EFLM group that conducted the survey.
Hungary represented about six percent of the responding laboratories, while French labs accounted for a plurality of respondents with 29 percent. Laboratories in the United Kingdom represented 13 percent of respondents, while another 11 percent of responses came from Italy. Between 20 and 73 laboratories in Serbia, Belgium, the Czech Republic, the Netherlands, Croatia, Slovenia, and Norway also participated in the survey, which was conducted online between September 2012 and March 2013. The survey drew responses from private and publicly owned labs serving different settings—hospitals, outpatient clinics, adults and pediatrics—in metropolitan and rural areas.
The data show a number of areas where European labs appear to be falling short of what they should be doing in managing critical results, Dr. Ajzner said. The major shortfall in European labs comes in the development of critical alert lists, thresholds, and auditing practices, Dr. Ajzner tells CAP TODAY. Such critical results management practices “should be performed in a shared policy between laboratory and clinical staff. Although this fundamental requirement is not fulfilled in many European laboratories, in some countries a majority of laboratories cooperate with clinicians in critical result management,” she says. In many European labs, the job of designing critical results lists “is mainly a task of lab professionals, and physicians are seldom involved,” Dr. Ajzner says. Less than half of European labs ask doctors about critical value thresholds, but among laboratories in Norway and the Netherlands, 72 percent and 88 percent, respectively, do involve clinicians in these decisions. That compares with 73 percent in the U.S. (Howanitz PJ, et al. Arch Pathol Lab Med. 2002; 126:663–669).
“I’m not proud to show you these results,” she said at one point in her talk. “In 44 percent of European laboratories, there is absolutely no protocol to read back the results and for labs to record this.” Less than a quarter of labs said they do require read-back of results but that they do not record that process. “The proper actions are done in only about 33 percent of European laboratories and in about 60 percent of laboratories in the U.K., where read-back and documentation is most often applied,” Dr. Ajzner said. That compares with a 91 percent rate of read-back and documentation found in a survey of more than 700 American labs (Dighe AS, et al. Arch Pathol Lab Med. 2008; 132:1666–1671).
European laboratories do have the upper hand in some areas of critical values management, Dr. Ajzner says. For example, the EFLM survey found that in nearly half of European labs, it is the medical doctor on duty who usually delivers the critical result, which Dr. Ajzner says provides an opportunity for immediate consultation at the time of notification.
The EFLM survey also provides key insight into the wide variation among labs in terms of which tests are selected for inclusion in critical value lists. For the adult lists, just 11 laboratory tests were shared by more than half of European labs and only three were listed by more than 90 percent. By comparison, 21 tests were shared by more than half of the 163 American labs that took part in a CAP Q-Probes study (Wagar EA, et al. Arch Pathol Lab Med. 2007;131:1769–1775). Eight tests were listed by more than 90 percent of these U.S. laboratories.
“The variations seen in the European survey and other studies on which values and thresholds are chosen may be attributed, in part, to differences in the patient populations and clinical settings that laboratories serve, as well as differences in the test methodologies they employ,” Dr. Ajzner says.
She adds, however, that these factors do not tell the whole story.
“There is a lack of published, evidence-based clinical outcome data for all but a handful of laboratory medicine tests, which is probably the main contributor to the disparity in [critical value] list composition among laboratories,” Dr. Ajzner says.
The shortage of outcomes evidence to drive decisionmaking on critical values was noted by another speaker at the AACC session, Andrea R. Horvath, MD, PhD. She is clinical director of South Eastern Area Laboratory Services North at Prince of Wales Hospital in Sydney, Australia, and a member of the CLSI committee that is developing the draft guideline on critical values. Dr. Horvath also served as a consultant on the EFLM survey.
“We don’t have true outcome data to tell us at what critical thresholds urgent notification of results would save lives,” Dr. Horvath said at the session. “For that reason, it’s extremely difficult for us to decide. There are no universal standards.”
“Laboratories all around the world face difficulties when designing alert lists,” she added. “There’s hardly any evidence to help them.”
Andrew N. Young, MD, PhD, another speaker at the AACC session, echoed this point in an interview with CAP TODAY. Dr. Young is medical director of Quest Diagnostics’ Pittsburgh office and heads the CLSI’s document development committee on reporting critical results.
“That’s one of the most significant challenges, which is that there is not a lot of high-quality evidence that defines a level of risk for any given test result,” he says. “A lot of it is based on collective clinical experience and judgment.”
“You can imagine that we’re not going to have a controlled clinical trial where one group of patients will have their critical test results communicated and another won’t,” Dr. Young says. “That would never be permitted, for good reason, so it’s unlikely we’ll ever get that kind of evidence.”
Jonathan R. Genzen, MD, PhD, sees some promise for definitive answers in the transformation to electronic record keeping. Dr. Genzen, who spoke at another AACC session (“Critical Values: Improving the Design, Practice, and Communication of Critical Laboratory Results”), is medical director of the automated core laboratory at ARUP Laboratories and assistant professor of pathology at the University of Utah School of Medicine. He also is the lead author of a review article on critical value communications (Am J Clin Pathol. 2011;135:505–513).
“I do wonder if we will reach a point where clinical data warehouses—basically databases that contain not just laboratory data but also clinical outcomes data as part of the electronic record—could provide enough patients and enough lab values and enough clinical outcomes to look back retrospectively and say that overall mortality was higher for patients in this threshold or that threshold,” Dr. Genzen tells CAP TODAY. “That could provide the type of evidence we’re looking for, more of an evidence-based threshold for critical values and critical value limits.”
That kind of analysis is entering the medical literature. For example, researchers examined a cohort of nearly 40,000 patients admitted to Sarasota Memorial Hospital and looked to associate a risk of death one year after discharge with small intervals of values for five routinely performed laboratory tests: serum creatinine, blood urea nitrogen, serum sodium, serum potassium, and serum chloride (Solinger AB, et al. Clin Chem Lab Med. 2013;51:1803–1813). The study led the authors to propose reducing the upper limits for some tests to avoid “high-normal” results where they found that the mortality risk rose two to three times above the average of the patient population studied. The hospital’s reference interval for potassium had been 3.5–5.1 mmol/L, but the research showed the upper limit should be cut to 4.3 mmol/L. Meanwhile, the reference interval for sodium was 136–145 mmol/L; the researchers proposed dropping the upper limit to 142 mmol/L.
This methodology “allows the potential for [decision limits] to be determined from unlimited data mining of any EMR or a variety of other sources without consideration of selecting and maintaining a healthy cohort,” the researchers wrote. “As the study can be done retrospectively at any hospital or laboratory with extant data, the statistical sample sizes are just about unlimited, meaning that 95 percent confidence intervals can be as small as desired.”
An earlier study that also mined electronic records for help in setting alert thresholds involved an examination of more than 100,000 sodium results over a six-month period. Pathologists from the State University of New York Health Science Center in Brooklyn looked beyond death risk to analyze how the lengths of stay and clinical actions changed for patients with hypernatremia and hyponatremia (Howanitz JH, et al. Am J Clin Pathol. 2007;127:56–59). Based on the study, researchers opted not to change their sodium thresholds of 120 mmol/L–155 mmol/L. They noted that increasing the lower sodium limit to 125 mmol/L would have generated more than 600 additional calls from the laboratory during the study’s six-month time frame.
Dr. Young says this kind of data mining could represent “a great approach” to using clinical outcomes to help set alert thresholds. “It does have some limitations,” he noted, insofar as this kind of retrospective review cannot conclusively demonstrate that the critical value was the cause of any particular patient’s death.
“But,” he says, “anything we can do to derive some objective assessment of risk that organizations can use, or that guideline development committees can use, to try to come to some level of agreement on starting points to help organizations say, ‘We’re going to reassess our approach to critical results and where do we begin?’ That’s a fantastic start.”
In light of the scarcity of gold-standard evidence to guide the setting of critical value limits, Dr. Horvath advised laboratory leaders to pursue a hierarchical approach to the process. Adapting a concept outlined in an earlier study (Sikaris K. Clin Biochem Rev. 2012;33:141–148), she said lab professionals and their clinician collaborators ought to consider, in descending order of importance, these five levels of information:
- Level one: clinical outcomes in specific clinical settings.
- Level two: consultation with clinicians in local settings.
- Level three: published professional recommendations of national or international expert bodies.
- Level four: national or international surveys of current practice (“the state of the art”).
- Level five: individual publications, textbooks, expert opinion.
Dr. Horvath noted the limitations of broad surveys of other health care organizations’ critical value lists and thresholds.
“These surveys represent very different health care settings and populations, and therefore the findings are not universally applicable,” she said. “You have to critically look at where the information comes from—inpatient versus outpatient. Survey findings may represent outdated published data and resources rather than a local consensus or the clinical need for best practice.”
She said the process of defining alert lists and thresholds should be a cycle of needs assessment, stakeholder involvement, evidence review, and consensus between the laboratory and the users of its service. That should be followed by continually monitoring and updating the alert lists as more and higher-quality evidence becomes available and experience accumulates with the use of certain alert lists and policies, Dr. Horvath said.
“You should ask the clinical users which tests they consider critical,” she added. “Ask, ‘What are you going to do if I call with such a result?’ Because if they’re not going to do anything with it, then you need to
She also advised following a formal process of risk analysis, an element upon which Quest’s Dr. Young expanded in his talk and that is likely to be included in the CLSI’s draft guideline (see “Sample risk analysis,” page 56). He said he hopes the CLSI guideline becomes a “one-stop shop” to help laboratories manage critical values.
“It’s good to use formal risk-management policies wherever possible when looking to define local thresholds for critical results,” Dr. Young said. “This could be defined with a few key steps, including risk analysis. What’s the risk of a bad outcome based on particular test results, as well as local points of system failure in terms of that result not getting to the right person who can take action? Is the test result likely to require time-sensitive evaluation, and would patient needs be enhanced by a special alert, or are routine workflows sufficient?
“You can assess the probability of a critical result with adverse harm, how severe the harm would be, and compare those risks with the risks of system failure,” he said.
This notion of formally examining the risk associated with a given lab value dovetails with a terminology change that will be proposed in the CLSI’s draft guideline. Instead of “critical result” or “critical value,” the document development committee is likely to propose the term “critical-risk result,” Dr. Young tells CAP TODAY. The committee also is proposing another category of “significant-risk results,” which are values that are not immediately life-threatening but deserve the prompt attention of clinicians.
“We added the term ‘risk’ to these because we felt that was the central concept behind this practice, that different approaches to communicating these results and reporting these results arise because the results signify increased levels of risk to the patient,” Dr. Young says. “And when organizations begin to consider whether certain results should be reported in that fashion or not, the central question should be: Is there patient risk involved?”
“With growing experience, a lot of organizations are seeing that there are adverse outcomes with results that don’t signify immediate risk, that don’t need to be communicated without delay but at the same time need to be directly communicated so that a clinician is given that information—and the system doesn’t just assume the results will be reviewed within the normal course of work and in a manner that’s timely enough for intervention.”
An example of a significant-risk result is a positive acid-fast bacillus smear in an outpatient, he says. That may not be considered a critical result because it is unlikely “to signify immediate, severe risk to the patient,” Dr. Young says. “However, the likelihood of adverse outcomes can still be reduced if the lab reports them directly to a responsible clinician within a reasonable time—eight hours, perhaps—especially if clinicians don’t have real-time remote access to lab results and there aren’t processes to ensure timely review of all ordered tests.”
Dr. Young says he and his colleagues who are developing the guideline are not trying to increase laboratory workloads by adding noncritical values to the results-communication picture.
“I don’t think we’d recommend that every organization has to come up with a list of significant-risk results, or that necessarily the same list would apply to all places,” he says. “There may be alternative means to ensure that results reported through standard processes get reviewed. . . . This can be done within the informatics department, or even through periodic, partial monitoring. There at least has to be some system to assess whether abnormal results are reviewed and followed up on.”
Dr. Young says laboratories in the U.S. and around the world can benefit from additional standardization on how they manage critical results, and that a CLSI guideline can meet that need.
“This process is more complicated than meets the eye. You can say, ‘Yeah, of course you should call these values and that’s what everybody else does.’ But when you start to dig into it a little bit, it’s not at all trivial to say: ‘These results are the ones to be communicated and this is the system to be used to do that, and this is the person who should make the call, and this is the person who should receive the call,’” Dr. Young says (see “Sample escalation protocol,” page 66).
“There was an increasing recognition that with greater attention to patient safety, which is unquestionably a good thing, this issue is coming to the forefront more and more.”
The CSLI’s draft guideline also is likely to caution against routinely repeating a test on the same specimen that yields a critical value, Dr. Young says.
“Simply repeating the test on the specimen has a low rate of finding analytical error,” he notes. A 2008 CAP Q-Probes found that 56 percent of the 121 labs participating repeated testing for all critical results, and 31 percent repeated testing for some results (Valenstein PN, et al. Arch Pathol Lab Med. 2008;132: 1862–1867).
The initial draft guideline has been submitted to the CLSI document development committee for review, which was scheduled to be completed by Sept. 3, according to Marcy Hackenbrack, MCM, M(ASCP), senior standards project manager at the institute. The draft guideline will then undergo additional internal review and editing, which could take another six weeks, she says. It will then be made available for a 60-day public review and comment period later this year. At that time, laboratory professionals and other interested parties can obtain a copy of the draft upon request by calling the institute’s customer service line (610-688-0100). Dr. Young says he hopes the guideline will be finalized and published in late spring 2015.
In the meantime, laboratory professionals must field the occasional query from clinicians upset about too many calls, or too few, regarding critical values. In his talk at the AACC meeting, Dr. Genzen said labs should take a formal approach to considering such requests (see “How to conduct a critical values audit”).
“This is very much clinician-driven, where a case arises about whether the threshold is appropriate or not. These often come about in an emotional context based on a certain patient,” Dr. Genzen tells CAP TODAY. “One thing I like about the critical values audit is that it’s a little more objective, and based on the literature, rather than based on a knee-jerk reaction regarding a specific patient scenario.”
The audit should assess the impact of the proposed change on clinical care and laboratory functioning and take into account patient age and gender, if relevant. Such a review may take time given other work priorities, but clinicians are likely to welcome it as a sign of collaboration and collegiality, Dr. Genzen says.
“For clinicians to know you’re actually evaluating the suggestion is just as important as how quickly you’re doing it,” he adds. “You currently can’t satisfy every single clinician with their own unique set of critical values. Maybe someday, from an IT perspective, we’ll be able to do that, but not yet. [Clinicians] want to know you’ve heard and evaluated their concern. To some extent, this gets to the issue of the laboratory being a partner in the diagnostics and clinical workup as opposed to just a clinical service. I think if clinicians understand the important role that labs play in doing this in terms of patient care, then it can be a productive instead of adversarial situation.”
A broader approach to revising the critical values list—with an eye toward reducing the burden on clinicians and the lab—was taken at University Hospitals Case Medical Center in Cleveland, where Christine Schmotzer, MD, is director of clinical chemistry. She described her laboratory’s approach at another AACC session (“Using Technology to Enhance the Value and Communication of Laboratory Results”). Dr. Schmotzer and her colleagues surveyed 17 other large academic medical centers as a way to benchmark UH Case’s critical values list.
The benchmarking led to changing the thresholds for bicarbonate, calcium, glucose, troponin, arterial blood gas, and oxygen partial pressure. The following were removed from the critical values list: amylase, chloride, CKMB index, folate, lipase, oxygen saturation, total bilirubin (adult), and vitamin B12. The UH Case laboratory also eliminated repeat calls for select analytes called critical within the previous 24 hours, and cut calls to certain units such as troponin to the cardiac intensive care unit.
Those changes helped slash the monthly volume of critical calls for many tests, such as a drop in platelet-count calls from 201 to 25, for troponin from 133 to 50, and from 66 to 26 for glucose, Dr. Schmotzer said in her talk.
“We want to be calling them when it is truly life-threatening and they need to be notified, and decrease the number of times we call them and they’re not going to act differently based on the phone call,” she tells CAP TODAY. “We want to call only when we need to call, so that the urgency gets pressed upon them when we do call.”
UH Case is considering whether to expand its site-specific and physician-specific critical values procedures.
“We’re looking at whether to call repeat blood gas results for our ICUs that do them so frequently and are closely monitoring their patients,” she says. “Different cutoffs we’re considering would be magnesium for the labor-and-delivery unit versus magnesium for the medical floor, and for BUN and potassium in dialysis patients.”
To offer such customization, Dr. Schmotzer says it is critical to document the unit’s desire to opt out of certain calls and have a way to automate the opt-out process.
“We can’t rely on the laboratory staff to say the troponin is from the cardiac ICU, so don’t call them. If we can’t do it electronically, we’re more reluctant to do it. We don’t need the lab staff member to think about it or have to consult another resource,” she says. “That will increase the risk of procedures not being followed correctly.”
Being able to offer this kind of customized approach to critical values management highlights a theme that was repeated at this year’s AACC meeting: the importance of collaboration in this vital area of patient safety.
“Understandably, the [critical values] policy often originates in the laboratory, but it’s not the laboratory’s decision to say what should or shouldn’t be called. That’s a group decision that has to involve clinicians as well as hospital administrators, so that organizations have policies that really address the clinical care of the patient, that ensure compliance with regulatory and accreditation standards, and that are reasonable for all parties,” says Dr. Young, who directed the clinical laboratory at Atlanta’s Grady Memorial Hospital before joining Quest.
“More and more organizations have begun to take that approach. This should not be a source of conflict but of great collaboration, with patient safety as the primary goal.”
Kevin B. O’Reilly is CAP TODAY senior editor.