Home >> ALL ISSUES >> 2013 Issues >> Microbiology QA, measure for measure

Microbiology QA, measure for measure

image_pdfCreate PDF

William Check, PhD

August 2013—In the May 20, 2013 issue of the New Yorker, Google co-founder Sergey Brin is quoted as saying of Internet security: “In big corporations people don’t understand what security people do, for the most part, and no one pays attention to them unless something goes wrong. Frankly, a lot of companies aren’t that interested in security.”

What Brin says of Internet security could also be said of quality assurance programs in laboratories: Health care systems aren’t that interested. “I feel that’s a fair comparison,” says Susan Butler-Wu, PhD, D(ABMM), assistant professor of laboratory medicine and associate director, clinical microbiology laboratory, University of Washington. All laboratories are doing quality assurance, she says, “but there are no best practices or any formulation of what goes beyond the regulations.”

To begin to address this deficiency, Amanda Harrington, PhD, D(ABMM), recently of the VA Puget Sound Health Care System and now director of microbiology at the University of Illinois at Chicago, conceived a symposium in which speakers and attendees could share their QA practices. She and Dr. Butler-Wu co-organized “Beyond the Basics: Modern Metrics for Clinical Microbiology,” which took place at the 2013 American Society for Microbiology meeting in May. “That’s what we wanted to do,” Dr. Butler-Wu says, “to go beyond minimum requirements.”

Every lab that uses an automated system for organism identification has an “internal sense” for certain IDs it doesn’t fully trust, says Dr. Susan Butler-Wu. In her lab, integrating trust/no-trust lists into the routine workflow reduced the number of organisms that require identification by sequencing. [Photo: Mike Siegel]

Says Dr. Harrington: “A lot of lab directors, myself included, think everyone else but me has figured out the right quality metrics, and we don’t talk about this. We wanted the symposium to be a starter for discussion.” She and Dr. Butler-Wu and their colleagues were not trying to say, Look at how we’re doing it. “It was more, ‘What are you doing?’” Dr. Butler-Wu says. “We wanted to get feedback from the audience and to start a conversation about what we should be doing or could be doing that goes beyond basic blood culture contamination rates.

“It worked pretty well,” she adds.

Here is what the presenters—Drs. Harrington and Butler-Wu and three others—put out for discussion: when to trust the identification from an automated ID system, what effect tracking corrected microbiology reports can have, how to track data without IT support, using IT to monitor Clostridium difficile testing, and real-time monitoring and clinical impact of result reporting in microbiology.

Dr. Butler-Wu addressed the first—when to trust automated identification systems. Cumitech 32A recommendations on verification of microbial identification instruments (“Unmodified, FDA-Cleared Tests, automated, multi-analyte”) require 90 percent agreement with an existing system or reference method. “That’s still one in 10 organisms where you don’t believe the identification,” she told CAP TODAY. “What can we do about that last 10 percent?”

Certain organisms are known in the literature to be problematic for automated systems:
◆ Coagulase-negative Staphylococci misidentified as Kocuria spp.
Brucella melitensis misidentified as Bergyella zoohelicum and Ochrobactrum anthropi.
Salmonella or Shigella misidentified as E. coli.

“This is true for the Vitek 2 GN card, which we were using, as well as for Microscan and Phoenix,” Dr. Butler-Wu says. “There is a list of organisms they cover but we don’t have a sense of how good they are in practice.” Every lab that uses an identification system has an internal sense for some things it doesn’t fully trust, she says. “For that particular culture, is there a way we can triage so we can say what we trust or we don’t trust?”

To make the trust/no-trust lists, the laboratory took advantage of its ability to perform definitive identification by 16S ribosomal DNA (rDNA) sequencing, which the lab uses for indeterminate or suspicious calls. The focus was on a broad group that presents problems: Gram-negative rods from cystic fibrosis sputum cultures, which accounted for more than 40 percent of the isolates the laboratory was sequencing. “We were having issues in our CF patient population,” Dr. Butler-Wu explains. “It’s established in the literature that routine phenotypic identification is not that accurate for these organisms. Rather than send every one of these non-fermenting Gram-negative rods for 16S rDNA sequencing, we wanted to know if some things were really right and others, when we get an identification, were bogus.”

Dr. Butler-Wu reviewed the results from 2,012 isolates that had been sequenced in University of Washington CF patients from 2006 to 2012 and correlated them with Vitek results. She found most organisms were reliably identified. On the other hand, she showed a dozen organisms for which Vitek identification was not trustworthy. For example, “When Vitek gave us an identification of Bordetella bronchiseptica, it was only correct in two out of 15 specimens.” Excluding cultures with mixed genera, the correct rate was two of 11. Other single identification correct call rates were 0/14 for Acinetobacter haemolyticus, 11/29 for Pseudomonas fluorescens, and 7/14 for P. putida.

Dr. Butler-Wu also showed what the incorrect calls were. For instance, the isolates incorrectly called Aeromonas salmonicidia (seven calls) were actually Neisseria perflava (1/7) and Pseudomonas aeruginosa (6/7). Likewise, all three isolates called A. sobria were in fact Burkholderia cepacia complex. “Some of these calls have real clinical and management consequences, such as the patient being taken off the transplant list,” Dr. Butler-Wu says. “We have to look further with those isolates.”

About a year after the trust list was disseminated, a substantial drop occurred in the number of organisms submitted for 16S rDNA sequencing. Regarding the one-year delay, she says, “Maybe this change took a while to penetrate.” (It’s the technologists who decide whether to send for sequencing.) Dr. Butler-Wu acknowledges that her evaluation of the impact of the no-trust list on sequencing frequency is not a rigorous one.

“In our case this was a static process,” she says. “Ideally, we should be continually monitoring and putting organisms off and on the no-trust list.”

Further decrease in orders for sequencing occurred in mid-2012 when the laboratory adopted mass spectrometry for routine bacterial and yeast isolates. “For most patients mass spec has replaced Vitek,” she notes.

The presentation by Linoj Samuel, PhD, D(ABMM), division head of clinical microbiology, Henry Ford Health System, was “Stay on Target: What Metrics Can Tell You About Culture Complications and Your Laboratory.” At the outset, he pointed out the increase in volume microbiology laboratories are seeing. At Henry Ford, for one, four hospital laboratories have been consolidated into one core laboratory, with no proportionate increase in the number of technologists. “As lab directors and managers we end up overseeing microbiology labs for four different hospitals with different patient populations.” Many others are in the same situation. “What tools can we use to improve oversight?” he asks.

One required metric is blood culture contamination rate, a simple metric but one that might be misleading. “We need to dig deeper to see whether it accurately represents what is going on,” Dr. Samuel says. A recommended rate is less than three percent. “We found at one of our emergency departments that, even though the contamination rate was less than three percent, there was room for improvement,” he says. Nurses tended to draw multiple sets from single sticks or lines. Moreover, multiple sets were contaminated with coagulase-negative staphylococci, which the laboratory algorithm did not flag as a contaminant. “We went back and provided education to the nursing staff on the importance of blood culture collection practices,” Dr. Samuel says.

The clinical microbiology division has adopted four levels of metrics: global, process-specific, workload, and test-specific. Tracking corrected reports is a global metric. Examples are Gram stain interpretation errors, errors in organism identification, result entry errors, and specimen processing errors. “Our pathology colleagues on the anatomic side have been doing this for many years,” he notes, citing two publications from the surgical pathologists in his department (Nakhleh RE, Zarbo RJ. Arch Pathol Lab Med. 1998;122:303–309; Meier FA, et al. Adv Anat Pathol. 2011;18:406–413). Dr. Samuel also presented data from his department chair, Richard Zarbo, MD, who started the initiative for pathologists and showed, with this tracking program, a reduction in misinterpretations from 12.4 amended reports per 10,000 surgical pathology cases in 2005 to fewer than one in 2010 and 2011. “This is our challenge in medicine,” Dr. Zarbo told CAP TODAY—“how to understand what is occurring ‘in the shop’ in real-time, at the level of each piece of work.”

Corrected reports need to be tracked systematically. Says Dr. Samuel: “We do cultures on second shift and weekends. Neither I nor the manager is here when that corrected report is made. So we needed to put a process in place to capture them daily and bring them to our attention.” In microbiology the same verbiage is used on all corrected reports, so the IT staff wrote a program that identifies when the language is used. “We get a daily report broken down by technologist, so we can provide feedback the very next day.”

A systematic report with errors by category shows problems that would not otherwise be visible. For instance, in one period a technologist made a Gram stain error in each of three consecutive months. They saw a trend and provided her with support and education, Dr. Samuel says. “For the rest of the year she never showed up on the report again. When you have 50-plus people in the lab and everyone is changing shifts and benches, it is hard to pin down who is making errors unless you follow them in a long-term report.”

The number of corrected reports per month fell in 2012 even as volume rose with no significant increase in staffing. “I think direct feedback helps a lot. It helps individuals correct their own way of doing things,” he says.

When working up colonies off a plate, for example, technologists are encouraged to do a Gram stain first. “Sometimes they think they can tell the identity from colony appearance and they work it up as such,” Dr. Samuel says. “So they tend to make errors. When they see their corrected error on this report it emphasizes to do the procedure as outlined.”

Get technical staff involved, he says. “When we first rolled out this program, it was not popular. Techs felt they were good workers and this was too much like Big Brother looking over their shoulder. But it was not punitive. No one was punished because of showing up on the report. We emphasized that almost everyone will show up at some time.” It’s not made part of their annual evaluation and it’s not on their record unless the error was significant enough to jeopardize patient safety. “We had to work hard to make the point that this was not to punish anyone but to improve lab performance and reduce the number of errors.” When it’s made an individual responsibility, he says, as opposed to quoting overall laboratory figures, that gets results.

Reporting time for positive blood culture Gram stains is an important process-specific metric. Dr. Samuel’s core laboratory gets all samples from affiliated Henry Ford hospitals. He targeted two time metrics in this process: how long it took from receipt in the local laboratory to the courier run, and how long from when the culture turned positive to when it was reported. “We call positive blood culture Gram stains to the floor,” he says. “It is one of our critical values.”

In 2008 only 37 percent of positive blood culture Gram stains were reported within two hours; 95 percent was the target rate. “Our technologists worked on this for a number of months,” Dr. Samuel says. Reporting improved but didn’t reach the goal. “Technologists felt they needed additional staffing but managers and supervisors felt there were workflow issues.” De-identified reporting times for individual technologists were then posted in the main laboratory area, with each technologist knowing his or her own identity. “Without additional staffing, reporting times suddenly got better,” Dr. Samuel says. Ninety percent of positive blood culture Gram stains were reported in two hours or less. Currently, 95 percent of positive results are reported in one hour or less.

Workflow is a workload metric and parses how many specimens are processed each hour of each day and what kind of specimens they are. When a workflow tracking metric is used, technologists coming in the next day can see how much work they will have to do and what kind, such as how many plates they will need to read out. “Supervisors can use nontraditional staffing schedules to solve workflow problems,” Dr. Samuel says.

Workflow tracking showed that second-shift technologists had to do too much work in the last two hours of their shift. “They needed to be able to close out their shift sooner and not leave work for the midnight shift.” Their goal was to reduce the number of specimens processed at the end of the shift and do more work earlier. “They took it on themselves to revise their own workflow using Lean principles,” Dr. Samuel says, “so there was lower specimen throughput in the last two hours.”

As an example of a test-specific metric, Dr. Samuel uses the Quanti-Feron Gold tuberculosis assay, which Henry Ford sends to a reference laboratory. “This test is very prone to indeterminate results if not handled correctly from the point of collection,” Dr. Samuel says. “This is something that should always be tracked, since an indeterminate result delays patient care.” Indeterminate results spiked in October 2010 to almost 90 percent, with a corresponding drop in negative results to 10 percent. A problem with the handling of specimens was found, after which the rate fell to near zero. Staff turnover can cause this kind of problem; tracking can target the responsible site.

In Dr. Samuel’s laboratory, four metrics are posted on a central bulletin board at any time. “Pick a couple of metrics, follow them, and then move on when you achieve your goal,” he told the ASM attendees. And update them frequently.

Dr. Amanda Harrington had a special challenge in setting up a QA metric: lack of IT support at Seattle’s VA Puget Sound Health Care System, where she was, until her recent move to UIC, assistant director of microbiology and molecular diagnostics, Department of Pathology and Laboratory Medicine. “Most ideas are proposed by vendors,” she says. “None of those solutions has been available to us.” The software wasn’t designed for her system or resources haven’t been available to implement the solutions. “This was also true for other audience members,” she says of the symposium. “A lot of what we can retrieve depends on our IT department. We are effectively held hostage by them. We know we’re sitting on a mountain of data; we just can’t get to it sometimes.”

Dr. Amanda Harrington: “Using a paper-based log, we were able to reduce the number of duplicate test requests by 50 percent.” [Photo: Paul Merideth]

Unable to extract data directly from the system, Dr. Harrington decided what parameters she wanted to follow and asked the technologists to collect, collate, and tabulate those data once per week and enter them into a spreadsheet. Do-it-yourself monitoring of culture review required a defined set of metrics recorded on a paper-based tally sheet and summarized in a pre-programmed Excel spreadsheet (see “Culture review metrics,” page 69).

CAP TODAY
X