Learn more: PMC Disclaimer | PMC Copyright Notice
Can Surveillance Systems Identify and Avert Adverse Drug Events? A Prospective Evaluation of a Commercial Application
Abstract
Objective
Computerized monitors can effectively detect and potentially prevent adverse drug events (ADEs). Most monitors have been developed in large academic hospitals and are not readily usable in other settings. We assessed the ability of a commercial program to identify and prevent ADEs in a community hospital.
Design and Measurement
We prospectively evaluated the commercial application in a community-based hospital. We examined the frequency and types of alerts produced, how often they were associated with ADEs and potential ADEs, and the potential financial impact of monitoring for ADEs.
Results
Among 2,407 patients screened, the application generated 516 high priority alerts. We were able to review 266 alerts at the time they were generated and among these, 30 (11.3%) were considered substantially important to warrant contacting the physician caring for the patient. These 30 alerts were associated with 4 ADEs and 11 potential ADEs. In all 15 cases, the responsible physician was unaware of the event, leading to a change in clinical care in 14 cases. Overall, 23% of high priority alerts were associated with an ADE (95% confidence interval [CI] 12% to 34%) and another 15% were associated with a potential ADE (95% CI 6% to 24%). Active surveillance used approximately 1.5 hours of pharmacist time daily.
Conclusions
A commercially available, computer-based ADE detection tool was effective at identifying ADEs. When used as part of an active surveillance program, it can have an impact on preventing or ameliorating ADEs.
Introduction
Readily deployable adverse drug event (ADE) monitors that effectively track and help prevent ADEs would be very helpful to hospitals without sophisticated electronic health records (EHRs). However, prior evaluations have focused on home-grown systems that are usually tied to sophisticated EHRs. 1,2 It is unknown whether a commercially-available system that might be deployable by a large number of hospitals can be used to identify and ameliorate ADEs in community-based hospitals.
The Vigilanz Corporation has developed the Dynamic Pharmacovigilance (DPV), which monitors laboratory and pharmacy data among hospitalized patients and uses preset rules to determine whether an adverse event might be taking place. 3 Given that commercial systems like this could be widely used by hospitals across the nation, as all that are required are demographics and computerized laboratory and drug data which many hospitals already have, we prospectively evaluated the ability of this application to identify ADEs, avert ADEs and examine the burden they might place on resources of a community-based hospital.
Background
Adverse events represent a major cause of morbidity and mortality for hospitalized patients and medications are the single biggest cause of this type of medical injury. 4,5 Each year in the United States, adverse drug events (ADEs) may contribute to the deaths of up to 140,000 Americans and have substantial financial costs to both patients and the healthcare system. 6 Despite their heavy burden, it has been difficult to monitor and prevent ADEs. Most hospitals still use voluntary reporting by clinicians as their primary source of surveillance data, although numerous studies have shown that this approach identifies less than 1% of all ADEs. 7 A more comprehensive approach, the use of chart review by nurses or pharmacists, is prohibitively expensive and therefore impractical for routine use, although sampling and use of trigger criteria can be helpful 8 but still expensive when applied to all patients. Computer-based monitors represent a sensible compromise: although less sensitive than chart review, they are far more effective at identifying ADEs than self-report, 9 and can detect a substantial proportion of ADEs. 1,2,10 These monitors were developed nearly 30 years ago, 9 and successive evaluations have shown that they can effectively identify nearly as many ADEs as manual chart review. 1 A recent Institute of Medicine report suggested that this likely represents the future of ADE detection. 11
A major challenge, however, has been implementing such monitors in routine care, much of which is delivered in medium-sized and small community hospitals that do not have sophisticated information systems. 12 Most of the data on computer-based monitoring of ADEs come from large academic centers that have internally-developed electronic health records (EHRs), which concomitantly built computerized monitoring tools. 1,2 Although most small hospitals do have an electronic patient tracking system (often known as an Admission/Discharge/Transfer system or ADT), as well as laboratory and pharmacy information in coded electronic format, 13,14 these systems are rarely linked. 15 Having an effective ADE surveillance system that can readily be installed in hospitals without EHRs but with these three databases (ADT, laboratory, and pharmacy) could make computerized monitoring much more accessible to a broad range of hospitals. Therefore, we sought to determine whether a commercially-available computer-based monitoring system that can be easily installed in most hospitals could help identify and prevent ADEs.
Methods
Setting
We used data from a 150-bed nonprofit, community-based teaching hospital located in Boston, MA. In fiscal year 2005, it had 8,200 discharges and cared primarily for general medical and surgical patients. The hospital used information systems sold by MEDITECH (Westwood, MA) that primarily had demographics, laboratory and radiology results, and discharge summaries available on most of the inpatient floors at the time of the study. The hospital did not have an EHR but was in the early stages of implementing a computerized physician order entry (CPOE) system. To install the DPV, the main requirement was that the hospital have three databases: one that contains laboratory results, another with pharmacy data, and a third with patient demographics such as an admission/discharge/transfer database. Prior surveys have found that most hospitals in the U.S. have these three sources of data available in electronic format. 13 The installation time for the vendor system is typically less than 90 days.
Patients
The patients in this study included all adults admitted to five inpatient units during a 4 month period from July to August and from November to December 2004. The units included two medical units, two general surgical units, and a medical intensive care unit. During the study period, a total of 2,407 patients were admitted to these units and screened by the DPV system.
Outcomes
The outcomes measured were presence of an adverse drug event (ADE) or a potential adverse drug event (PADE). An ADE was defined as an injury resulting from an intervention related to a drug. 4 An injury could be a sign (e.g., bleeding), a symptom (e.g., vomiting) or a dangerously abnormal laboratory value (e.g., hemoglobin of 6 gm/L). A potential ADE was defined as a medication error with the potential to cause injury but in which no actual injury occurred. 4 A medication error was defined as an error in the prescribing, dispensing, transcription, administration, or monitoring of a drug. 16 Actual ADEs and potential ADEs were considered to be preventable if the injury (or potential injury) was due to an identifiable medical error that could be prevented given today's technology and knowledge-base.
The DPV System and Rule Base
The rules were developed using combinations of medications, laboratory data and patient demographic data that met certain conditions. For example, a patient on gentamicin with a rising serum creatinine level would trigger an alert. Medications were identified by a pharmacy order (as opposed to a medication administration), and the rules allowed for linking of multiple medications and laboratory tests. An example of this would be a potential bleed in the setting of an order for heparin in conjunction with a recent abciximab order. The use of these medications together would activate the second part of the rule to monitor the lab results of the hematocrit and associated red blood cell count or hemoglobin level.
The rule set designates the alerts with a low, medium, high or critical priority, and a baseline level is established for the key lab within the rule. In the previous gentamicin example, a serum creatinine level greater than 1.3 mg/dL but less than or equal to 2.5 mg/dL would be a low priority alert. A serum creatinine level greater than 2.5 mg/dL but less than or equal to 3.5 mg/dL would constitute a medium priority alert, while a high priority alert would require a serum creatinine level greater than 3.5 mg/dL but less than or equal to 3.9 mg/dL. Finally, any serum creatinine level greater than 3.9 mg/dL would be classified as a critical alert. The system is designed with the flexibility to allow individual institutions to adjust these thresholds based on local practice and priorities.
A rule was considered to be activated if all of the rule's criteria were met (i.e., patient is on gentamicin and patient has an elevated creatinine). The activated rule could change to an alert if a designated amount of time, called the Good Medical Practice (GMP) interval, had passed and the medication had not been stopped, a new test had not been ordered, or the medication dose of interest had been decreased. Depending on the urgency of the alert, the GMP interval varied from 2 hours (e.g., critical serum potassium) to 1 day (e.g., colchicine and serum creatinine greater then 2.0 g/dL in last 1 day) before another alert was generated.
Case Identification
All high-priority alerts generated by the DPV system were evaluated each morning (excluding weekends, holidays, and other days when a reviewer was not available) by either a pharmacist or a physician. The alert report contained patient identifiers, demographics, and details regarding the alert. The reviewer evaluated the alert based on the clinical data available in the hospital computer system and standard references, such as Thomson's Micromedex, 17 and expert guidelines. If the reviewer thought the patient was being treated appropriately or if there was evidence that the clinicians knew about the unfolding event, no further action was taken. However, if the reviewer considered this to be a potentially important clinical event where a call to the physician might be valuable, the review would contact that provider and convey the information. The reviewer kept track of the time required for evaluating alerts and contacting the physician caring for the patient.
All patients with alerts that required contact with the physician underwent a manual chart review. For patients with alerts where the reviewer had decided that no MD contact was required, we reviewed 10% of the charts for further evaluation. Finally, because many of the alerts were never reviewed (because they occurred on days when the reviewer was not available), we also sampled 10% of all patients with such alerts to determine how many ADEs were missed due to the lack of daily evaluation of the alerts.
The manual chart review examined progress notes, consultation notes, laboratory and radiology notes, and nursing medication sheets to look for ADEs and potential ADEs. If the reviewer felt that an ADE or PADE might have occurred, the reviewer completed an incident sheet that contained key clinical information about the case and potential explanations for the event. A second physician (AKJ, the adjudicator), experienced in evaluating ADEs, subsequently determined whether the alert represented an ADE or PADE based on information from the chart review. The adjudicating physician used standard criteria employed in prior studies of ADEs/potential ADEs 4,18 and made the final decision regarding the presence or absence of an ADE/PADE using the modified Naranjo algorithm. 19 This approach has been found to be a reliable way to identify ADEs and has high inter-rater reliability. 4 The adjudicating physician also classified each event into categories of severity (fatal, life-threatening, serious, or significant) and preventability (definitely preventable, likely preventable, likely not preventable and definitely not preventable), using criteria employed by other studies. 4,18
Analysis
We had three primary sets of results. In the first analysis, we examined whether the VigiLanz monitoring system helped identify ADEs and potential ADEs. Second, we determined whether the surveillance process (generation of alerts, review by a physician or pharmacist, and subsequent calls to responsible clinician) would avert ADEs or potential ADEs. Third, we tracked the efficiency of the VigiLanz system by examining the amount of pharmacist time used during the surveillance process.
Funding
The VigiLanz Corporation provided limited financial support for this project. The hospital paid a discounted price (compared to the retail price) for the use of the software. The company provided no resources for the evaluation of its product and had no input into the analytic plan. The lead author (AKJ) did not receive any financial compensation, although one co-author (DWB) did receive an honorarium for a talk he gave on ADE monitoring at the annual meeting of the American Society for Health Systems Pharmacy. This talk was given prior to the initiation of any planning for the study. Finally, the company had no editorial input into the writing of the manuscript and was not shown any drafts of the manuscript before submission.
Results
During the study period, the application screened 2,407 general medical and surgical patients admitted to the wards or the intensive care unit of an academic community hospital. The average age of the patients was 73.7 years (standard deviation 14.8 years) and 52.8% of them were female. Overall, 88.4% of the patients were admitted to a ward and 11.6% were admitted to the ICU.
Among the 2,407 screened patients, the application generated 516 high priority alerts among 352 (15% of all) patients, in which both a set of clinical criteria had been met and the GMP timeframe had expired. The most common trigger for a high priority alert was a patient on enoxaparin with a rapidly rising or very high serum creatinine, accounting for nearly 15% of all alerts. Most of these cases were due to inappropriately high enoxaparin doses in the setting of rising creatinine (acute renal failure) or in patients with severe renal insufficiency (chronic renal failure). Other drugs and their use in the setting of renal failure were also responsible for triggering a large number of high priority alerts: levofloxacin inappropriately dosed in the setting of renal failure was the second most common cause of alerts (nearly 14% of all high priority alerts) and use of angiotensin-converting enzyme inhibitors (ACE-I, captopril and lisinopril) represented the 4th and 5th most common triggers of high priority alerts. Critically elevated serum potassium levels constituted 43 (8.3%) of all high priority alerts (▶).
Table 1
Table 1 Frequency of Alert Drug and Lab for All Alerts Generated by VigiLanz (N=516) ∗
| Alert Drug | Alert Lab | Number of Alerts | % of Alerts | Cumulative % of Alerts |
|---|---|---|---|---|
| Enoxaparin | serum Creatinine | 77 | 14.9% | 14.9% |
| Levofloxacin | serum Creatinine | 71 | 13.8% | 28.7% |
| None | Critical serum Potassium | 43 | 8.3% | 37.0% |
| Captopril | serum Creatinine | 38 | 7.4% | 44.4% |
| Lisinopril | serum Creatinine | 35 | 6.8% | 51.2% |
| Enoxaparin | Low Platelet Count | 18 | 3.5% | 54.7% |
| Ciprofloxacin | serum Creatinine | 18 | 3.5% | 58.1% |
| Vancomycin | serum Creatinine | 18 | 3.5% | 61.6% |
| Atenolol | serum Creatinine | 16 | 3.1% | 64.7% |
| Allopurinol | serum Creatinine | 15 | 2.9% | 67.6% |
| Furoseamide | serum Creatinine | 15 | 2.9% | 70.5% |
| Enoxaparin | Hematocrit | 14 | 2.7% | 73.3% |
| Digoxin | serum Creatinine | 14 | 2.7% | 76.0% |
| Digoxin | Serum Potassium | 13 | 2.5% | 78.5% |
| Famotidine | serum Creatinine | 12 | 2.3% | 80.8% |
| Heparin | Low Platelet Count | 10 | 1.9% | 82.8% |
| Aspirin, NSAIDs | serum Creatinine | 9 | 1.7% | 84.5% |
| Heparin | PTT | 9 | 1.7% | 86.2% |
| Famotidine | Low Platelet Count | 8 | 1.6% | 87.8% |
| Amiodarone & Digoxin | None | 5 | 1.0% | 88.8% |
| Gentamicin | serum Creatinine | 5 | 1.0% | 89.7% |
| Ibuprofen | serum Creatinine | 5 | 1.0% | 90.7% |
| Refecoxib | serum Creatinine | 4 | 0.8% | 91.5% |
| Clopidogrel | serum Creatinine | 3 | 0.6% | 92.1% |
| Acyclovir | serum Creatinine | 3 | 0.6% | 92.6% |
| Metformin | serum Creatinine | 3 | 0.6% | 93.2% |
| Ranitidine | serum Creatinine | 3 | 0.6% | 93.8% |
| Glyburide | serum Creatinine | 2 | 0.4% | 94.2% |
| Others | Others | 29 | 5.8% | 100% |
| Total | 516 | 100% | 100% |
∗ 352 patients generated 516 alerts. For patients with multiple alerts of the same type, only the first alert was counted.
NSAIDS = nonsteroidal antiinflammatory drugs; PTT = Partial Thromboplastin Time.
Alerts Reviewed and Their Outcomes
Of these alerts, 266 (52%) occurred on days when a reviewer was available to examine the alerts and make a determination as to the clinical relevance of the alert while 250 occurred on either weekends, holidays, or on days where a reviewer was not otherwise available. The alerts were typically reviewed between 12 and 24 hours after they were triggered by the system. Of the 266 alerts reviewed prospectively, 30 (11.3%) alerts (among 26 patients) were considered substantially important to warrant contacting the physician caring for the patient (▶).
Table 2
Table 2 Summary of Alert Characteristic Generated by VigiLanz during Study Period
| Alerts Reviewed (N=266) | |
| Alerts with MD contacted | 30 (100%) |
| Number of alerts with a ADE/PADE | 15 (50%) |
| Alert without MD contacted | 236 |
| Percent of alerts with ADE ∗ (95% CI) | 12% (7% to 20%) |
| Percent of alerts with PADE ∗ (95% CI) | 18% (14% to 23%) |
| Alerts not reviewed (N=250) | |
| Percent of alerts with an ADE ∗ (95% CI) | 35% (17% to 52%) |
| Percent of alerts with a PADE ∗ (95% CI) | 10% (4% to 25%) |
| Over-all Rate (N=516) | |
| Percent of alerts with an ADE ∗ (95% CI) | 23% (12% to 34%) |
| Percent of alerts with a PADE ∗ (95% CI) | 15% (6% to 24%) |
∗ Based on 10% sample of patients.
ADE = adverse drug event; PADE = potential adverse drug event.
When we reviewed cases that triggered reviewer calls, we found that there were 4 ADEs and 11 potential ADEs among the 30 alerts (positive predictive value of 15/30 or 50%) among these 26 patients. All 4 ADEs were serious and all 4 were also considered preventable. Of the 11 potential ADEs, 1 was classified as life-threatening, 8 as serious, and 2 as significant (▶). In all 15 cases, the resident physician caring for the patient was not aware of the ADE or PADE. In 14 of the 15 cases, the resident physician made a change in clinical care (changing the dose of the medication or stopping the offending medication).
Two ADEs and 8 potential ADEs in this group were related either to renal failure from a drug or to inappropriate dosing in the setting of renal failure (▶). One of these cases, sulindac and increasing creatinine, was a situation in which a patient with congestive heart failure and baseline renal insufficiency was given sulindac with resultant renal failure. The resident physicians were not aware that the sulindac was a likely culprit in this instance and after receiving the call from the reviewer, the medication was stopped. One ADE and one PADE were due to the use of anti-thrombotic agents in the setting of low platelet count leading either to bleeding (ADE) or to severe thrombocytopenia (PADE).
Table 3
Table 3 Frequency of Alert Drug and Lab for all ADEs and Potential ADEs Detected in Alerts with MD contact (N=15)
| Alert Drug | Alert Lab | Number of Alerts | Adverse Event |
|---|---|---|---|
| ADEs | |||
| Sulindac | Serum Creatinine | 1 | Renal Failure |
| Famotidine | Low Platelet Count | 1 | Bleeding |
| Digoxin | Serum Creatinine | 1 | Arrhythmia |
| Carbamazepine | Pancytopenia | 1 | Worsening infection |
| Total | 4 | ||
| Potential ADEs | |||
| Vancomycin | Serum Creatinine | 3 | Markedly elevated vancomycin levels |
| Levofloxacin | Serum Creatinine | 3 | Inappropriately high levofloxacin dose |
| Enoxaparin | Low Platelet Count | 2 | Potential bleeding |
| Enoxaparin | Serum Creatinine | 1 | Potential bleeding |
| Clopidogrel, aspirin | Low Platelet Count | 1 | Potential bleeding |
| NSAIDs, Aspirin | Serum Creatinine | 1 | Potential bleeding |
| Total | 11 | ||
ADE = adverse drug effect; PADE = potential adverse drug effect.
The DPV system helped identify several patients who had potential ADEs due to very high vancomycin levels. The triggering rule was vancomycin and elevated creatinine but on further examination, the reviewer found that vancomycin levels were nearly three times the upper limit of therapeutic range. When the reviewer contacted the medical teams, each was aware of the elevated levels but the clinician neither knew the reasons for the elevated levels nor the fact that there were other patients (cared for by other clinicians) who had similarly elevated levels during the same time period. The reviewer saw the multiple alerts and was able to determine that the underlying issue was dual administration of vancomycin in patients on hemodialysis. Three patients had received vancomycin on the wards where a CPOE system had been partly implemented, while the attending physician in the dialysis unit, where CPOE had not yet been installed, was using paper records to administer the dose in the dialysis unit. Neither the medical team nor the dialysis physician was aware that the other was giving the patient vancomycin. Catching this double-dosing allowed the hospital to create a new program in the pharmacy that would check for these duplicate orders.
When we examined a sample of the 236 alerts that were examined by the reviewer but no action was taken, we found that 12% of these alerts were associated with adverse drug events and another 18% were associated with potential adverse drug events. These typically involved abnormal potassium levels or rising creatinine in the setting of nephrotoxic drugs—but in each instance, the physician had already begun to address the issue by the time the reviewer had examined the alert.
Alerts Not Reviewed and Their Outcomes
There were 250 (48% of all alerts) among 200 patients that were not evaluated by a reviewer. An examination of a 10% sample of these charts revealed 7 ADEs and 2 potential ADEs for an adverse drug event rate of 35% (95% confidence interval 17% to 52%) and a PADE rate of 10% (95% CI 4% to 25%). Each of the 7 ADEs was designated as serious and all 7 were preventable (▶). Examples of ADEs in this context included a case of a patient admitted for a scalp laceration (and evaluation of a fall) who was on enoxaparin for prophylaxis. He had worsening renal failure in the hospital but did not have his enoxaparin dose adjusted and he subsequently had bleeding from a scalp laceration requiring transfusion. Another case was a patient admitted with a foot infection and renal insufficiency (due to dehydration) who was started on a high dose of gentamicin. The patient had worsening renal failure several days later which was likely due to the gentamicin. The gentamicin was eventually changed to a non-nephrotoxic antibiotic and the patient's renal function recovered.
When we combined all of our results from both those reviewed and those not reviewed and examined overall ADE and PADE rates among all high priority alerts, we found that 23% of alerts (95% confidence interval 12% to 34%) were associated with an ADE and an additional 15% of alerts (95% CI 6% to 24%) were associated with a PADE for a combined rate of 38% (95% CI 25% to 51%).
Comparison of Cost of Monitoring with Costs of ADEs
There were important costs of monitoring that must be considered. The reviewer typically spent 1.5 hours per day during the 70 days of the study period when he or she was able to review the alerts. If the hourly wage of the reviewer, who was typically a pharmacist, was $75 per hour (including benefits), the estimated costs for using the pharmacist were approximately $7,900 during the study period, annualized to approximately $40,000 (if we assume daily monitoring for 365 days per year). Further, for a 150-bed community-based hospital, the retail price for installation is approximately $25,000 with annual licensing fees of approximately $35,000. A hospital that chose to have daily monitoring by a pharmacist might spend up to $80,000 annually in pharmacist time, licensing cost, and installation (amortized over five years) cost. Given that each preventable adverse event costs well over $5,000 in 2007 dollars, 20 one would have to prevent approximately 16 ADEs to recoup the costs of the system and the time it takes a pharmacist to review the alerts and contact physicians when appropriate. Of course, whether hospitals recoup the costs of prevented ADEs is dependent heavily on payer-mix and how hospitals are reimbursed.
Discussion
We prospectively evaluated a commercially-available adverse drug event monitoring system in a community hospital and found that it was effective at identifying ADEs and potential ADEs. When used in conjunction with a pharmacist reviewer who examined the charts daily, the system identified a large number of ADEs and potential ADEs that were unknown to the physician caring for the patient in nearly every instance and lead to a change in clinician decision making. The system was efficient, requiring less than 90 minutes of pharmacist time to evaluate patients on 4 large hospital wards and an ICU. Only about half of signals were reviewed for logistical reasons; the sample of signals that occurred on off hours revealed an even higher rate of adverse events and near misses, which might be expected, as staffing is generally lower during off-peak times. Our study offers evidence that a commercially-available system was easily deployed in a community-based setting and was effective at detecting and preventing ADEs.
Others have evaluated electronic monitoring systems before, 1,2,9,21-23 but they have usually been examined in large academic centers using homegrown systems. In this study, we evaluated a commercially available program that can be implemented in community-based hospitals even if they do not have pre-existing advanced information systems. While others have examined ADE surveillance in community hospitals, 21,24 one recent study is particularly relevant. Kilbridge and colleagues recently used a combination of automated surveillance and voluntary reporting to compare rates and characteristics of ADEs in an academic and a community hospital in the Duke University Health System. 25 They found that the ADE rates in the community hospital were substantially higher than those in the academic medical center. While the Kilbridge evaluation was innovative in examining the effectiveness of computerized ADE monitoring in community hospitals, they used a home-grown system that worked well because the community hospital shared common structural components and IT infrastructure with the academic center.
Although few hospitals have adopted computerized surveillance systems to address ADEs, some community-based hospitals have implemented other measures. Cohen et al. examined efforts by Missouri Baptist Medical Center to decrease ADEs by attempting to change the safety culture of the institution, hiring a patient safety specialist and offering rewards to staff for safety ideas. 26 An automated surveillance system might have helped to determine if their efforts were successful.
For hospitals looking for tools to screen patients and track ADEs, our approach may be reasonable. While prior studies have found that approximately 4% to 6% of hospitalized patients suffer an ADE, the rate of ADEs among patients with alerts generated by the DPV system was substantially higher, suggesting that this tool likely identifies a group of high risk patients. Therefore, the DPV can even be used to identify a pool of high-risk patients. The ability to customize would allow individual hospitals to choose or modify rules to meet their clinical needs and to have the capacity to follow up signals. Furthermore, it is important to note that this approach is still beneficial even if CPOE is in place, because even though CPOE reduces medication errors, 18 many errors and ADEs still occur in hospitals with CPOE. 27 In addition, CPOE can create new errors, which computerized monitoring can help identify.
An important use of this program was the live-time alerts which, when used in conjunction with a reviewer, seemed effective at identifying ADEs and potential ADEs early and helping to change clinical management. We were only able to have a covering reviewer for approximately half the days that the system was running and therefore, reviewed high priority alerts on 152 patients. However, the reviewer intervened on 1 in 6 of these patients (30 phone calls on 26 patients) leading to a change in practice in 15 cases. Therefore, among all patients with a high priority alert, a reviewer was able to dismiss alerts on 80% of patients as being not significant without having to make a call. The phone calls, which occurred on 20% of high priority alerts, led to a change in clinical care in 50% of the cases.
There are two other benefits of this program. First, its ability to highlight the mini-epidemic of elevated vancomycin levels was extremely useful in allowing the hospital to design a program to prevent duplicate orders. Over time, systems such as DPV will likely help highlight other defects and errors that occur in hospitals' medication ordering and administration systems. A second benefit is that the DPV system highlighted important areas for future education to improve medication prescribing. For example, we found that a large number of alerts were due to inappropriate dosing in the setting of renal dysfunction. This could easily become a target for continuing medical education for the physicians who cared for patients in a particular hospital. Alternatively, an organization might decide to prioritize implementation of renal dosing decision support, which has been found to be effective. 28 By engaging in ongoing monitoring, hospitals have the ability to identify where important deficiencies might lie and target interventions to rectify those deficiencies.
Our study has important limitations. First, while we evaluated one particular system, there are other commercial systems now becoming available that have very similar features. It is possible, even likely, that they can be effective at identifying and possibly preventing ADEs. Secondly, the DPV system was examined with a limited set of rules and did not include data from other key areas such as microbiology, because coded microbiology data were not available. Prior studies have shown that a major cause of ADEs (and other adverse events) in community hospitals is antibiotic-associated colitis or other hospital-acquired infections. 21 A modified system that included data from these or other types of results (such as pathology or cardiology) would allow more robust rules to be created to improve quality and safety in these hospitals. Another limitation of the study is that although the alerts were generally evaluated in the morning (based on activations that had occurred up to 24 hours earlier), it is possible that physicians would have identified those issues themselves even without reviewer intervention.
Another important limitation of our study is that we only sampled 10% of the alerts that did not lead to phone calls by the reviewer for further chart review. Therefore, although we were able to make estimates about the rates of ADEs and potential ADEs in those and other groups of patients, given the small sample sizes, our confidence intervals for most of our estimates were wide. Although we were able to identify a high rate of ADEs using the DPV system, it clearly missed many types of ADEs, including those that lead to symptoms not identifiable by this screening tool (i.e., vomiting due to a medication). Finally, two important questions remain: are commercially-available systems truly widely deployable, and will other hospitals that adopt such systems see comparable benefits in detecting ADEs?
In conclusion, we examined the DPV surveillance tool in a community hospital and found that it readily identified a group of patients who were at high risk for an adverse drug event or potential adverse drug event. Further, we found that when a reviewer was available to spend less than 90 minutes a day to review the alerts and make phone calls, the DPV system allowed the reviewer to intervene and change clinical practice, likely averting a substantial number of ADEs. Although the system could benefit from additional refinement, its ability to be deployed in hospitals with rudimentary electronic systems makes it one effective option for hospitals dedicated to monitoring and reducing ADEs.
Footnotes
Dr. Jha was supported by the Physician Faculty Scholar program of the Robert Wood Johnson Foundation.
The VigiLanz Corporation provided limited financial support for this project. The hospital paid a discounted price (compared to the retail price) for the use of the software. The company provided no resources for the evaluation of its product and had no input into the analytic plan. The lead author (AKJ) did not receive any financial compensation, although one co-author (DWB) did receive an honorarium from VigiLanz for a talk he gave on ADE monitoring at the annual meeting of the American Society for Health Systems Pharmacy. This talk was given prior to the initiation of any planning for the study. Finally, the company had no editorial input into the writing of the manuscript and was not shown any drafts of the manuscript before submission. All data were collected and analyzed independently of the company.