Prevention of Intraoperative Awareness with Explicit Recall in an Unselected Surgical Population: A Randomized Comparative Effectiveness Trial

• Intraoperative awareness with explicit recall occurs in 0.15% of surgical cases

• Randomized trials of bispectral index monitoring have focused on high-risk patients, but few data are available in an unselected surgical population

• In this large effectiveness study (n = 21,601), no significant difference in intraoperative awareness with explicit recall was detected between bispectral index and anesthetic concentration protocols (0.08 vs.  0.12%, P = 0.48) but, by post hoc  analysis, bispectral index monitoring may decrease intraoperative awareness compared with routine care without a protocol

INTRAOPERATIVE awareness with explicit recall of surgical events is a potentially devastating event associated with posttraumatic stress disorder1and has an incidence of approximately 0.15% for all risk levels.2,3Processed electroencephalographic monitors have been developed to assess anesthetic depth and potentially prevent intraoperative awareness, which is considered a sentinel event by the Joint Commission.4The Bispectral Index® (BIS) monitor (Covidien, Boulder, CO) processes a frontal electroencephalographic channel to calculate a dimensionless number from 100 (awake) to 0 (no detectable brain activity) to provide a measure of the patient's level of consciousness; a BIS range of 40–60 is suggested to be consistent with the state of general anesthesia.5,–,7 

Past efficacy trials have evaluated the role of protocols based on the BIS monitor5,–,7and anesthetic concentrations6,7for the prevention of intraoperative awareness. However, these studies were performed exclusively in patients at high risk for the complication. A large cohort study did find that BIS monitoring decreased the incidence of intraoperative awareness in a broad surgical population compared with historical controls, but was limited by its observational design, changing practice patterns regarding end-tidal anesthetic concentration monitoring, and exclusion of patients not receiving neuromuscular blockers.8As such, there are currently no comparative effectiveness data to guide the decisions of providers or policymakers as they attempt to prevent intraoperative awareness in the more than 200 million major surgeries performed worldwide each year.9 

Similarly, there are no effectiveness data supporting the claim that anesthetic consumption is reduced with the use of a BIS monitor, which has been suggested to decrease inhaled anesthetic use by up to 38%.10These data are reinforced by meta-analyses of small efficacy trials of both inhaled and intravenous anesthesia.11,12It has recently been argued that decreased anesthetic use and the ensuing clinical benefits, such as faster recovery or reduced nausea and vomiting, make the BIS monitor cost-effective and that it should therefore be routinely incorporated.13 

Here we describe a comparative effectiveness study with active comparators and a two-sided superiority design. This randomized controlled trial compared alerting protocols based on either anesthetic concentration or BIS values in an unselected surgical population at three hospitals within a tertiary academic medical center. The primary outcome was the incidence of definite awareness; prespecified secondary outcomes included the incidence of definite or possible awareness, as well as anesthetic usage and recovery variables.

A detailed description of the experimental protocol for the Michigan Awareness Control Study (ClinicalTrials.gov No. NCT00689091) has been previously reported.14The conduct of the study and the reporting of results followed the Consolidated Standards of Reporting Trials guidelines.15 

The study received approval from the Institutional Review Board of the University of Michigan, Ann Arbor, and was deemed to be of minimal risk. A full discussion of the risks and benefits was conducted with each patient approached. Patient consent to interventions and follow-up was electronically documented in our perioperative information system (Centricity®, General Electric Healthcare, Waukesha, WI). Patients were recruited from three hospitals of the University of Michigan Health System (main university hospital, cardiovascular center, ambulatory surgery center) from May 2008 until May 2010. Inclusion criteria were age more than 18 yr, general anesthesia using inhalational or intravenous technique for any surgical case that did not involve the forehead, and availability for follow-up interviews. Exclusion criteria were intracranial procedures, adhesive allergy, psychosis, or history of traumatic brain injury. All patients enrolled in the study were blinded to group assignment and had the BIS electrode applied to the left side of the forehead by a member of the research staff before entering the operating room.

To detect a reduction in the incidence of intraoperative awareness from 0.15% to 0.04%,8we calculated a need for 14,072 per group, or a total n = 28,144 with 80% power and a significance level of 5%. We targeted a total recruitment of 30,000 patients, with a prespecified interim analysis after 20,000 patients were recruited (2/3 target sample).14A constant likelihood group sequential method with formal futility boundaries was used with a two-sided O'Brien–Fleming stopping rule. There was no contingency for early termination for efficacy. An acceptance region plot (or a futility region plot) was generated using SAS statistical software (SAS version 9.2, Carey, NC). The two-sided futility boundary (for the differences in proportions between the BIS and the anesthetic concentration group) at the planned interim analysis was from −0.0005434 to 0.0005434. The difference between the proportions observed at the interim analysis was 0.0003275422 (11/9,376 cases of definite awareness in the anesthetic concentration group minus 8/9,460 cases of definite awareness in the BIS group), which is within the stopping boundary for futility.

The University of Michigan Health System utilizes the Centricity® electronic perioperative information system in all of its operating rooms. Using this system, automated real-time analysis of BIS values or minimum alveolar concentration (MAC) was performed every 5 min, with the transmission of provider-specific electronic alphanumeric paging alerts in less than 60 s. Operating rooms were randomized every 3 months based on even or odd room numbers to either (1) electronic alerts in the event of median BIS values more than 60, or (2) electronic alerts for median age-adjusted MAC level of less than 0.5. The threshold of age-adjusted MAC less than 0.5 was chosen based on a retrospective analysis of electronically documented cases with and without awareness that occurred before the onset of the study,16as well as the high frequency with which thresholds of higher MAC are crossed.6In addition to the age-adjusted MAC of standard inhaled anesthetics, alerting based on anesthetic concentrations also reflected documented intravenous anesthetic infusions and bolus doses.16Paging alerts to the clinician electronically signed into and physically present during the case reported either the median BIS value or anesthetic concentration level for the prior 5 min epoch, followed by “Potentially insufficient anesthesia – please check vaporizers and intravenous lines.”

In the BIS-targeted rooms, BIS values appeared on the main monitoring screen and were automatically recorded. In the anesthetic alert-targeted rooms, BIS values neither appeared on the monitor nor were accessible intraoperatively. Other aspects of anesthetic care (e.g. , choice of anesthetic agents, benzodiazepines) were not standardized for this study.

Randomization was performed using a random-number, computer-generated block scheme based on even or odd operating room number. The blocks were defined within a specific year of the study based on the original start date of recruitment. The study year was divided into four quarters by calendar month (3 months per quarter). Within a specific study year, the odd-numbered operating rooms and even-numbered operating rooms were randomized to BIS alerting two times and anesthetic concentration alerting two times. If the odd-numbered operating rooms were randomized to one alerting protocol, the even-numbered operating rooms were randomized to the alternative alerting protocol for that quarter of the study year. Patients, postoperative interviewers, and all case reviewers were blinded to group assignment. Practitioners receiving pages regarding BIS or MAC values were not blinded to group assignment. However, practitioners were not made aware of the randomization scheme or dates for randomization change during the study.

The BIS monitors used in the Michigan Awareness Control Study were not free-standing devices, but modules that interfaced with the Solar 9500 (General Electric) anesthetic monitors used in our institution's operating rooms. During scheduled quality-control checks within the first 2 months of the trial, it became clear that in some instances there was a failure of BIS values to be generated. Technical representatives from both manufacturers confirmed this as a known software interface problem. Because the study was designed as an effectiveness trial, the decision was made to proceed and use the population receiving neither the BIS nor anesthetic concentration protocol as a post hoc “no intervention” group for the purpose of secondary analysis. Failure to generate BIS values was similar in both even- (17%) and odd-numbered (19%) operating rooms, which was the randomization scheme for alerting protocols.

Blinded, trained interviewers used the modified Brice interview17employed in other studies of intraoperative awareness2,5,–,7to screen patients 28–30 days after surgery via  telephone. A single interview was performed in contrast to past trials5,–,7because of the high number of patients recruited; the 28- to 30-day interview was chosen because it would likely detect the most clinically significant awareness events. If patients could not be reached by telephone after multiple attempts, a written form of the interview was sent to the patient. Any patients reporting intraoperative awareness during the Brice interview had a more detailed interview by an anesthesiologist committee member blinded to the intervention. All patients reporting intraoperative awareness were offered psychiatric care.

For those patients who reported awareness, three blinded experts independently determined whether the reported event was definite, possible, or no awareness based on the data obtained from the first two interviews (Brice screening and follow-up). These individuals also reviewed awareness events for the BAG-RECALL trial (ClinicalTrials.gov No. NCT00682825).7,18We compared interrater agreement using Fleiss's κ statistic for the three blinded assessments of awareness, which showed fair agreement (0.25). In the event of a conflict, a fourth blinded expert reviewer from another institution made the final determination; this expert reviews cases for the American Society of Anesthesiologists Anesthesia Awareness Registry. The qualitative aspects of the awareness report were classified using the Michigan Awareness Classification Instrument.19Class 1 is defined as isolated auditory perceptions, class 2 is tactile perceptions, class 3 is pain, class 4 is paralysis, and class 5 is paralysis and pain. If an event is also associated with distress, the class number is modified with a “D.”

Anesthetic usage, time to meeting recovery room discharge criteria, and incidence of postoperative nausea/vomiting were prespecified secondary outcomes.14Postanesthesia care unit discharge criteria include (among other variables): oxygen saturation more than 92% or preoperative baseline (at appropriate levels of supplemental oxygen), core temperature between 36° and 38° Celsius, normal heart rate and rhythm (or no worse than baseline status), other hemodynamic vital signs within normal physiologic range for age or within 20% of baseline values, normal neurologic evaluation, pain score of 4 or less, postoperative nausea and vomiting score ⩽2. BIS values, MAC values, and doses of propofol, midazolam, fentanyl, and morphine were assessed across all groups.

The primary outcome was the incidence of definite intraoperative awareness in the anesthetic concentration and BIS groups using modified intention-to-treat analysis. Modified intention-to-treat was defined as a patient who was randomized and was interviewed at 30 days. Prespecified secondary analysis was conducted to determine the combined incidence of definite and possible awareness as well as the classification of events. Significance was assessed using a two-tailed chi-square test. Confidence intervals were calculated using Newcombe method without continuity correction.20The average number of paging alerts generated in the groups was compared with the incidence of definite or possible awareness events using a linear regression r-squared test.

Patient characteristics, comorbidities, and risk factors for awareness were analyzed to determine if there were statistically significant differences between the anesthetic concentration and BIS groups in the modified intention-to-treat analysis. The Kolmogorov–Smirnov statistic was used to determine normality for the two continuous variables (age and body mass index). If the P  value was significant (<0.05), the assumption of normality was violated, and nonparametric analyses (e.g. , Mann–Whitney U test) were used. Nonparametric data are presented as median and interquartile range (25th–75th percentile). Parametric data are presented as mean ± SD. For categorical variables, a two-tailed chi-square test was used, where P < 0.05 was considered statistically significant. All categorical data are presented as number (percentage). For ease of interpretation we have defined cardiovascular disease as having one or more of the following conditions: history of myocardial infarction, congestive heart failure, valvular heart disease, dysrhythmia, endocarditis, peripheral vascular occlusive disease, angina, or orthopnea. We have defined lung disease as having one or more of the following conditions: history of pulmonary hypertension, chronic obstructive pulmonary disease, or dyspnea. We have defined liver disease as having one or more of the following conditions: history of cirrhosis, acute liver failure, or chronic liver failure. We have defined neuropsychiatric disease as having one or more of the following conditions: history of stroke or transient ischemic attack, seizures, depression, bipolar disorder, anxiety disorder, or posttraumatic stress disorder. We have defined alcohol abuse as having three or more drinks daily and/or high withdrawal potential.

For the other key secondary outcomes, all continuous elements were assessed for normality as described. We chose to use the post hoc  grouping variable (anesthetic concentration, BIS, and no intervention) to assess the secondary outcomes, and therefore post hoc  comparison testing was employed. The median BIS values were compared between the BIS and anesthetic concentration groups using a Mann–Whitney U test; a Kruskal-Wallis test was used to compare median anesthetic dosages and discharge times among the anesthetic concentration, BIS, and no intervention groups. A two-tailed chi-square test was used to compare the outcomes of nausea or vomiting among the three groups. Bonferroni correction was used for the Mann–Whitney U test variables. For the variables that were analyzed using the Kruskal-Wallis test, pairwise comparisons using a series of Mann–Whitney U tests were performed if the omnibus test was significant. For the Bonferroni correction, we started at an α level of 0.05. Based on the number of comparisons required, the new α level to measure significance was 0.002. We calculated a total of 22 comparisons based on the number of embedded Mann–Whitney U tests that were performed for Kruskal-Wallis tests with significant omnibus tests. Only those pairwise comparisons with P < 0.002 were reported in table 2as statistically significant differences. If there were no statistically significant pairwise comparisons, “NS” (no significance) was reported for ease of interpretation. Statistical software IBM SPSS statistics version 19 (IBM Corp, Somers, NY) was used.

A total of 21,601 patients were enrolled in the study at the time of interim analysis, with a 97% recruitment rate (fig. 1). As described in the methods, the study was terminated because of futility. Of the study cohort, 18,836 or 87% of the patients were available for postoperative interview assessing awareness at 1 month; 9,460 patients were randomized to the BIS group and 9,376 patients were randomized to the anesthetic concentration group (fig. 1). Patient characteristics and comorbidities for the modified intention-to-treat BIS and anesthetic concentration groups are demonstrated in table 1. There were no adverse events related to the study.

Of the 9,460 patients randomized to the BIS intervention and successfully interviewed, 3,384 or 36% did not have BIS data recorded because of technical issues described in Materials and Methods. This population was used for secondary analysis only as a post hoc  control group because it had neither intervention; there were more females (P < 0.001) and more patients with lung disease (P = 0.002) in this group. Neither female sex nor lung disease was shown to be associated with an increased incidence of intraoperative awareness in our recent companion randomized controlled trial.7 

The overall incidence of definite awareness in the study cohort was 19/18,836 or 0.1%. By modified intention-to-treat analysis, the incidence of definite awareness was 11/9,376 or 0.12% (95% CI: 0.07–0.21%) in the group randomized to the anesthetic concentration protocol and 8/9,460 or 0.08% (95% CI: 0.04–0.16%) in the group that was randomized to receive BIS monitoring (P = 0.48, fig. 2). Using the Michigan Awareness Classification Instrument, no statistical differences in event or distress classes were found between the groups. Post hoc  power analysis revealed that 102,951 patients in each group would be required to detect a difference between the two interventions. The 13% of recruited patients who did not complete interviews (e.g. , because of death or lack of response) were unlikely to skew the reported incidence of intraoperative awareness. Assuming the same incidence rates found in the modified intention-to-treat groups, 100,000 simulations were run to generate cumulative distribution functions that demonstrate the probability of a significant difference of outcome if the 2,765 patients not interviewed were included. Using a Fisher exact test, the likelihood of a significant difference with inclusion of this population was 0.016%.

By post hoc  analysis, the incidence of definite awareness was 11/9,376 or 0.12% in the anesthetic concentration group, 3/6,076 or 0.05% in the group that actually received BIS monitoring, and 5/3,384 or 0.15% in the no intervention group (P = 0.27). Based on the 0.12% awareness incidence in the anesthetic concentration group and the 0.05% awareness incidence in the group that received BIS monitoring, a post hoc  power analysis revealed that 29,996 patients in each group would be required to detect a difference between the two interventions. The combined incidence of definite and possible awareness cases was 0.08% in the group that received BIS monitoring, 0.20% in the anesthetic concentration group, and 0.38% in the no intervention group (P = 0.006, fig. 3). By post hoc  analysis, the cohort receiving no intervention had 4.7 times more definite or possible awareness events compared with the cohort receiving the BIS protocol (P = 0.001; 95% CI: 1.7–13.1). Of patients with definite or possible awareness receiving BIS monitoring, 50% had no 5-min epoch of BIS values less than 60 during the case and 50% had at least one 5-min epoch of median BIS value more than 60.

By secondary analysis using post hoc  grouping, the average number of alerts in the no intervention group (0/case), anesthetic concentration group (1/case), and BIS group (2.2/case) varied inversely with the incidence of definite and possible awareness events (r²= 0.951).

The secondary outcome measures of anesthetic use and recovery times were performed using the post hoc  comparison groups of anesthetic concentration, BIS, and no intervention. Because the decision was made to present the data using the three post hoc  groups instead of the modified intention-to-treat grouping (BIS or anesthetic concentration), Bonferroni corrections were performed as described in Materials and Methods. Data are presented in table 2, with only significant pairwise comparisons reported. There was a statistically significant difference in the median MAC for pairwise comparisons of anesthetic concentration to no intervention groups and also for BIS to no intervention groups. Intraoperative propofol bolus dosing showed a significant pairwise comparison between the BIS and no intervention groups. The total midazolam dose showed no statistically significant differences. Total fentanyl and total morphine use had statistically significant pairwise comparisons for all combinations of the three grouping variables. Although statistically significant, the clinical relevance of these differences is unclear.

Median time to meeting recovery room discharge criteria was 98 min (interquartile range: 66–140) for the anesthetic concentration group, 95 min (interquartile range: 64–138) for the BIS group, and 94 min (interquartile range: 64–133) for the no intervention group. There was a significant pairwise comparison between the no intervention and anesthetic concentration groups. There was no evidence for reduced recovery time in patients receiving BIS monitoring compared with no intervention. There was no statistically significant difference among the three groups for reduced nausea or reduced vomiting upon first assessment in the recovery room (table 2).

This is the largest prospective randomized controlled trial ever conducted on the prevention of intraoperative awareness, and the only such effectiveness trial. This negative study was unable to determine if an alerting protocol based on BIS values or anesthetic concentration was superior in preventing definite intraoperative awareness. Other conclusions of the study are that (1) comparative effectiveness trials with definitive results regarding the prevention of intraoperative awareness in unselected patients will likely not be feasible, (2) post hoc  secondary analysis suggests that a protocol based on the BIS monitor probably reduces awareness events compared with routine care without a protocol, (3) increased provider alerting is a possible mechanism for decreasing awareness events when comparing two protocols, (4) the BIS monitoring protocol used in this trial is not associated with a reduction in the use of anesthetic drugs in routine clinical practice, and (5) the BIS monitoring protocol used in this trial is not associated with reduced recovery time or incidence of nausea and vomiting in routine clinical practice.

The B-Aware study demonstrated that a BIS-guided protocol significantly reduced the incidence of intraoperative awareness in a high-risk population compared with no intervention.5Subsequently, the B-Unaware study demonstrated no difference between a BIS-guided and MAC-guided protocol in the high-risk population,6a finding supported by the recent BAG-RECALL trial.7The current study differs from all past trials in that it assessed awareness prevention in an unselected, representative surgical population as opposed to the high-risk population alone. The primary results of our study are consistent with the B-Unaware and BAG-RECALL trials in that no statistically significant difference in the prevention of intraoperative awareness could be demonstrated between anesthetic concentration and BIS monitoring protocols. However, the results of the post hoc  secondary analysis are consistent with the B-Aware trial5in that the BIS monitor showed a trend toward reducing the incidence of awareness events compared with a group with no intervention. One methodological similarity of the current trial, the B-Aware trial5and the observational study by Ekman et al.  7is that anesthetic administration was not restricted to potent inhaled agents alone, as it was in the B-Unaware and BAG-RECALL trials.6,7Our study supports the conclusion of a recent Cochrane database review suggesting that the BIS monitor may reduce awareness when compared with assessing clinical signs alone, but not when compared with a protocol based on anesthetic concentration.12 

The use of the BIS monitor in the current study generated approximately twice as many alerts as that of the anesthetic concentration protocol. Therefore, increased alerting could potentially be a mechanism of decreased definite or possible awareness events, an interpretation supported by the results of our companion trial. In the BAG-RECALL study, the alarm frequency based on anesthetic concentration was approximately 2-fold higher than that based on BIS values; the higher alarm rate with the anesthetic concentration protocol was associated with fewer definite and possible awareness events. The different alerting threshold in BAG-RECALL (0.7 MAC) and the current trial (0.5 MAC) likely explains the ostensibly disparate outcomes. It is important to note that there was a high incidence of false positive alerting, which mitigates any conclusion regarding alerting protocols as a method of preventing awareness.

Efficacy trials and meta-analyses have suggested that the BIS monitor can significantly reduce consumption of anesthetic drugs, which leads to improved outcomes such as faster recovery or reduced nausea and vomiting. These data have been used to argue that the BIS monitor is cost-effective and should be routinely adopted for every general anesthetic.13The BIS protocol used in the current study was not shown to reduce anesthetic dosing, which is in contrast to the recent Cochrane database review.12Furthermore, the BIS protocol used in the current study was not associated with reduced recovery time or reduced incidence of nausea and vomiting compared with routine care. One hypothesis to explain the discrepancy is that conclusions derived from efficacy trials or meta-analyses based on such trials are not sufficiently robust to hold in a test of effectiveness. Another hypothesis to explain the discrepancy is that the difference in BIS-guided protocols between the current and past studies led to disparate outcomes.

Limitations of our study include insufficient numbers to answer with precision whether and to what extent there is a difference in the definite awareness incidence between protocols based on BIS values and anesthetic concentrations. This limitation likely reflects the rarity of intraoperative awareness in an unselected surgical population and is informative regarding the future investigation of protocols to reduce awareness. Another limitation of the trial was the proportion of patients randomized to the BIS protocol who did not receive BIS monitoring. However, this unplanned technical issue has yielded useful secondary findings and is mitigated by the following considerations: (1) even complete compliance would almost certainly not have been sufficient to detect a significant difference in the modified intention-to-treat groups; (2) the population receiving neither intervention yielded useful information regarding the effect of anesthetic protocols compared with routine care, a matter of recent controversy;21(3) the incidence of definite and possible awareness events in the no intervention group was equivalent to that previously reported,2,3which validates the methodology of the trial and suggests that a single interview at 30 days was sufficient to detect clinically relevant intraoperative awareness; and (4) the number of prospectively studied patients who received BIS monitoring nonetheless exceeds all major efficacy trials combined.5,–,7 

In conclusion, this effectiveness study could not detect a difference between BIS and anesthetic concentration protocols in reducing the incidence of definite intraoperative awareness with explicit recall. By post hoc  analysis, we demonstrated that the BIS monitor may play a role in reducing intraoperative awareness compared with no intervention. These findings are consistent with conclusions of a Cochrane review based on various efficacy studies.12In contrast to the Cochrane review, the BIS protocol used in this study was not associated with improved recovery.

The authors thank the many anesthesia technicians, anesthesiology residents, anesthesiology faculty, biomedical engineers, information technology staff, medical students, nurse anesthetists, research assistants, and research support staff in the Department of Anesthesiology at the University of Michigan Medical School, Ann Arbor, Michigan, for their essential contributions to this study. The authors thank Christopher Kent, M.D., Assistant Professor, Department of Anesthesiology, University of Washington, Seattle, Washington, as well as Alex Evers, M.D., Professor and Chair, Department of Anesthesiology, Washington University, St. Louis, Missouri, and Charles Hantler, M.D., Professor, Department of Anesthesiology, Washington University, for serving as expert reviewers of awareness cases.