Development and validation of a delirium predictive score in older people

Abstract

Background: delirium is frequently under diagnosed in older hospitalised patients. Predictive models have not been widely incorporated in clinical practice.

Objective: to develop and validate a predictive score for incident delirium.

Design and setting: two consecutive observational prospective cohorts (development and validation) in a university affiliated hospital.

Subjects: inpatients 65 years and older.

Methods: in the development cohort patients were assessed within the first 48 h of admission, and every 48 h thereafter, using the confusion assessment method to diagnose delirium and data were collected on comorbidity, illness severity, functional status and laboratory. Delirium predictive score (DPS) was constructed in the development cohort using variables associated with incident delirium in the multivariate analysis (P < 0.05), and then tested in a validation cohort of comparable patients, admitted without delirium. Receiver operating characteristic (ROC) analysis and likelihood ratio (LR) were calculated.

Results: the development cohort included 374 patients, incident delirium occurred in 25. After multivariate analysis incident delirium was independently associated with lower functional status (Barthel Index) and a proxy for dehydration (elevated urea to creatinine ratio). Using these variables, DPS was constructed with a performance in the ROC curve area of 0.86 (95% CI: 0.82–0.91) and (−) LR = 0.16 and (+) LR = 3.4. The validation cohort included 104 patients and the performance of the score was ROC 0.78 (95% CI: 0.66–0.90).

Conclusions: This simple predictive model highlights functional status and a proxy for dehydration as a useful tool for identifying older patients that may benefit from close monitoring and preventive care for early diagnosis of delirium.

Introduction

Delirium is one of the most frequent complications in geriatric patients who are hospitalised with a prevalence approaching 42% in medical settings [1]. It adversely affects prognosis in terms of longer hospital stays also higher rates of morbidity, functional disability, institutionalisation, dementia and mortality [1–4]. Although there is robust evidence regarding the impact of delirium, it has been difficult for clinicians to address this disorder as it continues to be under diagnosed and undertreated [5,6]. Delirium is considered a benchmark for quality of care [7].

To improve prognosis researchers and clinicians have suggested a global approach to management that includes primary prevention to reduce incidence of delirium and secondary prevention towards early detection and treatment [5,8]. Although these steps are essential for patient care, identifying risk factors for delirium upon admission is the first step towards better preventive and diagnostic strategies, and has been suggested as a quality indicator for delirium care in hospitals [9].

Attempts to develop predictive models for delirium in older hospitalised medical patients have been previously conducted. Inouye et al. [10] developed a predictive model for incident delirium based on four independent baseline risk factors: vision impairment, severity of acute illness, cognitive impairment and dehydration. Pompei et al. [11] developed a model that included demographic characteristics, cognitive and functional status, depression and alcoholism. Furthermore, the model by O'Keeffe and Lavan [12] is based on the identification of dementia, severe illness and elevated serum urea levels. These models have 79% of accuracy in the best case, according to their ROC performance [12].

Previous models have provided important information about the main risk and precipitating factors for delirium validated in different prospective cohorts. Nevertheless, they are not regularly used in clinical practice, because they require special assessments that are not commonly used in medical settings [Acute Physiology and Chronic Health Evaluation II (APACHE), MMSE, visual assessment, among others]. Moreover, cognitive assessments if not done properly, may misdiagnose delirium or previous cognitive impairment, both conditions that are highly under diagnosed in acute settings [13–15].

The aim of this study was to develop a clinical model for incident delirium that could be easily used at admission, to help clinicians classify patients according to their risk of developing delirium during hospitalisation and so facilitate awareness, prevention, early detection and treatment.

Methods

Study design

Two different prospective cohorts were studied: the first cohort was used to develop the clinical model (development cohort) and the second was used to test the model (validation cohort). The study protocol was approved by the ethics committee of the Medical Faculty of the Pontificia Universidad Católica de Chile; and informed consent was obtained from all participants or their surrogates prior to enrolment.

Development cohort

Details about this cohort (enrollment and characteristics) have been published previously [4]. In summary, consecutive patients 65 years and older, admitted to the general medical ward of a university affiliated hospital in the previous 48 h, were enrolled. Exclusion criteria included evidence of severe aphasia, coma and inability to participate in cognitive assessments.

Participants were assessed for delirium by a psycho-geriatric medical team within 48 h of admission and every 48 h thereafter until discharge or for a maximum of 12 days. Delirium was established using the confusion assessment method (CAM) [16], which was adapted and validated for Spanish speakers [17]. This method relies on clinically relevant information obtained from the caregiver and the patient. It provides a diagnostic algorithm for delirium based on the presence of the two cardinal features: (i) acute onset and fluctuating course; and (ii) inattention, and at least one of the two secondary features: disorganised thinking or altered level of consciousness.

Data collected at enrolment included demographic characteristics (age and gender) and laboratory test values from the first 48 h since admission (serum sodium, creatinine, urea nitrogen (BUN), albumin, sedimentation rate, C-reactive protein). The serum BUN to creatinine ratio was calculated as a proxy for dehydration, according to previous reports of delirium risk [10]. To assess the severity of acute illness Acute Physiology and Chronic Health Evaluation II (APACHE II) [18] was calculated, considering that if a laboratory measurement was not determined there was no reason to assume that the missing variable had an abnormal value, and we imputed the mean normal value. To estimate burden of comorbidity, the Charlson comorbidity index was used [19]. The investigation team did not ask for any additional cognitive or laboratory measurement, and only those asked by treating physicians were recorded.

Baseline functional status was estimated at patient performance 2 weeks prior to admission using the Barthel Index [20] and Pfeffer Functional Assessment Questionnaire [21], with information given by a caregiver. The Barthel Index measures performance ability in 10 basic activities in daily life, including need of assistance in feeding, moving from bed to chair, personal toileting, getting on and off toilet, bathing, walking on level surface, going up and down stairs, dressing, controlling bowels and bladder. Using a quantitative estimation of the patient's level of dependency and yielding a score from 0 (poor) to 100 (good). The Pfeffer Functional Activities Questionnaire [21], is also a functional assessment, but assesses instrumental activities such as the ability to handle own money, do shopping, laundry, using transport, dispensing medication, prepare meals and turn on/off the oven, remembering names of friends or relatives, ability to be left alone at home safely and to walk without disorientation in a neighbourhood; scores exceeding 7 are associated with dementia, and used as a proxy for prior dementia [22].

If delirium was present in the first study visit, the patient was considered to have prevalent delirium and was excluded for further analysis. If the patient developed delirium after the first visit, then this was considered as incident delirium and included in the delirium cohort. If delirium was not identified, the patient was included in the non-delirium cohort.

Statistical analysis and model development

A descriptive analysis was carried out, based on the calculation of means and standard deviation (SD) for continuous variables, and frequencies as percentages for categorical variables. Univariate analyses (t and χ² tests) were performed to determine the association between patient clinical and laboratory characteristics and incident delirium in the development cohort. Variables significantly associated with incident delirium (P < 0.05) were included in a stepwise logistic regression analysis with forward selection. Using variables that remained significant in this multivariate analysis a predictive risk score for incident delirium was constructed using the linear prediction rule. The score was calculated and tested for the ability to predict delirium in the development cohort using receiver operating characteristic (ROC) analyses [23] and likelihood ratios [24].

Validation cohort

A new cohort was enrolled in the same setting, using the same inclusion/exclusion criteria, to test the predictive value for incident delirium of the developed model. The performance of the rule in the cohort was explored using ROC curve analyses [23].

Results

Development cohort

A total of 542 patients were enrolled with a mean age of 78 years (SD 7.6), 62% female. Delirium was present at the first assessment in 168 patients (31%), because they were considered to have prevalent delirium they were excluded from further analysis.

The remaining participants were 374 patients and their characteristics are described in Table 1. During their hospital stay, 25 patients developed incident delirium.

Incident delirium was significantly associated (P<0.05) with older age, lower baseline functional status (lower Barthel Index and higher Pfeffer functional score), higher Charlson comorbidity index, serum creatinine levels, and higher BUN creatinine ratio. Other variables such as gender, APACHE score, albumin, haemoglobin, erythrocyte sedimentation rates, sodium levels, and C-reactive protein levels were not significantly associated with delirium (Table 1).

The relationship among significant risk factors, using the Spearman analysis, showed that the higher correlation was between Barthel and Pfeffer (coefficient Rho −0.746 P < 0.001), as was expected being both functional measures (Supplementary data are available in Age and Ageing online, Table S2).

The performance of the score using the ROC curve, showed an area under the curve of 0.86 (95% CI: 0.82–0.91) for the risk stratification system, as shown in Figure 1.

Testing for different cut points, we selected a value of −240, because of its high sensitivity and specificity, with values equal or lower showing a negative predictive value for incident delirium of 0.99 with a specificity of 0.74 and sensitivity of 0.88, with a negative likelihood ratio LR 0.16 (95% CI: 0.06–0.47) and positive LR of 3.39 (95% CI: 2.83–4.06).

This means that if one starts with a 20% pre-test probability of incident delirium, a DPS score <−240, reduces the post-test probability near to 3%. In the same example, if the DPS score is >−240, it increases the post-test probability of incident delirium to up to 45%.

In the case of using conventional units for BUN and creatinine values, the score would be the following: Delirium predictive score (DPS) = [5 × BUN (mg/dl)/creatinine (mg/dl)] – (3 × Barthel index), the cut point in this case is −160 and the performance of the model is the same.

Validation cohort

The validation cohort included 104 patients admitted without delirium, with a mean age of 75.5 years (SD: 7.4) and a baseline Barthel index mean score of 91.3 (SD: 14.5). These values were not statistically different from the development cohort (t-test P > 0.05). The general laboratory data (sodium, albumin, creatinine, BUN, and C-reactive protein levels) were also not statistically different from the development cohort. During hospitalisation delirium developed in 12 patients of the validation cohort. The predictive performance of the model in the validation cohort showed an area under the ROC curve of 0.78 (95% CI: 0.66–0.90) for the DPS (Figure 2).

Discussion

This study presents a simple predictive model (DPS) for incident delirium that helps clinicians classify older medical patients at admission according to their risk of developing delirium. It is based on two variables: previous functional status measured as lower Barthel Index and BUN creatinina ratio, a proxy for dehydration.

When the DPS was subsequently tested in a different cohort, it confirmed its predictive validity, meaning that if a patient has a previous good functional status and is not admitted with dehydration it is very unlikely that he/she will develop delirium, independent of age.

Functional status or the ability to perform daily life activities is currently recognised as basic information for geriatric practice that is easily obtained from a patient's medical history or caregiver. While there are different scales to evaluate functional status, the Barthel index has been validated in different languages and different settings and is considered one of the most reproducible, reliable and time-efficient tests for this purpose [25]. Functional disabilities is a relevant determinant of frailty, and frailty has been associated with a higher delirium risk that contributes to its poor outcomes [26, 27], our findings support these previous reports.

On the other hand, cognitive impairment is known as an important risk factor for delirium [28], as well as for poor outcomes during hospitalisation [29]. In this study, previous cognitive status was assessed by the Pfeffer scale, which evaluates functional abilities including many functions dependent on cognition, with information given by caregiver. This scale has been used and is currently accepted as a proxy for cognitive impairment [22], recognising the intimate link between functional status and chronic cognitive impairment. Our data showed Pffefer to be highly correlated with the Barthel index, but the prediction ability regarding delirium, after multivariate analysis was only significant for the Barthel index, so Pffefer was not finally included in the score. Delirium risk could be related to muscular performance as a proxy for frailty, this hypothesis requires further study.

The second predictive variable is a proxy of dehydration, measured as a high blood BUN creatinine ratio [30]. Hydration is one of the most common risk factors for delirium, especially relevant in older patients and is considered a main target of delirium prevention strategies [8]. Tendency to dehydration in the elderly is multifactorial in origin, including decreased in total body water, diminished urinary concentration capacity, reduced thirst response to dehydration and many times poor access to water if the patient is functionally impaired as may happen in hospital settings.

To develop this delirium risk score, different variables were tested, only age, Pfeffer Functional score, APACHE II and creatinine levels were significant in the univariate analysis but they lost their association in the multivariate analysis, showing that dehydration (elevated BUN/creatinine) and functional status were stronger predictors.

This finding matches some of the previous delirium predictive models [10], dehydration was also found by Inouye and O'Keefe [12] as a relevant risk factor of delirium and functional disability was present in Pompei's model [11]. Unlike others our model does not include Mini-Mental State Examination, vision impairment nor APACHE score, we believe this is an advantage because they are not routine assessments in daily practice and may limit models applicability.

A potential limitation of the study is that incident delirium rate was lower than expected, maybe associated with delays in admission in our medical care system that could increase the probability of being admitted with delirium. Another potential limitation of the study is that the ability of the score to predict cases of delirium could be better [(+) LR 3.4], but on the other hand, it has good performance in terms of a negative likelihood ratio [(−) LR 0.16], so it helps to focus delirium prevention strategies and active delirium assessments in those with a positive DPS (higher risk). These findings cannot be generalised to surgery or intensive care patients, so future studies are required in these settings.

Despite these constraints, the DPS model has several important strengths. First, it was developed in a large sample of geriatric patients, with little exclusion, and therefore represents real-life, hospitalised, older medical patients. Moreover, CAM screening was completed fully and performed every 48 h for up to 2 weeks before discharge, by geriatricians and psychiatric medical doctors and this optimises diagnosis specificity and sensibility. The predictive model generates gradients for delirium risk and was prospectively validated in a clinically distinct population.

This model helps to identify patients who are very unlikely to develop delirium during a hospital stay without the need to assess variables that require a more exhaustive evaluation. Its simplicity and applicability can help to facilitate awareness for diagnosis as well as to focus assessment and prevention strategies with cost-effectiveness in general practice.

This DPS predictive model highlights the relevance of functional assessment and adequate hydration in every older patient admitted to hospital settings as a useful tool to promote optimal clinical care and to contribute to focus resources in higher risk patients.

Author contributions

The submitted manuscript reflects the combined efforts of all its authors.

Conflict of interest

None declared.

Funding

This work was supported by DIPUC grants No. 2005/15PI from Pontificia Universidad Católica de Chile. There was no sponsor involvement in any aspect of this clinical research study.

Supplementary data

Supplementary data mentioned in the text is available to subscribers in Age and Ageing online.

References