Simone Rios Fonseca Rittera,b; Thayana Louize Vicentini Zoccolia; Marina Machado Pereira Linsa; Anne Freitas Cardosoa; Marco Polo Dias Freitasa; Einstein Francisco Camargosa,b
OBJECTIVE: Delirium is one of the most frequent syndromes among elderly patients admitted to emergency units and, despite presenting well-established symptoms and signs, there are still diagnostic failures. Thus, the aim of the study was to adapt the Abbreviated Mental Test (AMT) as a screening tool for delirium in elderly adults admitted to an emergency department.
METHODS: This cross-sectional study was conducted at the emergency department of a university hospital in Brasília, Brazil between April and June, 2014. We evaluated 90 patients of both sexes, aged 60 years or older. The Confusion Assessment Method O (CAM) Instrument was considered the gold standard for diagnosing delirium. The complete translated AMT, administered in four different culturally adapted models, as well as condensed models, were compared to the CAM receiver operating characteristics curve (significance < 0.05; H0: AUC = 0.5). Inter-rater agreement was evaluated with the kappa test, using SPSS version 220.127.116.11.
RESULTS: The prevalence of delirium was 25.6%. The best of the four AMT models presented sensitivity and specificity of 78.3% and 85.1%, respectively, with good inter-rater reproducibility (Kappa = 0.793). The best condensed model included four questions, with sensitivity and specificity of 82.6% and 82.1%, respectively, and a kappa of 0.746.
CONCLUSIONS: Compared to the gold standard, the adapted AMT (complete or condensed) was adequate as an alternative for quick delirium screening in elderly patients admitted to an emergency department, especially for unaccompanied patients with no prior cognitive deficit.
Keywords: delirium; emergencies; health services for the aged.
OBJETIVO: O delirium é uma das síndromes mais frequentes em idosos admitidos nas unidades de urgência e, apesar de apresentar sintomas e sinais bem estabelecidos, ainda há falhas de diagnóstico. Dessa forma, o objetivo do estudo foi adaptar o Abbreviated Mental Test Score(AMT) como teste de rastreio para delirium em idosos admitidos em uma unidade de urgência.
MÉTODOS: Estudo transversal realizado no serviço de urgência de um hospital universitário em Brasília, Brasil, no período de abril a junho de 2014. Foram avaliados 90 pacientes de ambos os sexos, com 60 anos ou mais de idade. O Confusion Assessment Method (CAM) Instrument foi estabelecido como padrão ouro para diagnóstico de delirium. Foram aplicadas questões do AMT em 2 versão traduzida e adaptada culturalmente, com quatro modelos distintos e outros reduzidos, todos comparados ao CAM a partir da curva característica de operação do receptor (ROC) (significância < 0,05; H0: ASC = 0,5) e à reprodutibilidade interavaliadores a partir do índice Kappa, por meio do SPSS, versão 18.104.22.168.
RESULTADOS: A prevalência de delirium foi de 25,6%. O melhor dos quatro modelos do AMT apresentou sensibilidade de 78,3%, especificidade de 85,1%, com boa reprodutibilidade interavaliador (Kappa = 0,793). O melhor modelo reduzido foi o com quatro questões, com sensibilidade de 82,6%, especificidade de 82,1% e Kappa de 0,746.
CONCLUSÕES: A adaptação do AMT (completa e reduzida) mostrou-se adequada como alternativa para o rastreio breve de delirium em idosos admitidos em unidades de urgência quando comparada ao padrão ouro, principalmente para pacientes desacompanhados e sem déficit cognitivo prévio.
Palavras-chave: delirium; urgência; assistência à saúde do idoso.
The number of older adults who receive emergency services is increasing, mainly due to increased life expectancy and, consequently, comorbidities, in addition to the fact that the prognoses of older patients are worse than those of younger patients in these settings.1,2 Delirium is one of the most frequent syndromes among older patients admitted to these units, and these are strategic sites for detection and early clinical intervention, which can significantly improve prognosis.3
The diagnosis of delirium is clinical, characterized by: sudden onset with fluctuations during the day, altered level of consciousness, global disturbance of cognition or perceptual and behavioral abnormalities (apathy, the most common, can lead to a large proportion of undiagnosed cases) and evidence of an organic cause.4-6 The syndrome is associated with longer hospitalization, increased costs, cognitive and functional decline, higher mortality rates and institutionalization.3,5,7
Based on the high prevalence and incidence of cognitive changes in elderly adults, it is recommended that all patients admitted to emergency departments be evaluated for the possibility of delirium or dementia, especially those who are not alert or are disoriented, whose behavior changes while in the unit or who appear to have some other type of alteration; this step is critical for subsequent care.1,8-10 There is much discussion in the literature about which screening method for this syndrome is the best, although a number of reliable and validated tools exist.1,7,9,10 Although the systematic use of delirium scales increases the detection rate, some are too long to be used routinely.4,10 According to Hendry et al., the ideal screening tool for delirium should be brief, require little or no training and be appropriate for the clinical setting in which it is used.10
The Confusion Assessment Method (CAM) is a bedside scale developed to assist untrained professionals in the rapid and accurate diagnosis of delirium.11,12 It presents high sensitivity (94 to 100%), specificity (90 to 95%), inter-rater reliability and a high negative predictive value (90 to 100%). It is particularly useful for those at risk, such as elderly adults hospitalized for clinical or surgical reasons.11 This scale has been translated and validated for Brazil, and its systematic use in emergency departments may increase the diagnosis rate and reduce the morbidity and mortality associated with delirium.4 To be diagnosed with delirium according to the CAM, acute onset and fluctuating attention must be present, as well as disorganized thinking and/or a change in consciousness level.4,13
Delirium can also be screened using brief cognitive assessment tools.14 The Abbreviated Mental Test (AMT) is a ten-item screening tool originally described as capable of recognizing cognitive impairment in patients with scores below 7.15 The instrument has been routinely used to detect and monitor the presence of cognitive impairment, is easy to apply and is often recommended as a tool of choice in the initial assessment of elderly hospital patients.16,17 An English study on surgical patients aged 65 and older validated the AMT as a screening tool for delirium based on serial evaluations.18
Thus, this study aimed to adapt the AMT and search for evidence of validation as a screening test for delirium in elderly emergency department patients.
This cross-sectional study with a convenience sample was conducted at the Emergency Care Center of the University Hospital of Brasília (Hospital Universitário de Brasília — HUB). This unit serves Brazilian Unified Health System patients from the metropolitan area of the Federal District.
The included individuals were patients aged 60 or older who had been admitted for observation between April and June 2014 and considered capable of completing the evaluations. Elderly patients who were discharged immediately after the evaluation and those who were critically unstable, requiring intensive support and continuous monitoring, were excluded since the study protocol might have prolonged the treatment and thus impacted their prognosis.
The sample was calculated based on the prevalence of delirium in emergency departments, which ranges from 10 to 30%.5,7,19,20 Considering a prevalence of 20%, an absolute precision of 10% and a significance level of 5%, a sample of 61 participants was calculated. The ideal sample size, taking a possible loss of 20% into account, was 74 individuals.
The English version of the AMT was used as the reference for the present study. The instrument was translated into Brazilian Portuguese separately by two experienced geriatrics researchers fluent in English, and the translations were then synthesized into a unique instrument culturally adapted for the Brazilian population. All ten questions of the instrument were translated, although question 8 (“Year of the First World War”) was considered culturally inadequate for elderly Brazilians. Therefore, four alternate questions were developed for testing (see Chart 1): “What is Brazil’s Independence Day?” (“Qualéo dia da independência do Brasil?’"), “In what year was Brazil discovered?” ( “Qualéo ano do descobrimento do Brasil?”), “Who discovered Brazil?” (Quem descobriu o Brasil?), “Who was president when Brasilia was built?” (“Quem foi o presidente da república que construiu Brasília?”).
Elderly patients admitted to the Emergency Care Center were evaluated at two different times by two geriatric residents. In the first evaluation, sociodemographic, psycho-behavioral and clinical data were obtained. A version of the CAM that had been translated and validated for Brazil was administered to the patient’s companion,4 while a complete adapted AMT version was administered to the patient. At the time of the first evaluation, the complementary exam data requested by the doctor on call were also collected. Within 12 hours of the first evaluation, a second evaluator, blind to the results of the prior assessment, again administered the adapted AMT.
During the first evaluation, the elderly patient and/ or caregivers were asked why they sought emergency care, besides variables such as age (in years), sex (male/female), education, marital status (married, single, widowed, other), previous morbidities, current medications, lifestyle (alcohol and smoking) and, for patients, self-perception of health (excellent, very good, good, bad or very bad).
During the study period, 113 patients aged 60 older were admitted to the HUB Emergency Care Center for observation and invited to participate in the study. Three refused to participate and 11 were excluded because they did not undergo AMT reassessment. Since the AMT is a tool for cognitive screening and the objective of this study was to evaluate its applicability for acute screening, nine patients were excluded due to having been previously diagnosed with dementia and, therefore, had a recognized cognitive impairment. Thus, the final sample consisted of 90 patients.
The variables were described as means or frequencies and their respective standard deviations and percentages. The outcome variable was the occurrence of delirium as diagnosed by the CAM, which is considered the gold standard for this purpose.1 The translated and adapted AMT models were compared with the gold standard (CAM) using receiver operating characteristic (ROC) curve analysis.
The first and second evaluators completed the AMT with the patient, and the first evaluator also filled out the CAM. The AMT evaluations were used to determine the inter-rater reliability and reproducibility of the adapted AMT. Only the second rater’s evaluation was used for validation because the first evaluator also administered the CAM and was no longer blind to the delirium diagnosis.
The inter-rater analysis was performed using the kappa method, which evaluates the agreement of qualitative data. Kappa values vary between 0 and 1, with a kappa of 1 expressing perfect agreement. A value between 0.21 and 0.40 is considered weak; 0.41 to 0.60 is moderate; 0.61 to 0.80 is substantial; and 0.81 to 0.99 is almost perfect agreement. These were the cut-off values used in the present study.21
Four complete ten-question screening models were initially tested, each containing one of the four replacements for the original AMT question 8. The abbreviated models were then tested by selecting questions from individual tests. All models were compared with the CAM based on the ROC curve, a significance < 0.05 and the largest areas under the curve (AUC), using a null hypothesis at AUC = 0.05 (H0: AUC = 0.5). Finally, three models with the highest sensitivity and specificity standards were selected, one complete model and two condensed versions (one with seven and one with four items).
To evaluate the likelihood that patients who tested positive actually had delirium and that those who tested negative actually did not, positive predictive values (PPV) and negative predictive values (NPV) were calculated for each of the three models, as well as true positives (TP: positive responses to both the AMT and the CAM), true negatives (TN: negative for both tests), false positives (FP: positive for the AMT but negative for the CAM) and false negatives (FN: negative for the AMT but positive for the CAM). The PPV were calculated as the ratio between TP and the sum of TP and FP, while the NPV were calculated as the ratio between theTN and the sum of the TN and FN. The statistical analyses were performed in SPSS version 22.214.171.124.
This study was approved by the Research Ethics Committee of the Faculty of Health Sciences of the University of Brasília (CEP/FS-UnB), case 15390513.1.0000.0030. An informed consent form was signed by the patient or caregiver, depending on the patient’s cognitive ability at the time of the evaluation.
In the sample of 90 patients (50 men: 55.6%), the mean age of the group was 71.6 ± 8.1 years; 23 (25.6%) patients had delirium according to the CAM diagnosis; 88 patients reported their education level, and 67 (76.1%) had up to four years of schooling; 38 (42.2%) patients did not have a spouse; of the 87 who reported their self-perception of health, 29 (32.2%) considered it to be poor or very poor.
To assess their significance separately, each of the 13 translated and adapted AMT questions (10 items, including the four alternates for question 8) was tested separately based on the area under the ROC curve. Age, time, year, hospital name, recognition of two people, date of birth, and the ability to count backwards from 20 to 1 were questions that, in isolation, were significant in identifying acute mental confusion in the sample, as shown in Table 1.
Based on the individual sensitivity analysis for each question, models were constructed to identify the best questionnaire for recognizing delirium in elderly patients treated in emergency departments. The four initial models included ten questions, differing in cultural adaption for question 8. To develop a test with adequate sensitivity and specificity that could be quickly administered, condensed models were then produced using only questions that, according to the individual analysis, had the greatest potential for identifying mental confusion. The tested questionnaires are shown in Table 2.
Eight models were created to test for appropriateness: four ten-item models (each of these featured one culturally appropriate alternative for question 8); two seven-item models; one five-item model and one four-item model. For the ten-item questionnaires, the cut-off point for minimum correct responses was 7 or more. For the seven-item models, two cut-off points were tested: six or more and five or more correct responses. For the five-question model, the cut-off was 4 or more correct responses; and for the four-item model, the cut-off was three. For the five- and four-item questionnaires, only questions that, in the isolated analysis, presented area under the ROC curve above 0.690 and 0.700, respectively, were considered.
The inter-rater agreement analyses for the various models are shown in Table 3. All models showed substantial agreement according to the kappa test.
The delirium diagnosis agreement data for the CAM and the three most adequate AMT models, i.e., with ten (model C), seven and four questions, are shown in Table 4. The ten-(AMT10) and four-item (AMT4) models are shown in Charts 2 and 3, respectively.
As shown in Table 4, the PPV and NPV were calculated for the three best models proposed. For the complete model (C-AMT10), the PPV was 64.3% and the NPV was 91.9%. For the seven-item abbreviated AMT, using a cut-off point > 5, the PPV was 66.7% and the NPV was 92.1%. For the four-item abbreviated AMT, the PPV was 61.3%, and the NPV was 93.2%.
The present study aimed to adapt the AMT as a screening test for delirium in elderly patients admitted to emergency departments in Brazil. The complete model (C-AMT10) performed well: with a cut-off of seven or more correct responses, it had a sensitivity of 78.3% and a specificity of 85.1% for recognizing delirium. The prevalence of delirium in this study was 25.6%, which is consistent with the literature.5,7’19’20’22’23 To the best of our knowledge, the AMT is the first tool adapted for the Brazilian population to evaluate delirium in emergency units that can be administered directly to the patient, without the need for informants. Together with the short administration time (less than 1 minute), the fact that the AMT does not rely on informants, who are often unavailable during the first evaluation of elderly patients,18 allows a quick assessment of cognitive status in emergency departments, which may in fact reduce the time of delirium diagnosis.
If a test can be abbreviated without appreciable loss of discriminatory power, it becomes even more useful, due to the fact that it will be better accepted and receive greater patient cooperation.15 The performance of the seven-item model with a cut-off point of five or more correct responses was superior to that of the ten-item model (AMT10).
Brief cognitive tests have been commonly used in hospitals in England and have been validated for acute care units, including the ten- and four-item AMT, with good sensitivity for detecting delirium, as well as for identifying patients with previously diagnosed severe dementia. The AMT10 was not superior to the AMT4, which favors the use of the latter.2,10,14 The specificity of the AMT is greater in patients with larger deficits (two or more errors for the complete test), suggesting that both severe disorientation and inability to perform simple cognitive tests are useful markers for diagnosing delirium.7 Among the seven questions found to be significant for identifying acute mental confusion, the original AMT4 involves individually significant questions (date of birth, hospital name, age, and year), and scores below 4 on this model are strongly correlated with the complete ten-item test.24 H owever, this specific set of questions was not selected when validating the four-item assessment tool because a different association of questions presented better AUC, sensitivity and specificity.
Because this four-item model (AMT4) with a cut-off point of three or more correct responses showed even better sensitivity (82.1%) than the seven- and ten-item tests while retaining good specificity (82.6%), it could be the best choice for use in emergency departments, thus guaranteeing a high index of diagnostic suspicion and close follow-up of these patients. The kappa index for the ten- (AMT10), seven- and four-item (AMT4) models indicated adequate reproducibility, similar to that of the Brazilian CAM validation study.2
No screening tool is perfect by itself. It should be emphasized that in clinical practice delirium could signify an important underlying disease, which requires sensitivity.3 Simple tools, such as the four-item AMT, have relatively good sensitivity to detect impairment, which requires subsequent evaluation. In addition, the test’s simplicity makes it accessible for use by non-specialists.1,7,10 The results of this study reinforce this finding and validate a four-item screening tool (AMT4) for the Brazilian population. In a hospital setting, the AMT proved to be an effective tool and comparable to the Mini-Mental State Exam for predicting a patient’s cognitive condition in both the full test25 and the four-item condensed version, which was strongly correlated with the complete AMT10.26 The AMT4 presented the best NPV, which was comparable to that of the Brazilian CAM, thus making it the best model to rule out the presence of this syndrome.4
This study has certain limitations, and it is important to highlight the exclusion of patients with dementia, which may have underestimated the prevalence of delirium and prevented this tool from being evaluated in this specific population profile. Likewise, the exclusion of patients with higher and lower clinical stability may also have impacted the prevalence rates.
Because it is an instrument for evaluating current cognition, we recognize that it cannot predict the patient’s prior cognition. However, recognizing that dementia is a known risk factor for the development of delirium,7,22,27-29 the screening of any cognitive impairment already benefits the identification of susceptible patients. Since the present study validated the use of a tool for evaluating delirium in elderly patients admitted to an emergency department, the extrapolation of the results to very different environments is questionable. Therefore, it would be advisable to replicate it in new studies.
Compared to another previously validated instrument, the adapted AMT, in both its complete and condensed forms, was found to be adequate as an alternative for delirium screening in elderly patients admitted to emergency departments, being useful mainly for unaccompanied patients with no previous cognitive deficit. The adapted AMT also presented good inter-rater reproducibility. Due to the AMT4’s low PPV but high sensitivity and NPV, a two-step approach is suggested for identifying delirium: first, applying the instrument, then complementing this with a more detailed clinical evaluation of identified patients.
CONFLICT OF INTERESTS
The authors declare no conflicts of interest.
1. Ellis G, Marshall T, Ritchie C. Comprehensive geriatric assessment in the emergency department. Clin Interv Aging. 2014;9:2033-43. https://doi.org/10.2147/CIA.S29662
2. Dyer AH, Briggs R, Nabeel S, O`Neill D, Kennelly SP. The Abbreviated Mental Test 4 for cognitive screening of older adults presenting to the Emergency Department. Eur J Emerg Med. 2017;24(6);417-22. https://doi.org/10.1097/MEJ.0000000000000394
3. Elie M, Rousseau F, Cole M, Primeau F, McCusker J, Bellavance F. Prevalence and detection of delirium in elderly emergency department patients. CMAJ. 2000;163:977-81.
4. Fabbri RM, Moreira MA, Garrido R, Almeida OP. Validity and reliability of the Portuguese version of the Confusion Assessment Method (CAM) for the detection of delirium in the elderly. Arq Neuropsiquiatr. 2001;59:175-9. http://dx.doi.org/10.1590/S0004-282X2001000200004
5. Cordeiro RLR, Ventura MM, Damian PB, Gomes ALR, Leite SP. Fatores relacionados ao óbito e à internação prolongada em uma enfermaria de geriatria. Geriatr Gerontol Aging. 2016;10(3):146-50. http://dx.doi.org/10.5327/Z2447-211520161600009
6. Martins S, Fernandes L. Delirium in elderly people: a review. Front Neurol. 2012;3:101. https://doi.org/10.3389/fneur.2012.00101
7. Bellelli G, Morandi A, Davis DH, Mazzola P, Turco R, Gentile S, et al. Validation of the 4AT, a new instrument for rapid delirium screening: a study in 234 hospitalised older people. Age Ageing. 2014;43:496- 502. https://doi.org/10.1093/ageing/afu021
8. Foreman MD, Milisen K. Improving recognition of delirium in the elderly. Primary Psychiatry. 2004;11:46-50.
9. Rosen T, Connors S, Clark S, Halpern A, Stern ME, DeWald J, et al. Assessment and Management of Delirium in Older Adults in the Emergency Department: Literature Review to Inform Development of a Novel Clinical Protocol. Adv Emerg Nurs J. 2015;37:183-96; quiz E3. https://doi.org/10.1097/TME.0000000000000066
10. Hendry K, Quinn TJ, Evans J, Scortichini V, Miller H, Burns J, et al. Evaluation of delirium screening tools in geriatric medical inpatients: a diagnostic test accuracy study. Age Ageing. 2016;45:832-7. https://doi.org/10.1093/ageing/afw130
11. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113:941-8.
12. Sieber FE. Postoperative delirium in the elderly surgical patient. Anesthesiol Clin. 2009;27:451-64, table of contents. https://doi.org/10.1016/j.anclin. 2009.07.009
13. Adamis D, Treloar A, MacDonald AJ, Martin FC. Concurrent validity of two instruments (the Confusion Assessment Method and the Delirium Rating Scale) in the detection of delirium among older medical inpatients. Age Ageing. 2005;34:72-5. https://doi.org/10.1093/ageing/afi019
14. Lees R, Corbet S, Johnston C, Moffitt E, Shaw G, Quinn TJ. Test accuracy of short screening tests for diagnosis of delirium or cognitive impairment in an acute stroke unit setting. Stroke. 2013;44:3078-83. https://doi.org/10.1161/STR0KEAHA.113.001724
15. Hodkinson HM. Evaluation of a mental test score for assessment of mental impairment in the elderly. Age Ageing. 1972;1:233-8.
16. Holmes J, Gilbody S. Differences in use of abbreviated mental test score by geriatricians and psychiatrists. BMJ. 1996;313:465. https://doi.org/10.1136/bmj.313.7055.465
17. Burleigh E, Reeves I, McAlpine C, Davie J. Can doctors predict patients' abbreviated mental test scores. Age Ageing. 2002;31:303-6.
18. Ní Chonchubhair A, Valacio R, Kelly J, O'Keefe S. Use of the abbreviated mental test to detect postoperative delirium in elderly people. Br J Anaesth. 1995;75:481-2.
19. Miller MO. Evaluation and management of delirium in hospitalized older patients. Am Fam Physician. 2008;78:1265-70.
20. Jackson TA, MacLullich AM, Gladman JR, Lord JM, Sheehan
B. Undiagnosed long-term cognitive impairment in acutely hospitalised older medical patients with delirium: a prospective cohort study. Age Ageing. 2016;45:493-9. https://doi.org/10.1093/ageing/afw064
21. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37:360-3.
22. Pendlebury ST, Lovett NG, Smith SC, Dutta N, Bendon C, Lloyd-Lavery A, et al. Observational, longitudinal study of delirium in consecutive unselected acute medical admissions: age-specific rates and associated factors, mortality and re-admission. BMJ Open. 2015;5:e007808. https://doi.org/10.1136/bmjopen-2015-007808
Delirium screening in elderly adults
23. Chin YC, Koh GC, Tay YK, Tan CH, Merchant RA. Underdiagnosis of delirium on admission and prediction of patients who will develop delirium during their inpatient stay: a pilot study. Singapore Med J. 2016;57:18-21. https://dx.doi.org/10.11622%2Fsmedj.2016007
24. Swain DG, Nightingale PG. Evaluation of a shortened version of the Abbreviated Mental Test in a series of elderly patients. Clin Rehabil. 1997;11:243-8. https://doi.org/10.1177/026921559701100308
25. Swain DG, O'Brien AG, Nightingale PG. Cognitive assessment in elderly patients admitted to hospital: the relationship between the
Abbreviated Mental Test and the Mini-Mental State Examination. Clin Rehabil. 1999;13:503-8. https://doi.org/10.1191/026921599670895633
26. Swain DG, O'Brien AG, Nightingale PG. Cognitive assessment in elderly patients admitted to hospital: the relationship between the shortened version of the Abbreviated Mental Test and the Abbreviated Mental Test and Mini-Mental State Examination. Clin Rehabil. 2000;14:608-10. https://doi.org/10.1191/0269215500cr368oa
27. Taylor D, Lewis S. Delirium. J Neurol Neurosurg Psychiatry. 1993;56:742-51.
28. Chan PK. Clarifying the confusion about confusion: current practices in managing geriatric delirium. BCMJ. 2011;53:409-15.
29. Caplan G. Managing delirium in older patients. Aust Prescr. 2011:16-8. https://doi.org/10.18773/austprescr.2011.012
June 15 2018.
Accepted em June 15 2018.