Validation of the Freund Clock Drawing Test as a screening tool to detect cognitive dysfunction in elderly cancer patients undergoing comprehensive geriatric assessment

We aimed to validate the Freund Clock Drawing Test (CDT), with its predefined cutoff score of ≤4, as a screening tool to detect elderly cancer patients in need of a more in‐depth cognitive evaluation within a comprehensive geriatric assessment (CGA).


Introduction
As a result of the aging of populations, there is currently a demographic evolution particularly in Western countries. These demographic changes have triggered an increased interest in the multidisciplinary management of elderly patients since the latter is a heterogeneous group that is in need of a more individualized treatment approach [1,2]. Tailored care can be facilitated through a comprehensive geriatric assessment (CGA), which has been the cornerstone in the management of geriatric patients for years [3]. implement a CGA in an elderly oncology population, with success, as it has now been proposed as the key treatment approach [7,8].
The Folstein Mini Mental State Examination (MMSE) is a standard validated measure to screen cognitive function within a CGA. Studies have noted that up to 40% of elderly cancer patients present with cognitive abnormalities that warranted further evaluation. Cognitive dysfunctions can influence the ability to weigh the risks and benefits of cancer therapy, comply with the suggested treatment plan, and decrease the ability to recognize the symptoms of toxicity that need medical attention [9]. The MMSE can be used to screen for dementia and to estimate the severity of cognitive impairment in a general population and in elderly cancer patients [10][11][12]. However, in an oncogeriatric population, where the majority of patients has a normal cognitive function, such assessment can be experienced as tedious and time-consuming, as it may take up to 10-15 min to carry out [13,14]. More recently, the Clock Drawing Test (CDT) has been proposed as a quick and simple screening tool to assess cognitive dysfunction as it can be completed in only 5 minutes [15]. The CDT evaluates multiple domains of cognition including memory, comprehensive and executive function, visuo-spatial ability, and abstract thinking [16,17]. Furthermore, when given a pre-drawn circle, the CDT is not influenced by education age [18]. Although the CDT has the characteristics of an attractive screening tool, an easy and straightforward scoring method and validated cutoff scores were still lacking. Therefore, our research group retrospectively reviewed the Freund scoring system, as it has been reported in literature as a fast, easy, and trustworthy scoring method [18]. A retrospective analysis on 105 elderly cancer patients at the General Hospital Groeninge showed that a cutoff score of ≤4 for the CDT had a good area under the curve (AUC), sensitivity (Se), and specificity (Sp). The same cutoff score appeared optimal in a general geriatric population. Furthermore, the Freund scoring system demonstrated high interrater reliability [11,19].
In this prospective trial, our primary endpoint was to prospectively validate the Freund CDT, with its predefined cutoff score of ≤4, as a screening tool to detect cognitive deterioration in elderly cancer patients within a CGA.

Patient selection
This prospective study (PROACTIVE trial, ClinicalTrials. gov identifier: NCT01749995) was conducted from November 2012 till December 2013 in patients aged 70 years or older with a histologically confirmed diagnosis of a solid cancer or hematologic malignancy at all four sites of the General Hospital Groeninge (Kortrijk, Belgium).
Patients, receiving their primary oncology care (surgery, course of (neo)adjuvant or palliative chemotherapy, radiotherapy, targeted therapy, palliative care, experimental treatment as part of a clinical trial,…) could be included before or at the start of a line of treatment but not during a line of treatment. Eligible patients were screened with the G8-questionnaire before or after they had received their cancer diagnosis, as part of routine clinical practice [20]. Patients who screened positive on the G8 (cutoff ≤14) were evaluated with a full CGA and were subsequently invited to participate in this trial. In a limited number of cases, a CGA was performed irrespective of the G8 test score because of a referral by the treating physician on the basis of clinical suspicion of vulnerability or frailty. This trial was approved by the ethical committee of the General Hospital Groeninge (Kortrijk, Belgium).

Comprehensive geriatric assessment and cognitive measures
Cognitive function was assessed as part of a routine oncogeriatric assessment or CGA. The CGA comprised several domains, each assessed with a standard validated measure: nutrition (Mini Nutritional Assessment-Short Form [21]), functional status (activities of daily living, instrumental activities of daily living [22,23]), physical status (number of falls, JAMAR ® Hydraulic Hand Dynamometer [24]), depression (Geriatric Depression Scale-15 [25]), cognition (MMSE, Freund CDT [12,19]), polypharmacy (number of drugs), and comorbidities (Charlson Comorbidity Index [26]). In accordance with previous reports, patients were deemed vulnerable if they presented with impairments in two or more domains within the CGA [3,27]. The CGA, including MMSE and Freund CDT, was conducted by an oncopsychologist or research associate with experience in the field of oncogeriatrics. Both had received training from an occupational therapist, enabling them to conduct and score the Folstein MMSE according to international guidelines [28]. Patients were considered to be potentially cognitively impaired if they presented with a test score of 23 or less [13]. Potentially cognitively impaired means that a patient has to be referred to a neurologist or memory clinic for a more in-depth cognitive assessment. For the CDT, patients were given a pre-drawn circle and were verbally instructed to put all the numbers of a clock on it and set the time at ten past eleven, as this has been reported to be the most sensitive for detecting neurocognitive impairments [29]. The Freund scoring system uses a 7-point rating scale ranging from 0 to 7, indicating a potentially very poor to excellent cognitive function, respectively. The scoring system is divided into three categories, namely, the ability to correctly reproduce all numbers, to position them accurately in the circle, and to appropriately replicate the hands at the indicated time (Table 1). For every item, one point can be awarded [11,19]. According to our predefined cutoff score, patients were considered to be potentially cognitively impaired if they had a score of 4 or less [11].

Statistical analyses
All statistical analyses were performed by using SPSS software (version 21; IBM SPSS Statistics, Chicago, IL). Descriptive statistics were conducted to present patient and tumor characteristics, and CGA and cognitive test results. Scatter graphs were plotted to evaluate if a linear relationship was present between education age and MMSE and CDT test scores. Based on the linearity of this association, Pearson or Spearman's rank correlation coefficients were calculated to examine the association between age, education age, and MMSE and CDT test scores. Education age can be defined as the number of years that patients went to school, starting from primary education. In advance sample size calculations were based on the hypothesis of equality with 0.70 of the area under the receiver operating characteristics (ROC) curve (ClinicalTrials.gov identifier: NCT01749995). In our scenario, a sample with an unequal allocation ratio of four, consisting of a sample of at least 32 from the positive group and at least 128 from the negative group, would achieve at least 80% power to detect a difference of 0.15 between the area under the ROC curve under the null hypothesis of 0.70 and an AUC under the alternative hypothesis of 0.85 using a two-sided z-test at a significance level of 5%. ROC curves were plotted to evaluate the diagnostic performance, in terms of AUC, of the Freund CDT in determining patients who are potentially cognitively impaired compared with the Folstein MMSE as gold standard.
The cutoff for determining impairment was defined as having a MMSE score of 23 or less [13]. Se and Sp with 95% confidence intervals (95%CI) were calculated at our predefined cutoff score of ≤4. Positive and negative predictive values were also determined (PPV and NPV, respectively).

Comprehensive geriatric assessment and cognitive measures
Three patients (1.5%) screened negative on the G8questionnaire (cutoff ≤14) and were evaluated with a full CGA based on a referral from their treating physician. On  Table 3). When subdividing patients into groups by age and education age according to Crum et al. (1993), the cutoff remained optimal (data not shown) [30].

Discussion
Assessing cognitive function provides health care workers valuable information on the mental reserve of the patient as patients presenting with memory impairment can have difficulties understanding treatment instructions and may not be alert for the signs and symptoms of treatment related  toxicities that need further evaluation [31]. The Folstein MMSE is a commonly used instrument to screen for dementia and is validated for use in several patient populations. Nevertheless, the MMSE is time-consuming and confronting in the many cognitively fit patients that undergo a CGA as part of their cancer care. Previous work from our group suggested that the Freund CDT with a cutoff score of ≤4 could replace the MMSE within the CGA, resulting in gain in time for health providers and increased comfort for patients [11]. The current study was able to prospectively validate the retrospectively identified cutoff score and could therefore be practice changing. A good screening tool needs a high Se and high NPV as it reduces the number of false-negative cases. Our results show that the Freund CDT, with a cutoff score of ≤4, has indeed the properties of an excellent screening instrument as we have found a Se of 94.3% and NPV of 97.7%. Further, the Freund CDT provided a high Sp of 87.4%. In this trial, our primary endpoint was to validate the CDT on the basis of the diagnostic accuracy of the test. We stated that a sample with an unequal allocation ratio of four, consisting of a sample of at least 32 from the positive group and at least 128 from the negative group, would achieve at least 80% power to detect a difference of 0.15 between the AUC under the null hypothesis of 0.70 and an AUC under the alternative hypothesis of 0.85 using a twosided z-test at a significance level of 5%. In our sample, results show an AUC (AUC ± SE) under the ROC curve of 0.95 ± 0.17. Hereby, we can accept the alternative hypothesis as an AUC under the ROC curve, of at least 0.85 was achieved. As this cutoff score was also determined in our previous retrospective study (in oncogeriatric and general geriatric patients) and in the original paper by Freund et al., we can assume the robustness of this cutoff score [11,19]. Further, we can state that the cutoff score of ≤4 is the most optimal cutoff score for use in an oncogeriatric population.
In our sample, 27.0% of patients presented with a potential cognitive deficit that needed further evaluation on the basis of the MMSE. This is in line with previous research reporting cognitive deterioration in up to 50% of patients [9]. Further, it has been noted that the Folstein MMSE can be influenced by education age, whereas the CDT is less dependent of education age when given a pre-drawn circle [18,30]. Spearman's correlation coefficients showed a significant statistical association between MMSE test scores and education age. This was not the case for the Freund CDT.
Initially, it was our objective to validate the Freund CDT as a pre-screener within a CGA. Because results show such an excellent AUC of 0.95 with Se of 94.3% and Sp of 87.4%, we could assume that an assessment with the MMSE may be redundant and that results on both screening tools will be nearly equal. However, McNemar test revealed a significant difference between both test outcomes disputing the latter statement (p = 0.001; data not shown). This highly significant result reflects a minor discordance in 21 out of 196 patients, of which 18 are considered fit by MMSE were classified vulnerable by CDT and 3 out of 196 are considered vulnerable by MMSE were classified as fit by CDT. Nevertheless, selecting the Freund CDT above the Folstein MMSE has some advantages. First, the Freund CDT defined more patients as vulnerable leading to a more sensitive test. Second, within a CGA, we try to select those domains that can influence and increase the risk on morbidity and mortality. As it is not our intention to diagnose patients but merely to detect potential vulnerabilities, we need a screening tool that gives us valuable information in less time. The Freund CDT can be administered in approximately 5 minutes and has been previously reported as a good screening tool in other populations that can be carried out in very little time [15]. Third, the Freund scoring system is user-friendly and has been reported with a high interrater reliability [11,19]. Fourth, in our and other patient populations, the MMSE can be experienced as tedious and annoying, whereas the CDT has been described previously as a non-threatening cognitive assessment [32]. Last, it has been noted that the MMSE can be influenced by education age, whereas the CDT-when given a pre-drawn circle-is not influenced by education age [18,30]. Our results support this statement.
The results of this trial need to be interpreted with caution because of some limitations. We considered the Folstein MMSE as the gold standard against the Freund CDT. Although the MMSE is a commonly used validated measure, it is not a diagnostic test. Cognitive malfunction detected by the CDT may slightly differ from that detected by our gold standard. Therefore, it is important to remember that both MMSE and CDT are screening tools and that they should always be followed by an intensive diagnostic neuropsychological assessment when a potential cognitive impairment is detected [33]. Further, the MMSE cutoff of ≤23 may not be sufficient for detecting mild cognitive impairment nor may it be sufficient for detection dysfunctions in patients with less than 9 years of education [30,34]. Although our population has a mean education age of 10.3 years, 6.6% of patients received less than lower secondary education (Table 2). However, in our study, we did not intend to diagnose patients but to select those who may present with a potential vulnerability that needs closer evaluation. Next, this study was conducted in oncogeriatric patients receiving a routine oncogeriatric assessment. Most patients consenting for this trial had been assessed with a CGA because of a positive test score on the G8-questionnaire. In our clinic, patients deemed fit -on the basis of their G8 screening score-are only evaluated with a CGA when required by the physician. Therefore, this trial includes only a minority of fit patients. The cutoff score achieved may thus not be representative for patients who screened negative on the G8 or patients who are evaluated with other screeners such as VES-13 [20]. However, the G8-questionnaire contains seven items from the Mini Nutritional Assessment and age. One of the items included in the G8-questionnaire concerns cognition and depression. This item has previously shown to correlate with MMSE test scores [35,36]. Last, we did not consider the chronobiology [37]. However, in our sample, as patients were seen throughout the day, we suggest a minimal bias by biological rhythms.
Overall, we can conclude that in this prospective trial, we were able to validate the Freund CDT with a cutoff score of ≤4 as a screening tool to detect cognitive dysfunction in elderly cancer patients undergoing a CGA.
Our results indicate that it could potentially replace the MMSE as a stand-alone screening instrument, leading to a more time-efficient CGA.