The value of clock drawing in identifying executive cognitive dysfunction in people with a normal Mini-Mental State Examination score
=====================================================================================================================================

* Angela Juby
* Shirley Tench
* Victoria Baker

## Abstract

**Background:** Executive cognitive dysfunction can precede the memory disturbances of dementia. People with executive cognitive dysfunction can have a normal Mini-Mental State Examination (MMSE) score but still have severe functional limitations. We evaluated the usefulness of clock drawing in identifying people with executive dysfunction who have a normal MMSE score.

**Methods:** We reviewed the charts of consecutive patients referred between July 1999 and June 2000 to a multidisciplinary geriatric assessment clinic because of concerns about functional inabilities. The patients had all undergone the Executive Interview for the diagnosis of executive cognitive dysfunction as well as an MMSE and clock-drawing test (scored by 2 methods: one described by Watson and colleagues [the Watson method] and one described by Sunderland and colleagues [the Sunderland method]).

**Results:** We reviewed the charts of 68 patients (40 women, 28 men); their mean age was 79 years (range 55–94). Thirty-six patients had an MMSE score of less than 24, and 32 had a “normal” MMSE score (24–30). Among those with a normal MMSE score, 22 had an abnormal Executive Interview score. Using the Executive Interview as the gold standard, the sensitivity and specificity of the Watson method of scoring clock drawings to predict an abnormal Executive Interview score were 59% and 70% respectively; the corresponding values were 18% and 100% for the Sunderland method.

**Interpretation:** The presence of an abnormal MMSE score alerts clinicians to the possibility of cognitive impairment. For patients referred for geriatric assessment who have a normal MMSE score, a clock-drawing test, scored by either the Watson or the Sunderland method, is a moderately sensitive and specific adjunct for detecting executive cognitive dysfunction.

The change in population demographics1 makes it increasingly important for physicians to be able to assess cognitive decline adequately. Alzheimer's disease is the most common form of dementia2 and usually presents with memory loss. However, disturbances in executive cognitive functioning often precede the memory decline.3 Such disturbances result in difficulties with instrumental activities of daily living4 (e.g., bathing, dressing, cooking, shopping, driving and taking medications). They produce a dissociation between volition and action; for example, patients do not lose their ability to dress but, rather, are unable to initiate these tasks or choose weather-appropriate clothes. The challenge is to identify executive cognitive dysfunction in such patients.

In the past it was felt that there was no urgency to diagnose Alzheimer's disease early, because only supportive therapy could be offered to patients and their caregivers. However, efficacious symptomatic treatment is now available: 3 cholinesterase inhibitors recently licensed in Canada have been shown to produce benefits in some patients.5,6,7 There is suggestion of increased benefits in patients if treatment is started as early as possible in the course of their disease. Early diagnosis of cognitive deficits also facilitates future planning (e.g., wills, enduring power of attorney and advanced directives) at a time when the patient may still be competent to make these important decisions.

Routine measures of cognition, such as the Folstein Mini-Mental State Examination (MMSE),8 often fail to identify executive dysfunction even if it is quite severe.9 There are detailed neuropsychological tests and more extensive bedside tests available to evaluate executive function specifically, but most of them are impractical for busy physicians. We undertook this study to determine the value of a clock-drawing test as an adjunct to the MMSE in identifying potential executive dysfunction in a clinical setting.

## Methods

We reviewed the charts of consecutive patients who attended the outpatient geriatric assessment clinic at the University of Alberta Hospital, Edmonton, between July 1999 and June 2000. All had been referred by family practitioners, homecare workers and family members because of symptoms of functional decline. Patients suspected of having executive cognitive dysfunction by the assessment team underwent the Executive Interview (EXIT)10 as well as a normal comprehensive geriatric assessment by a geriatric clinical nurse specialist, an occupational therapist and a geriatrician.

The EXIT is a 25-item interview that takes 15–20 minutes. It is designed to be administered at the bedside by non-neuropsychiatrically trained personnel. The EXIT score strongly correlates with the findings of an extensive battery of diagnostic neuropsychiatric tests (*r* range 0.64–0.83), and interrater reliability between 2 physicians is high (*r* = 0.90).10 The interview includes items that detect frontal release signs (e.g., grasp reflex), motor or cognitive perseveration (e.g., echopraxia), verbal intrusions, disinhibition, loss of spontaneity (e.g., word fluency), imitation behaviour, environmental dependence and utilization behaviour. Each item is scored from 0 (intact) to 2 (specific incorrect response or failure to perform a task). The total score is from 0 to 50, a higher score indicating executive cognitive dysfunction. We used a cutoff score of 15 or greater to indicate executive dysfunction (as suggested by the original authors10). In all cases the EXIT was administered by an experienced occupational therapist.

The MMSE was conducted and the score determined according to the guidelines for the standardized MMSE.11

For the clock-drawing test, each patient was given a predrawn circle (to minimize the effect of education) and asked to “place the numbers on it to make it look like a clock.” The placement of hands to read “10 past 11” was requested after the first task was completed to the best of the patient's ability. This time has been reported to be the most sensitive for detecting neurocognitive dysfunction.12 All clock drawings were scored independently by the principal investigator (A.J.) without knowledge of the EXIT or MMSE scores, or the medical diagnoses. Clocks were scored by means of 2 methods.13,14 The scoring method described by Watson and colleagues13 (referred to in this article as the Watson method) is objective and based on dividing the clock into quadrants (Fig. 1). The score is determined from the number of digits in each quadrant (0–3 = normal score and 4–7 = abnormal score). This method takes into account the digit positioning only and not the positioning of the hands of the clock. The method described by Sunderland and colleagues14 (referred to in this article as the Sunderland method) takes into account hand positioning, and the score is determined using a 10-point scale (10 = perfect and 0 = very poor) (Fig. 2); scores of 6 or more are considered normal.

![Figure1](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/167/8/859/F1.medium.gif)

[Figure1](http://www.cmaj.ca/content/167/8/859/F1)

**Fig. 2: Method described by Sunderland and colleagues**14 for scoring clock drawings. As described in Fig. 1, patients are given a predrawn circle and asked to draw a clock and the time as “10 past 11.” Top: Scoring criteria. Bottom: Examples of clock drawings and scores derived using this method. Scores of 6 or more are considered normal.


![Figure2](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/167/8/859/F2.medium.gif)

[Figure2](http://www.cmaj.ca/content/167/8/859/F2)

**Fig. 1: Method described by Watson and colleagues**13 for scoring clock drawings. Patients are given a predrawn circle and asked to draw numbers on it to make it look like a clock. They are then asked to draw the hands of the clock to read “10 past 11.” Top: Scoring criteria. Bottom: Examples of patients' clock drawings and scores derived using this method.

The scores for the EXIT, clock-drawing test and MMSE were obtained from each patient's chart. The charts were divided into 2 groups according to whether the MMSE score was normal (24–30) or abnormal (< 24). We then evaluated the specificity and sensitivity of the 2 methods of scoring the clock drawings for the detection of executive cognitive dysfunction in patients with a normal MMSE.

Ethics approval for this study was obtained from the Research Ethics Board at the University of Alberta Hospital.

## Results

We reviewed the charts of 68 patients (40 women, 28 men) referred for geriatric assessment; their mean age was 79 years (range 55–94). The distribution of the MMSE, EXIT and clock-drawing scores is shown in Fig. 3.

![Figure3](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/167/8/859/F3.medium.gif)

[Figure3](http://www.cmaj.ca/content/167/8/859/F3)

**Fig. 3: Distribution of patients referred to a geriatric assessment clinic according to their Mini-Mental State Examination (MMSE) scores, Executive Interview (EXIT) scores and clock-drawing test scores.** 

Of the 36 patients with an abnormal MMSE score (< 24), 35 (97%) had an abnormal EXIT score, 30 (83%) had an abnormal clock-drawing score by the Watson method, and 13 (36%) had an abnormal clock-drawing score by the Sunderland method. Patients in this group were not evaluated further, as the presence of an abnormal MMSE score already raises the suspicion of cognitive dysfunction.

Of the 32 patients with a normal MMSE score (24–30), 22 (69%) had an abnormal EXIT score. Sixteen (50%) of the 32 patients had an abnormal clock-drawing score according to the Watson method, and 4 (12%) had an abnormal clock-drawing score according to the Sunderland method. Of the 22 patients with a normal MMSE score and an abnormal EXIT score, the Watson clock-drawing score was abnormal for 13 (59%), and the Sunderland score was abnormal for 4 (18%). The corresponding figures among patients with a normal MMSE score and a normal EXIT score were 3 (30%) and 0 (0%).

According to these findings, the Watson method had a sensitivity of 59% and a specificity of 70% for detecting executive cognitive dysfunction in patients with a normal MMSE score. The corresponding values for the Sunderland method were 18% and 100%. The sensitivity and specificity for combined clock-drawing scores was also evaluated: among patients with a normal MMSE score they were 59% and 70% respectively. In none of the cases was the Watson score normal and the Sunderland score abnormal.

On completion of the geriatric assessment, the clinical diagnoses were Alzheimer's disease (in 19 cases), vascular dementia (in 18), unspecified “dementia with frontotemporal features” (in 15), depression and anxiety (in 7), mixed dementia (in 2), Parkinsonism with dementia (in 2), alcohol abuse (in 3), neurosyphillis (in 1) and no cognitive impairment (in 1).

Subgroup analysis of the 15 participants with nonspecific frontotemporal features revealed a mean MMSE score of 24 (range 13–29) and a mean EXIT score of 24 (range 21–26). Among the 10 patients in this group who had a normal MMSE score, the sensitivities of the Watson and Sunderland methods (60% and 20% respectively) were similar to those obtained for the whole patient group.

## Interpretation

Our study differs in several ways from those previously published. First, we compared the clock-drawing scores with the EXIT scores and not the MMSE scores, and the 2 methods used to score the clock drawings have not been previously evaluated in comparison with the EXIT. Royall and colleagues15 found significant correlation between 6 different scoring methods of clock-drawing tests and EXIT scores (*r* = 0.56–0.78). However, they did not include the Watson method. In addition, they looked at the direct correlation between the MMSE, EXIT and clock-drawing scores and not at groups with normal versus abnormal MMSE scores, as we did. Also, their study population differed from ours in that it comprised preselected patients with Alzheimer's disease and healthy elderly control subjects, which allowed the authors to evaluate the ability of the clock-drawing test to identify “hidden” cognitive dysfunction. Second, in our patient group the clock-drawing test was not used to diagnose dementia but, rather, to identify specific executive cognitive dysfunction that may or may not be significant enough to fulfill the criteria for dementia outlined in the *Diagnostic and Statistical Manual of Mental Disorders*, fourth edition.16 Third, the patients evaluated in our study had normal (“non-dementia”) MMSE scores. Manos,17 in a subanalysis of 14 patients from a previous publication,18 found that his 10-point clock-drawing test had a sensitivity of 71% among patients with a normal MMSE score for the specific diagnosis of Alzheimer's disease and not for executive cognitive dysfunction from any cause. We are unaware of any study that has examined the relation between clock drawings and executive cognitive dysfunction in a group with normal MMSE scores.

The need for all physicians to be comfortable with cognitive assessments is increasing, and primary care physicians are the most likely to be able to detect early cognitive decline for consideration of treatment and future life planning (e.g., wills, advance directives). Executive cognitive dysfunction can have a significant impact on decision-making capacity, and this condition is particularly challenging to diagnose in someone with a normal MMSE score.19 Ideally, every physician would have rapid access to a referral centre or have a trained occupational therapist or neuropsychologist in his or her office. However, these scenarios are not the reality of modern medicine. A detailed cognitive assessment in a busy office is often not possible, but the administration of an MMSE and a clock-drawing test to patients with suspected functional decline is possible.

The clock-drawing test has been shown to be an acceptable, non-threatening assessment20 that is reliable and effective for diagnosis and longitudinal assessment of cognition12,13,14,20,21,22,23,24,25 and correlates with the MMSE score.15,26 Like the MMSE, the clock-drawing test can be education dependent,27 particularly when a predrawn circle is not provided. Unlike many other tools for measuring cognitive function, the clock-drawing test is independent of ethnic background.20,28

When the MMSE score is abnormal, the suspicion of cognitive impairment is already raised. Under these circumstances the clock-drawing test score is often abnormal (as was the case among 83% of the patients in our study [Watson method]), and it reinforces the suspicion of cognitive impairment. The more difficult cases involve patients with a history of unusual or abnormal function or whose physician has concerns about their higher cognitive functioning (e.g., medication compliance or competency assessment) and whose MMSE score is normal. In such cases the clock-drawing test can be particularly useful to examine cognitive domains not evaluated by the MMSE.

The best method for scoring clock drawings is still being debated, as highlighted by the abundance of scoring methods available. In one study29 the scoring methods described by Mendez and associates30 and by Shulman31 were shown to be superior to the Watson and Sunderland methods in their ability to diagnose Alzheimer's disease. (The method by Mendez and associates requires the use of a 20-point scale; the Shulman method requires the use of a 6-point scale to score subjects' drawn and predrawn circles.) The investigators commented that “many clocks are drawn but few formally scored.” This emphasizes the problem of using difficult scoring systems in real practice. Although the Watson method may not be as comprehensive as others, it is easy to use and to score drawings objectively, even by those with limited scoring experience, and it does not require the tester to carry standardized clock pictures. Although the Watson method does not account for the positioning of the clock hands (felt by some to be an important frontal executive function), we found it to be a more sensitive indicator of executive dysfunction than the Sunderland method, which did account for hand positioning. We found the Sunderland method to be more specific, but less sensitive, than the Watson method in identifying executive dysfunction in patients with a normal MMSE score. In a practical sense, both of these scoring methods could be used: if the numbers are abnormally spaced, or if the hands are incorrectly placed. Shulman31 reinforced the utility of the clock-drawing test both initially and in long-term follow-up of patients with cognitive deficits.

The process of the MMSE and clock-drawing test deserves comment. We are unaware of data that look at the difference of doing the clock-drawing test before or after the MMSE. However, good neuropsychiatric practice would suggest that the clock-drawing test not be interposed in the middle of the MMSE, because this may affect the validity of the MMSE score and does not follow the guidelines established for the standardized MMSE.11 The circle for the clock-drawing test should be provided to the patient on an otherwise blank sheet of paper to lessen potential distractors.

Our study has limitations. The patients represent a distinct group who were referred because of concerns about their continued independent function, and so there was a mixture of cognitive disorders. Most of the patients were referred by their family physicians, and although they may not be representative of this age group in all community practices, they undoubtedly represent some of the patient population in family practitioner's offices. The retrospective nature of the study may also be considered a limitation; however, it allowed for the staff to be blinded to the purpose of the study when the MMSE, clock-drawing test and EXIT were administered. The person who scored the clock drawings was blinded to the other test results; hence potential bias in scoring was minimized. The patients did not have formal neuropsychological testing to evaluate frontal lobe dysfunction; however, this was not felt to be necessary because the EXIT has been previously validated against a battery of neuropsychological tests.10,27,32 Some may consider the participation of an occupational therapist a limitation to the generalizability of our findings to other practice settings. However, the clock drawings were scored by a clinician; the occupational therapist performed only the EXIT. A final limitation may be the methods used to score the clock drawings; however, they were chosen for their ease of administration (predrawn circle provided to patients) and objective scoring to try to mimic a busy community practice as much as possible.

Studies have demonstrated that dementia is underdiagnosed in the primary care setting, with one showing that less than half of patients with Alzheimer's disease are being identified.33 Even when primary care physicians know the value of dementia screening, they rarely do the testing.34,35 Reported barriers to testing include increasingly abbreviated office visits, lack of routine use of cognitive screening tools, difficulty interpreting cognitive test results, lack of specificity and sensitivity of screening tools, and the risk of offending patients.36 Clock-drawing tests have been found to be effective in screening elderly patients in hospital for cognitive impairment.37 A high correlation has been found between the perceptions of family members or caregivers who inform health care professionals of suspected cognitive problems of a patient and the patient's actual cognitive impairment.38 Thus, it might be best to target this group of patients instead of screening on the basis of age alone, which has a low positive predictive value for dementia.39

Our study highlights the fact that a normal MMSE score does not exclude significant cognitive dysfunction. The addition of a clock-drawing test can enhance the evaluation by assessing domains of cognition not examined by the MMSE alone. The clock drawing should be scored objectively whenever possible. An abnormal clock drawing may suggest executive cognitive dysfunction and should prompt further testing or referral. The clock-drawing test is not designed to be the only form of cognitive evaluation and instead should be an adjunct to the MMSE to identify, or explain, functional issues encountered by physicians.

## Footnotes

*   *This article has been peer reviewed.* 
    
    *Contributors:* Dr. Juby was responsible for the study concept and design, initial and final literature searches, scoring of clock drawings, statistical analysis and manuscript preparation. Ms. Tench was responsible for administering the Executive Interviews, and Ms. Baker was responsible for patient chart location, administering the MMSE and clock-drawing tests in some cases and performing an interim literature search. All authors reviewed and approved the final version of the manuscript.
    
    *Acknowledgements:* We thank the staff and volunteers at the Seniors Clinic at the University of Alberta Hospital for their support with this study and Ms. Suzanne Gamble for her excellent librarian support.
    
    *Competing interests:* None declared.

## References

1.  1. Desjardins B, Dumas J. *Current demographic analysis: population aging and the elderly*. Ottawa: Statistics Canada; 1993. Cat no 91-533E.
    
    

2.  2. Canadian study of health and aging: study methods and prevalence of dementia. CMAJ 1994;150(6):899-913.
    
    [Abstract](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNTAvNi84OTkiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWovMTY3LzgvODU5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3. Binetti G, Magni E, Padovani A, Cappa SF, Bianchetti A, Trabucchi M. Executive dysfunction in early Alzheimer's disease. J Neurol Neurosurg Psychiatry 1996;60:91-3.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiam5ucCI7czo1OiJyZXNpZCI7czo3OiI2MC8xLzkxIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqLzE2Ny84Lzg1OS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  4. Royall DR, Cabello M, Polk MJ. Executive dyscontrol: an important factor affecting the level of care received by elderly retirees. J Am Geriatr Soc 1998; 46:1519-24.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9848812&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000077358700005&link_type=ISI) 

5.  5. Rogers SL, Friedhoff LT and the Donepezil Study Group. The efficacy and safety of donepezil in patients with Alzheimer's disease: results of a US multicenter, randomized, double blind, placebo-controlled trial. Dementia 1996;7:293-303.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8915035&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1996VP86800001&link_type=ISI) 

6.  6. Rosler M, Anand R, Cicin-Sain A, Gauthier S, Agid Y, Dal-Bianco P, et al. Efficacy and safety of rivastigmine in patients with Alzheimer's disease: international randomised controlled trial. BMJ 1999;318:633-8.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTgvNzE4NC82MzMiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWovMTY3LzgvODU5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.  7. Raskind MA, Peskind ER, Wessel T, Yuan W. Galantamine in Alzheimer's Disease — a 6-month, randomized, placebo-controlled trial with a 6-month extension. The Galantamine USA-1 Study Group. Neurology 2000;54:2261-8.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibmV1cm9sb2d5IjtzOjU6InJlc2lkIjtzOjEwOiI1NC8xMi8yMjYxIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqLzE2Ny84Lzg1OS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  8. Folstein MF, Folstein SE, McHugh PR. Mini-Mental State: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189-98.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0022-3956(75)90026-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1202204&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1975AY27900005&link_type=ISI) 

9.  9. Royall DR, Mahurin RK. EXIT25, QED, and DSM-IV: very early Alzheimer's disease. J Neuropsychiatry Clin Neurosci 1994;6:62-4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1176/jnp.6.1.62&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8148642&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1994MV31800015&link_type=ISI) 

10. 10. Royall DR, Mahurin RK, Gray KF. Bedside assessment of executive cognitive impairment: the Executive Interview. J Am Geriatr Soc 1992;40:1221-6.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1447438&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1992KA80900006&link_type=ISI) 

11. 11. Molloy DW, Clarnette R. *Standardized Mini-Mental State Examination: a user's guide*. Troy (ON): New Grange Press; 1999.
    
    

12. 12. Freedman M, Leach L, Kaplan E, Winocur G, Shulman KI, Delis DC. *Clock drawing: a neuropsychological analysis*. New York: Oxford University Press; 1994.
    
    

13. 13. Watson YI, Arfken CL, Birge SJ. Clock completion: an objective screening test for dementia. J Am Geriatr Soc 1993;41:1235-40.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8227899&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1993MF52800011&link_type=ISI) 

14. 14. Sunderland T, Hill JL, Mellow AM, Lawlor BA, Gundersheimer J, Newhouse PA, et al. Clock drawing in Alzheimer's disease: a novel measure of dementia severity. J Am Geriatr Soc 1989;37:725-9.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2754157&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1989AJ47000007&link_type=ISI) 

15. 15. Royall DR, Mulroy AR, Chiodo LK, Polk MJ. Clock drawing is sensitive to executive control: a comparison of six methods. J Gerontol 1999;54B:328-33.
    
    

16. 16. American Psychiatric Association. *Diagnostic and statistical manual of mental disorders*. 4th ed. Washington: The Association; 1994. p. 142-52.
    
    

17. 17. Manos PJ. Ten-point clock test sensitivity for Alzheimer's disease in patients with MMSE scores greater than 23. Int J Geriatr Psychiatry 1999;14:454-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/(SICI)1099-1166(199906)14:6<454::AID-GPS951>3.0.CO;2-N&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10398355&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000081209700007&link_type=ISI) 

18. 18. Manos PJ. The utility of the ten-point clock test as a screen for cognitive impairment in general hospital patients. Gen Hosp Psychiatry 1997;19:439-44.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0163-8343(97)00072-8&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9438188&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000071239900008&link_type=ISI) 

19. 19. Schindler BA, Ramchandani D, Matthews MK, Podell K. Competency and the frontal lobe. The impact of executive dysfunction on decisional capacity. Psychosomatics 1995;36:400-4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0033-3182(95)71649-8&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7652142&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 

20. 20. Shulman KI, Pushkar Gold D, Cohen CA, Zucchero CA. Clock drawing and dementia in the community: a longitudinal study. Int J Geritr Psychiatry 1993;8:487-96.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/gps.930080606&link_type=DOI) 

21. 21. Shulman KI, Shedletsky R, Silver I. The challenge of time: clock drawing and cognitive function in the elderly. Int J geriatr Psychiatr 1986;1:135-40.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/gps.930010209&link_type=DOI) 

22. 22. Wolf-Klein GP, Silverstone FA, Levy AP, Brod MS. Screening for Alzheimer's disease by clock drawing. J Am Geriatr Soc 1989;37:730-4.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2754158&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1989AJ47000008&link_type=ISI) 

23. 23. Tuokko H, Hadjistavropolous T, Miller JA, Beattie BL. The clock test: a sensitive measure to differentiate normal elderly from those with Alzheimer's disease. J Am Geriatr Soc 1992;40:579-84.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1587974&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1992HX64900006&link_type=ISI) 

24. 24. Huntzinger JA, Rosse RB, Schwartz BL, Ross LA, Deutsch SI. Clock drawing in the screening assessment of cognitive impairment in an ambulatory care setting: a preliminary report. Gen Hosp Psychiatry 1992;14:142-4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0163-8343(92)90040-H&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1592251&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1992HN41600009&link_type=ISI) 

25. 25. Libon DJ, Malamut BL, Swenson R, Prouty Sands L, Cloud BS. Further analyses of clock drawing among demented and nondemented older subjects. Arch Clin Neuropsychol 1996;11:193-205.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0887-6177(95)00026-7&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14588923&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1996UB66200003&link_type=ISI) 

26. 26. Juby A. Correlation between the Folstein Mini-Mental State Examination and three methods of clock drawing scoring. J Geriatr Psychiatry Neurol 1999;12: 87-91.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BqZ3AiO3M6NToicmVzaWQiO3M6NzoiMTIvMi84NyI7czo0OiJhdG9tIjtzOjIwOiIvY21hai8xNjcvOC84NTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

27. 27. Ainslie NK, Murden RA. Effect of education on the clock drawing dementia screen in non-demented elderly persons. J Am Geriatr Soc 1993;41:249-52.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8440847&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1993KP99800010&link_type=ISI) 

28. 28. Borson S, Brush M, Gil E, Scanlan J, Vitaliano P, Chen J, et al. The Clock Drawing Test: utility for dementia detection in multiethnic elders. *J Gerontol A Biol Sci Med Sci* 1999;54(11):M534-40.
    
    

29. 29. Storey JE, Rowland JT, Basic D, Conforti D. A comparison of five clock drawing methods using ROC (receiver operating charactersitic) curve analysis. Int J Geriatr Psychiatry 2001;16:394-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/gps.352&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11333427&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000168442000008&link_type=ISI) 

30. 30. Mendez MF, Ala T, Underwood KL. Development of scoring criteria for the clock drawing task in Alzheimer's disease. JAm Geriatr Soc 1992;40:1095-9.
    
    

31. 31. Shulman KI. Clock-drawing: Is it the ideal cognitive screening test? Int J Geriatr Psychiatry 2000;15:548-61.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/1099-1166(200006)15:6<548::AID-GPS242>3.0.CO;2-U&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10861923&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000087808800014&link_type=ISI) 

32. 32. Cahn-Weiner DA, Sullivan EV, Shear PK, Fama R, Lim KO, Yesavage JA, et al. Brain structural and cognitive correlates of clock drawing performance in Alzheimer's disease. J Int Neuropsychol Soc 1999;5:502-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1017/S1355617799566034&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10561930&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000083339700003&link_type=ISI) 

33. 33. Ross GW, Abbott RD, Petrovitch H, Masaki KH, Murdaugh C, Trockman C, et al. Frequency and characteristics of silent dementia among elderly Japanese-American men. JAMA 1997;277:800-5.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1997.03540340034029&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9052709&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997WM08200032&link_type=ISI) 

34. 34. Somerfield MR, Weisman CS, Ury W, Chase GA, Folstein MF. Physician practises in the diagnosis of dementing disoreders. J Am Geriatr Soc 1991;39: 172-5.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1991948&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1991EX54800010&link_type=ISI) 

35. 35. Glasser M. Alzheimers disease and dementing disorders: practices and experiences of rural physicians. Am J Alzheimers Care Rel Disord Res 1993;8(4):28-35.
    
    

36. 36. Bush C, Kozak J, Elmslie T. Screening for cognitive impairment in the elderly. Can Fam Physician 1997;43:1763-8.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9356757&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997YB57600019&link_type=ISI) 

37. 37. Death J, Douglas A, Kenny RA. Comparison of clock drawing with the Mini Mental State Examination as a screening test in elderly acute hospital admissions. Postgrad Med J 1993;69:696-700.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6InBvc3RncmFkbWVkaiI7czo1OiJyZXNpZCI7czoxMDoiNjkvODE1LzY5NiI7czo0OiJhdG9tIjtzOjIwOiIvY21hai8xNjcvOC84NTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

38. 38. Jorm AF, Jacomb PA. The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): socio-demographic correlates, reliability, validity and some norms. Psychol Med 1989;19:1015-22.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1017/S0033291700005742&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2594878&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1989CD23500024&link_type=ISI) 

39. 39. Brodaty H, Clarke J, Ganguli M, Grek A, Jorm AF, Khachaturian Z, et al. Screening for cognitive impairment in general practice: toward a consensus. Alzheimer Dis Assoc Disord 1998;12:1-13.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/00002093-199803000-00001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9539404&link_type=MED&atom=%2Fcmaj%2F167%2F8%2F859.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000073775500001&link_type=ISI)