Errors in ADAS-Cog Administration and Scoring May Undermine Clinical Trials Results

Background: The Alzheimer's Disease Assessment Scale - cognitive subscale (ADAS-cog) is the most widely used cognitive outcome measure in AD trials. Although errors in administration and scoring have been suggested as factors masking accurate estimates and potential effects of treatments, there have been few formal examinations of errors with the ADAS-cog. Methods: We provided ADAS-cog administration training using standard methods to raters who were designated as experienced, potential raters by sponsors or contract research organizations for two clinical trials. Training included 1 hour sessions on test administration, scoring, question periods, and required that raters individually view and score a model ADAS-cog administration. Raters scores were compared to the criterion scores established for the model administration. Results: A total of 108 errors were made by 80.6% of the 72 raters; 37.5% made 1 error, 25.0% made 2 errors and 18.0% made 3 or more. Errors were made in all ADAS-cog subsections. The most common were in word finding difficulty (67% of the raters), word recognition (22%), and orientation (22%). For the raters who made 1, 2, or ≫ 3 errors the ADAS-cog score was 17.5 (95% CI, 17.3 - 17.8), 17.8 (17.0 - 18.5), and 18.8 (17.6 - 20.0), respectively, and compared to the criterion score, 18.3. ADAS-cog means differed significantly and the variances were more than twice as large between those who made errors on word finding and those who did not, 17.6 (SD=1.4) vs. 18.8 (SD=0.9), respectively (χ2 = 37.2, P<001). Conclusions: Most experienced raters made at least one error that may affect ADAS-cog scores and clinical trials outcomes. These errors may undermine detection of medication effects by contributing both to a biased point estimate and increased variance of the outcome.