Study Reveals Limitations of Cognitive Tests in Diagnosing Concussions Among College Athletes

When college athletes are evaluated for a possible concussion, the diagnosis is based on an athletic trainer or team physician's assessment of three things: the player's symptoms, physical balance and cognitive skills. 

Research published today suggests that almost half of athletes who are ultimately diagnosed with a concussion will test normally on the recommended cognitive-skills test. 

If you don't do well on the cognitive exam, it suggests you have a concussion. But many people who are concussed do fine on the exam."

Dr. Kimberly Harmon, study's lead author

She is a professor of family medicine and section head of sports medicine at the University of Washington School of Medicine.

The study findings appear in JAMA Network Open.

Harmon said her sideline experiences as a team physician for the UW Huskies caused her to wonder how to accurately interpret the cognitive-screening portion of the Sport Concussion Assessment Tool (SCAT). Introduced in 2004 by the Concussion in Sport Group, the SCAT (now in its fifth iteration, SCAT5) was intended to standardize the gathering of information from athletes with a potential head injury. 

The SCAT first prompts an athlete about whether they are experiencing any of 22 symptoms such as headache, nausea or blurred vision, and symptom severity. Then the tool tests the athlete's cognition in several ways. 

First come questions of orientation. (What day is it? What month is it?) Then a test of immediate memory, in which a list of 10 words is read aloud to the athlete, who is asked to restate the list. This sequence is repeated three times. Then the athlete's concentration is tested by having to repeat short sequences of numbers in reverse order. Then comes a prescribed evaluation of the athlete's balance, after which the athlete is again asked to recall the 10 words from the first list. 

In Harmon's experience as a team physician, she saw that "some people were concussed and they did well on the recall tests. Some people weren't concussed and they didn't do well. So I thought we should study it," she said.

The study involved 92 NCAA Division I athletes who sustained a concussion between July 13, 2020, and Dec. 31, 2022, and who had a concussion evaluation within 48 hours. The investigators also recruited 92 of the concussed players' teammates as matched control subjects, each of whom was given the SCAT5 screening within two weeks of the incident concussion. 

All athletes in the study had previously completed NCAA-required baseline concussion screenings. The investigators found no significant differences in baseline scores between the athletes with and without concussion. 

Harmon and colleagues analyzed the study participants' SCAT5 responses and found that the word-recall tests had little predictive value for concussion. In fact, almost half (45%) of the concussed athletes performed at or above their baseline cognitive-test results, the researchers reported.

Instead, the study showed that the most accurate predictor of concussion were the athletes' responses to questions about their symptoms. 

"If you get hit in the head and go to the sideline and say, 'I have a headache. I'm dizzy. I don't feel right,' I can say with pretty good assurance that you have a concussion," Harmon said. "I don't need to do any testing. The problem is that some athletes don't want to come out. They don't report their symptoms or may not recognize their symptoms. So then you need an objective, accurate test to tell you whether you can safely put the athlete back on the field. We don't have that right now."

During in-game evaluations for a concussion, team trainers and physicians must quickly synthesize the available evidence and make their best clinical judgment about a player's health. The responsibility for a safety-first decision, though, also lies in part with the athletes, the study's authors wrote:

"Although an increase in symptoms is highly suggestive of concussion, this relies on accurate reporting by the athlete who may not report symptoms because of a desire to return to play, a fear of letting teammates down, minimizing the seriousness of concussion, difficulty discerning symptoms, a delay in symptom development, or other reasons."

"We are still short of the holy grail, which is an objective test for concussion," Harmon said. "For now, this study shows how important it is for athletes to disclose their symptoms." 

The study was funded in part by the Jack and Luellen Cherneski and the Chisholm Foundation.