Debating the utility of computerised neurocognitive testing in the sports concussion arena

The purpose of this article was to contribute to an argument regarding the utility of computerised baseline and follow-up neurocognitive testing within the sports concussion arena. Heated debate around this issue via a number of contributions has appeared recently in the journal Current Sports Medicine Reports , with its use being roundly condemned by one party as ‘scientifically unfounded’ and therefore ‘financially irresponsible’. It is proposed that this vehemently negative viewpoint is located in a ‘smoke and mirrors’ portrayal of the validity of such neurocognitive screening, being substantiated on questionable extrapolations from laboratory-type group research to the clinical situation. The stance runs counter to the tenets of modern clinical neuropsychology, and is incompatible with more rigorous scientific pointers from current research. Abreast of the latest concussion in sport consensus recommendations, it is concluded that there is compelling support for the burgeoning use of computerised neurocognitive evaluation in the sports concussion arena as the optimal and most responsible healthcare currently available in this arena. SAJSM , vol 23 No. 4 2011


Introduction
In response to Randolph's recent article published in Current Sports Medicine Reports (CSMR), in which he argues against the utility of computerised neurocognitive baseline testing in the sports concussion context, 1 the present author submitted some critical commentary regarding the underpinnings of his argument, including material to suggest that his stance is counter-productive in terms of optimal healthcare and unwarranted. 2In turn, Randolph and McGrew have presented a number of challenging responses, 3,4 including the statement that 'to advocate for the addition of baseline neuropsychological testing for athletes at this time is scientifically unfounded, financially irresponsible and should be roundly condemned for confusing and raising false expectations in the athletes that are subjected to it as well as adding additional burden to an already overtaxed medical system'.4   However, considerable concern remains about what appears to be biased and uncritical dismissal of the use of neurocognitive screening in this forum generally by these authors (Randolph and McGrew), and an unwarranted demeaning of the ImPACT test in particular. 1 In this regard, the present author's previous commentary 2 drew attention to the limitations of evidence that is derived exclusively from group research to facilitate optimal health service, rather than incorporating other crucial factors such as theoretical pointers and clinical case observations given the multitude of genetic, biologic and co-morbid variations as has been eloquently argued elsewhere.

5-7
Further, the author's prior commentary 2 draws attention to Randolph's surprising criticism of the ImPACT test because it produces a multifunction test report rather than yielding a single indicator of brain dysfunction, whereas this can be considered to be one of its most significant strengths.A 'single entity' assessment approach has long been thoroughly discredited in modern clinical neuropsychology.

8,9
Finally, in terms of the various contributions to the debate in question, Kutcher, 10 in more balanced fashion, warns that Randolph's target in his most recent article is 'conspicuously narrow', and that the practitioner is advised to take cognizance of the complexity of the issue, being 'as leery of fool's gold as dirty bathwater'.
Accordingly, the purpose of the present commentary is to expand on the intricacies involved in this issue, with particular reference to Randolph's most recent article published in the CSMR. 1 The argument to be presented accords with sentiments expressed in a rebuttal to Randolph's earlier article in the JAT, 11 in which it was stated that 'the article does not present a balanced and representative review of the literature … and does nothing to move our understanding of concussion forward … In addition, the authors' opinions are inconsistent with current thought within the field of neuropsychology'. 12

Statistical misrepresentation in light of clinical applications
Extracted from a study of Broglio et al., 13 data are presented by Randolph to bolster an argument about the poor test-retest reliability and relative sensitivity of the ImPACT test. 1,3Much is made of a 40% false positive rate of impairment relative to baseline that occurred on at least one of the four ImPACT test composites as reported in that study. 1,3Using the analogy of a thermometer, it is suggested that one would not wish to use such an instrument if it were so unreliable as to 'classify 40% of healthy individuals as pyretic and 40% of pyretic individuals as healthy'. 3However, there are a number of serious difficulties inherent in these extrapolations, as follows.
Firstly, in the Broglio et al. study, three different computerised neurocognitive tests were administered to testees at baseline, day 45 and day 50.However, in the regular clinical situation, it is only a single neurocognitive test that is administered at baseline, and likewise on each follow-up test occasion.
Therefore, uncritical extrapolation of research outcome from the Broglio study to document clinically applicable test-retest reliability, false positive indications of impairment, etc., in respect of any of these three programmes in isolation, has seriously compromised validity on methodological grounds, due to the confounding effect on each other of the multiple test administrations.
Further, from a neuropsychological perspective, to draw an analogous comparison between the single score measurement of a thermometer and a psychometric instrument is inappropriate.Psychometric test scores do not have stand-alone positivistic meaning in their own right (as with a temperature thermometer), rather being a relativistic type of measurement requiring highly contextualised interpretation in the hands of those who have been trained in their use. 14A single change score in isolation on a test such as ImPACT, that incorporates at least four core composite scores in the clinical profile, 15 would be viewed sceptically in terms of its implications, and interpreted closely in relation to: the site of the injury; the symptom reports; the educational/ occupational

Abstract
The purpose of this article was to contribute to an argument regarding the utility of computerised baseline and follow-up neurocognitive testing within the sports concussion arena.Heated debate around this issue via a number of contributions has appeared recently in the journal Current Sports Medicine Reports, with its use being roundly condemned by one party as 'scientifically unfounded' and therefore 'financially irresponsible'.It is proposed that this vehemently negative viewpoint is located in a 'smoke and mirrors' portrayal of the validity of such neurocognitive screening, being substantiated on questionable extrapolations from laboratory-type group research to the clinical situation.The stance runs counter to the tenets of modern clinical neuropsychology, and is incompatible with more rigorous scientific pointers from current research.Abreast of the latest concussion in sport consensus recommendations, it is concluded that there is compelling support for the burgeoning use of computerised neurocognitive evaluation in the sports concussion arena as the optimal and most responsible healthcare currently available in this arena.

Ann B Shuttleworth-Edwards, PhD
Professor of Psychology, Director National Sports Concussion Initiative (NSCI), Department of Psychology, Rhodes University, Grahamstown, South Africa Correspondence to: Professor Ann Edwards (a.edwards@ru.ac.za)

Debating the utility of computerised neurocognitive testing in the sports concussion arena
background and special abilities and disabilities of the athlete; and, importantly, norms that are applicable to the testee in terms of the demographic influences of age, education and cultural affiliation.
Essentially, the point that is being made here is that false positive indications derived from any scores, and particularly a single score, in the de-contextualised group research context such as pertains to the study of Broglio et al., have minimal relevance in the appropriately conducted clinical neuropsychological examination.Accordingly, Randolph's 40% false positive challenge regarding the use of the ImPACT test is entirely without substance on clinical interpretive grounds, as well as on the most fundamental of methodological grounds.
In contrast to these dubious implications derived from the study of Broglio et al. are indications from many more rigorous studies, and some recent studies in particular, that serve to demonstrate an entirely opposite picture in terms of the discriminatory ability of the ImPACT programme. 17,18In these studies a high level of predictive value on the length of recovery is established for the ImPACT programme of 73% taking both the cognitive tests and symptom reports into account, with an increase of 24% in sensitivity over the use of the symptom reports alone. 17Early postconcussion neurocognitive assessment on ImPACT in an emergency department detected neurocognitive deficits that clinical grading could not, and correlated with the severity of deficit at 3-and 10-day post-injury follow-up. 18

Benefits of neuropsychological evaluation and concluding comments
Research outcome aside, there are non-debatable clinical advantages in having neuropsychological post-concussion evaluations available in the hands of the trained neuropsychologist.Such benefit includes the ability to demonstrate the extent of cognitive malfunction during the recovery process in order to provide guidelines for scholastic or occupational purposes.Even fairly brief periods of unrecognised and/ or unsubstantiated cognitive dysfunction in the wake of a concussion may have quite serious negative consequences within such settings, and calls for highly individualised recommendations supported by objective criteria.These goals are massively facilitated by the availability of follow-up computerised test data per se, and especially in comparison with baseline data.
In sum, taking all the above into consideration, in direct opposition to the expressed sentiments of Randolph and McGrew, the ImPACT test appears to be exactly the kind of tool that the present author would like to add to an investigative armamentarium with a view to enhancing medical management in respect of the sports concussive injury.It has clearly been endorsed by hundreds of clinicians worldwide in the burgeoning use of this particular tool.However, as cautioned by Kutcher, 10 whether using ImPACT or some other instrument of this type such as ANAM, CogState Sport or Headminder, this procedure should not be followed blindly as part of a marketing ploy.Nor should the process be seen as a shortcut to ruling brain dysfunction in or out. 14,19Rather it should take place via discretionary clinical evaluation of the optimal mechanism to employ in order to meet the health needs within the particular sports concussion arena, by a team of suitably qualified medical and neuropsychology practitioners.As such, the practice is fully in accordance with the recommendations of the most recent consensus on concussion in sport. 20Used appropriately within the confines of legitimate practice in clinical neuropsychology, the incorporation of computerised neurocognitive screening can be considered to have positively revolutionised the management of the sports concussive injury in the direction of becoming massively more refined than it was previously, thereby allowing for more optimal and responsible recovery and rehabilitative healthcare.

A note on the ImPact programme
The ImPACT programme was developed within a research context approximately ten years ago, 15 and is currently in its 4th updated webbased and technologically sophisticated version.It consists of a userfriendly series of tests and a symptom check list that take about 25 minutes to complete, and produces an automated report that collates the neurocognitive and symptom outcome, for multiple test-taking occasions.There is a spectrum of functional modalities assessed based on traditional well-researched neuropsychological stimuli, including visual and verbal memory (immediate and delayed), visual motor speed, reaction time, as well as a cognitive efficiency index, and a test of impulse control that provides an indicator of test-taking validity.
Restricted web-based access to the report by designated clinicians enables high level neuropsychological interpretation of the outcome and feedback within a matter of hours, regardless of location of the testee.The test is acknowledged as the most widely used of all computer programmes that have been especially devised for use in the sports concussion arena. 1