Outcome assessment selection criteria
Definition
Validity
Extent to which a scale measures what it claims to measure
Reliability
Degree to which results are replicable
Sensitivity or precision
Ability to detect change
Time required to complete the measure
Time required for administration
Expertise required to complete the measure
Amount of training and knowledge required by the rater
Comparability
Frequency of use by other professionals
Continuity
Time of recovery in which the measure is reasonably applicable and sensitive to change
Phone/internet capability
Ability to administer the measure by phone or internet
Standardization
Degree to which the scale was thoughtfully developed for the population
Supportive documentation
Quality of manualized information pertaining to administration procedures, limitations, etc.
Age limitations
Degree to which the instrument is relevant across the lifespan
Comprehensiveness
Degree to which the instrument captures the multidimensional nature of outcome after TBI
Availability of tool
Ease with which an instrument can be acquired
The remainder of the chapter will serve as a review of outcome measures commonly used following TBI. We will focus on measures used in the assessment of early and late outcomes from hospital discharge to late follow-up (1 year or greater post-injury). Clinically relevant information about each measure will be included in tables. Practical considerations regarding the use of measures in clinical practice will be discussed. Finally, a case example will be used to highlight the potential uses of various measures.
Case Example:
Mr. Smith is a 48 year–old, right–handed, married gentleman who sustained a traumatic brain injury with polytrauma secondary to a motor vehicle accident. At the time of his injury, Mr. Smith was employed full–time as a foreman in a construction company. Mr. Smith completed college and denied any history of academic difficulties. He reported a history of alcohol dependency, for which he received inpatient and outpatient treatment about 8 years prior to this injury. He was then sober for almost 7 years, but returned to regular, heavy alcohol consumption about 14 months prior to injury. There was also report of remote history of recreational drug use.
Regarding his motor vehicle accident, Mr. Smith had positive loss of consciousness at the scene and was transported via EMS to a local level one trauma center, where his Glasgow Coma Scale (GCS) score upon arrival to the emergency room was 7. The initial cranial CT revealed a left frontotemporal subarachnoid hemorrhage, right frontal contusion, and subdural hematoma, without midline shift. Medical workup additionally revealed a positive blood alcohol level. Mr. Smith did not require neurosurgical intervention and there was no report of early seizure activity. He began following commands 7 days post–injury.
Mr. Smith was transferred to an acute rehabilitation hospital about 3 weeks post–injury. Duration of posttraumatic amnesia, based on serial assessment with the Galveston Orientation and Amnesia Test (GOAT), was 1 month. Results of a baseline neuropsychological evaluation at approximately 5 weeks post–injury were notable for impairments in memory, executive functioning, and cognitive and motor processing speed. Behaviorally, Mr. Smith’s wife noted that he displayed reduced frustration tolerance since his injury. He also presented with poor awareness of his injury–related deficits and their implications.
Mr. Smith was transferred from the acute rehabilitation setting to a post–acute residential treatment facility at about 2½ months post–injury, where his treatment program focused on facilitating his ongoing physical recovery as well as improving his awareness and implementing cognitive compensatory strategies. At 4 months post-injury, Mr. Smith transitioned to an outpatient day treatment program, where he worked with a vocational counselor to facilitate his vocational re–integration. Although the combined severity of his persistent cognitive and physical impairments did not allow for his return to his pre–injury position, his vocational counselor worked with Mr. Smith to identify alternative vocational options. Mr. Smith and his wife stayed with family during his post–acute recovery period so that he could attend the outpatient day treatment program, but eventually transitioned back to their home in a rural community.
Global Measures of Outcome
For an overview of practical considerations regarding the use of global measures in clinical practice please refer to Table 2. Global outcome measures such as the Glasgow Outcome Scale (GOS) [21] are broad-based measures of outcome after TBI. As such, these measures play a very limited role in informing individualized treatment planning and may actually be best suited to describe outcomes in groups of cases recovering from severe TBI [22]. There is evidence, however, that ratings on global outcome measures in the first several months post injury are predictive of long-term psychosocial outcomes [23, 24]. These measures are characteristically rapid to administer, typically requiring the rater to briefly synthesize several indicators of impairment, activity, and participation before assigning a categorical rating to the person with TBI. The relative insensitivity to change in early global outcome measures, such as the five-category GOS, led to the development of global measures with a greater number of categories (i.e., Extended GOS) [25] and others that were summed across individual items to create an ordinal score (i.e., DRS; [6]). While the items on the DRS still measure a combination of impairment, activity, and participation factors with no attempt to assess the person’s perception of their own level of functioning, the breaking down of broadly described categorical outcomes into more plainly delineated items represented an important transition to more circumscribed measures of activity and participation.
Table 2
Global measures of outcome
Instrument (reference) | Content & administration | Relevant clinical issues |
---|---|---|
The GOS is a measure of global outcome that classifies individuals into 1 of 5 rank-ordered categories: good recovery, moderate disability, severe disability, vegetative state, dead. The GOS requires only a few minutes to rate; ratings can be obtained by medical record review | • “Gold standard” in neurosurgical outcome studies • Quick and easy to complete, categories correspond to those used by “laypersons,” making it clinically useful in delivering prognostic information | |
The GOS-E has 8 rank-ordered categories: upper good recovery, lower good recovery, upper moderate disability, lower moderate disability, upper severe disability, lower severe disability, vegetative state, dead. The GOS-E may take 5–15 min to rate; ratings are made via structured interview | • GOS less sensitive to recovery after TBI beyond 6 months post-injury [41] | |
• GOS-E addressed limitations of GOS, by adding more specific categories to improve sensitivity and a structured interview to improve reliability; found to be associated with neuropsychological test findings and measures of disability [42] | ||
Disability Rating Scale (DRS) [6] | The DRS is a measure of global outcome intended to assess general functional changes over the course of recovery after brain injury, from coma to community. The DRS consists of 8 items corresponding to the following areas of functioning: eye opening, verbalization, motor response, level of cognitive ability for feeding, toileting, and grooming, overall level of independence, and employability. Scores range from 0 (no disability) to 29 (extreme vegetative state). The DRS can be rated in-person or via phone interview with the individual or his/her support network or from retrospective medical record review. Time to rate the DRS depends on knowledge of individual (range <1–15 min) | • Developed in rehabilitation setting; can be used both in inpatient and follow-up evaluation to track recovery over long term |
• Found to be more sensitive than GOS to changes in recovery; however, still insensitive to changes for those at higher range of functioning (i.e., lower end of scale) as well as of subtle but sometimes significant changes made by an individual in a specified time window. Likelihood for ceiling effects increase as time post injury continues | ||
• DRS scores have been used to predict return to competitive employment after TBI. DRS scores have also been shown to correlate with supervision needs | ||
Rancho Level of Cognitive Functioning Scale (LCFS) [43] | The LCFS was developed to measure cognitive functioning in individuals emerging from coma to facilitate treatment planning and to assess recovery and outcome. Individuals are categorized into 1 of 8 levels: I no response; II generalized; III localized; IV confused-agitated; V confused; inappropriate; non-agitated; VI confused-appropriate; VII automatic-appropriate; VIII purposeful-appropriate | • Limited sensitivity to subtle changes in recovery. Can be difficult to classify individuals into one category when they manifest characteristics of multiple categories |
• LCFS scores have been shown to be predictive of returning to work and school [44] | ||
Supervision Rating Scale (SRS) [45] | The SRS is a 1-item instrument that measures amount of supervision a person is receiving from caregivers. Responses are rated on a 13-point ordinal scale and range from 1 (no supervision needs) to 13 (patient in physical restraints). Ratings are optimally based on direct observation but can be extrapolated from information in medical record in certain cases; completion time is brief | • Ratings are based on amount of supervision actually received, not what is judged or predicted to be needed. Ratings reflect level of supervision due to cumulative impact of different (i.e., cognitive, behavioral, physical) symptoms |
• SRS ratings have been shown to be strongly associated with ratings on DRS and GOS. SRS rating also shown to have consistent relationship with independence in self-care and instrumental activities of daily living [45] |
Case Example—continued. Mr. Smith’s treatment teams in the acute rehabilitation setting and at the post–acute residential facility may have found ratings on global outcome measures, such as GOS and DRS ratings made at the time of discharge from the acute trauma center, helpful as prognostic indictors for long–term outcome. While precise predictions regarding specific long–term activity and participation restrictions would be unreasonable, the treatment staff at these facilities could have used results of initial injury severity characteristics and global outcome indicators to tentatively prepare the person with TBI and their family for the most likely scenarios regarding supervision needs or likelihood of returning to competitive employment [26]. Prognostic information of this sort could be essential in helping the family to prepare for the long–term consequences of a significant TBI (e.g., loss of income, need for supervision, etc.).
Measures of Disability and Activities of Daily Living
Measures of disability and activities of daily living provide specific information about a range of behaviors considered essential for self-care (See Table 3). Behaviors are typically rated in terms of the patient’s level of independence, or by how much assistance and supervision are necessary for the patient to safely and successfully complete common tasks. These basic self-care behaviors typically have been the focus of interventions during the acute and sub-acute recovery periods following a significant brain injury, thus making measures of activities of daily living ideally suited for tracking individual recovery during initial inpatient care. Of note, however, is that as inpatient, hospital-based rehabilitation lengths of stay have decreased, some of these goals are now frequently being addressed in the post-acute phase of recovery [27].
Table 3
Measures of disability and activities of daily living
Instrument (reference) | Content & administration | Relevant clinical issues |
---|---|---|
Functional Independence Measure (FIM) [5] | The FIM is an 18-item ordinal scale consisting of 13 physical independence items (i.e., self-care, sphincter control, transfers, mobility) and 5 cognitive items (i.e., communication and social cognition). Scores on each item range from 1 (total assistance required) to 7 (complete independence). The Uniform Data System for Medical Rehabilitation (UDS) provides training materials and includes standards required for inter-rater reliability. The FIM can be completed in 20–30 min via clinician conference, observations, or telephone interview | • Widely used in acute inpatient rehabilitation setting to assess changes in level of functioning between admission and discharge |
• Correlated with GOS and DRS | ||
• Insensitive to more subtle changes expected after acute inpatient rehabilitation discharge; ceiling effects at 1 year post-injury [7] | ||
• Few items emphasizing cognitive, behavioral, and communication, and community functioning—therefore, less relevant for TBI population | ||
Functional Assessment Measure (FAM) [28] | The 12-item FAM was developed as an adjunct to the FIM, to enhance applicability to those with brain injury, and does not stand alone. Items are also rated on a 7-point ordinal scale. The FAM addresses cognitive, behavioral, communication, and community functioning considerations. The combined 30-item FIM + FAM requires approximately 30 min to complete | • Ceiling effects less problematic when FAM added to the FIM [7] |
• May be a more valid indicator of disability during follow-up assessments and for post-acute rehabilitation settings, given emphasis on community functioning [46] | ||
Mayo-Portland Adaptability Inventory-4 (MPAI-4) [29] | The MPAI-4 contains 29 items with 3 subscales (Ability Index, Adjustment Index, Participation Index) assessing physical, cognitive, emotional, behavioral, and social sequelae as well as obstacles to community integration that may be encountered after brain injury. The MPAI-4 was designed to assist in: (1) clinical evaluation/rehabilitation planning during the post-acute period of recovery; (2) evaluating the effectiveness of post-acute rehabilitation programs; and (3) better understanding long-term outcome after acquired brain injury. May be completed by the individual with injury, his/her significant other, and/or treating clinical staff. A manual for the MPAI is available online | • Ratings on MPAI at admission shown to correlate with outpatient rehabilitation outcomes [48] |
• MPAI ratings and time since injury shown to be predictive of job placement after participation in vocational rehabilitation [49] | ||
• Staff MPAI ratings predictive of vocational and independent living outcome 1 year after completion of outpatient rehabilitation [50] | ||
Since the raters must observe and evaluate many different behaviors, measures of disability and activities of daily living typically require longer administration times relative to global outcome measures. The design of these measures (i.e., many items rated individually on Likert-type scales with similar underlying anchors), however, allows for the application of powerful test development techniques such as Item Response Theory. The FIM [5] is one of the most widely used measures of activities of daily living. One important limitation of the FIM is its relative insensitivity to change in the post-acute stages of recovery as the focus of TBI rehabilitation transcends basic ADLS and transitions to maximizing functioning in the community [7]. It should be noted, however, that FIM scores measured during acute recovery are predictive of important psychosocial outcomes such as employment [23]. The relatively low ceiling of the FIM was addressed by the development of the FAM [28], an extension of the FIM which includes a greater focus on functioning in the community. While this attempt to raise the ceiling of the FIM by creating additional items relevant to community reintegration was conceptually sound, the FIM + FAM still suffers from significant limitations in monitoring long-term outcome from TBI [7].
The Mayo-Portland Adaptability Inventory-4 (MPAI-4) [29] offers an alternative to earlier measures of activity and was developed specifically to monitor outcome following TBI. Rather than rating the level of independence in completing various self-care behaviors, the first 21 items of the MPAI-4 require the rater to determine the extent to which various cognitive, emotional, and social problems interfere with activities. The MPAI-4 also includes 8-items comprising a Participation Index [30], on which the rater is asked to determine how much assistance is required for successful completion of various instrumental activities essential for community participation. The Participation Index ostensibly makes the MPAI-4 an ideal measure for tracking a patient’s recovery after discharge into the community. Another advantage of the MPAI-4 is the effort to allow for completion of the measure by the clinician, significant other, and the patient [15]. Allowing significant others to rate the survivor of brain injury seems ideal in cases where involvement of the treatment team may be limited after the injury, and the ability to compare self-ratings to clinician and significant other ratings provides the potential of exploring the extent of problems with awareness on self-reporting.
Case Example—continued: As with global outcome indicators, rehabilitation staff at the acute care hospital and residential treatment facility certainly found the FIM and FAM useful in making predictions regarding Mr. Smith’s long–term psychosocial outcomes, such as employability. The design of these measures of activities of daily living also makes them ideal for selecting specific intervention targets (i.e., treatment planning) and tracking progress in therapy. In the case of Mr. Smith, the results of the FIM and FAM indicated need for additional physical and cognitive rehabilitation. Furthermore, the rehabilitation staff collected MPAI–4 ratings from the Mr. Smith, his rehabilitation team members, and his wife. Results from the three raters were used to assess and redress issues with awareness.
Measures of Community Participation
Community participation, as defined by independent living, social and leisure activity, and productivity, is the ultimate goal of persons with a history of TBI and the most important indicator of long-term success for rehabilitation programs [31]. Widely used measures, such as the Craig Handicap Assessment and Reporting Technique (CHART) [32] and the CIQ [33], focus on observable indicators of outcome (e.g., frequency of behaviors) (See Table 4). While measurement of observable behavior should arguably result in a more psychometrically robust measure, it has been suggested that a limitation of these and other (e.g. MPAI-4) participation measures is the lack of method to address individual differences in priorities [31]. More recently developed measures of participation, such as the Participation Objective, Participation Subjective (POPS) [34], allow the person with TBI to share their subjective impressions regarding the importance of each activity (Participation Subjective), in addition to indicating the frequency with which they engage in that activity (Participation Objective). As intended by the design of this measure, the subjective scales are more highly related to measures of well-being than objective scales. The Participation Assessment with Recombined Tools—Objective (PART-0) [35] and Subjective (PART-S) [36] also separate the objective and subjective constructs of participation.
Table 4
Measures of community participation
Instrument (reference) | Content & administration | Relevant clinical issues |
---|---|---|
Craig Handicap Assessment and Reporting Technique (CHART) [32]
![]() Stay updated, free articles. Join our Telegram channel![]() Full access? Get Clinical Tree![]() ![]() ![]() |