Inclusion Criteria for Measurement Tools of Interest to SCI
Initially the measures targeted for this review included any and all tools for which there was at minimum one study that examined psychometric properties (reliability, validity, responsiveness) using a spinal cord population. More specifically, only those peer-reviewed manuscripts that directly reported values for their sample of SCI individuals were included. For version 1, a list of 168 tools was originally derived (approximately 10 were different formats of a similar tool), and the number of tools increased with the version 2 and 3 updates as several other tools were found that were recently developed or had their psychometric properties evaluated with an SCI population. Given the vast number of tools, inclusion criteria was further narrowed to only include: Level 1 studies, defined by Kalpakjian et al. (2009) as studies with a primary aim to evaluate the psychometric properties of a measure; those tools that are familiar and of interest to clinicians (47 for version); as well as some tools (N = 4) such as the Barthel Index that are commonly known and used internationally. As a result, 47 tools were included in the original SCIRE document, 29 were added in version 2, and 15 more in version 3, for a total of 91 tools.
For version 1, a table identifying all tools was developed and clinicians (nurses, occupational therapists, physiatrists, physical therapists, psychologists, recreation therapists and social workers) from GF Strong Rehabilitation Centre (Vancouver, British Columbia) and Parkwood Hospital (London, Ontario) were surveyed. Tools were then selected for review based on receiving at least 5 tallies of interest and/or familiarity. A similar process was carried out for identifying pertinent tool for inclusion in the updates. Another table was developed that identified all new tools along with those tools that were not included in the original version. Clinicians and scientists then reviewed the list and selected tools to include in updates.
Note, while we recognize that in many randomized controlled trials investigators assess the reliability between their raters. Most often these efforts are to ensure stability of the results within the research team and therefore the results are not to generalize to the larger pool of tool users. As a result, the statistical model chosen to calculate the intra class correlation coefficient (ICC) for example is different given this purpose (Shrout & Fleiss 1979). Given the dearth of RCTs conducted in SCI research the number of studies lost is potentially very small.
Searching the Literature
The Pubmed, MedLine, CINHAL, Embase, HaPI, PsycInfo, and Sportdiscus electronic databases were searched (1986 to January 2006 for version 1, from 2006 to January 2008 for version 2, and from 2008 through to the end of 2009 for version 3) in an effort to locate papers reporting on measures. Additional searching was conducted by archiving the references of papers obtained from the electronic search. The key word spinal cord injury was used across each of the databases while the following terms varied in combination with spinal cord injury depending on the database used: validation studies, instrument validation, external validity, internal validity, criterion-related validity, concurrent validity, discriminant validity, content validity, face validity, predictive validity, reliability, interrater reliability, intrarater reliability, test-retest reliability, reproducibility, responsiveness, sensitivity to change, evidence-based medicine, outcome measures, clinical assessment tools, scales and measures. A database file was established using RefWorks to organize potential articles of interest. After eliminating duplicate manuscripts data extractors reviewed titles and abstracts in order to retain relevant papers. At this point all of the articles were read and the relevant information (reliability, validity and responsiveness coefficients and descriptions) was extracted. See Appendix 1 on page 8 for a copy of the data extraction form.
Classifying the Tools
To assist with the process of organizing the tools we used a conceptual framework developed by the World Health Organization called the International Classification of Functioning, Disability and Health or ICF for short (WHO 2001). See Figure 1. The advantages of using this framework include: 1) it is well recognized and used by the international community; 2) it was created to provide standard language for use when discussing health and health-related domains; and 3) other reviews of outcome measures have used the ICF for similar purposes (Salter et al. 2005).
Figure 1. Overview of the International Classification of Function, Disability and Health
According to the clinical practice guidelines (Consortium for Spinal Cord Medicine 1999) the expected outcomes after SCI range from basic physiological function such as motor/sensory function to higher level outcomes such as functional independence and social integration. The ICF accounts for these within the 4 constructs of its Functioning and Disability component. The tools were classified according to the body function/structure, activity and participation constructs. Table 1 briefly outlines the definitions.
Table 1. Definitions for Body Function/Structure, Activity and Participation
Quality of life (QOL) surveys are a newly emerging category of outcome measurements. To capture these assessment tools, we included an additional dimension in order to help classify QOL tools. While some clinical physiologists might question whether QOL is a true ‘outcome’ tool, the Food and Drug Administration (FDA) asks that QOL assessments be included as part of any clinical trial protocol and program (often as a secondary outcome measure). Since the perceived benefit by the target market population of any therapeutic intervention is an important consideration we have included QOL in our review.
Because disability is no longer understood as a feature of the individual, but rather as the outcome of the interaction between an individual with a health condition and psychological and environmental processes, measures to assess environmental factors, such as physical, social or attitudinal barriers, and psychological factors, such as self-efficacy, are becoming more and more important. According to the ICF, the interaction between an individual with a health condition and environmental barriers may lead to disability at the participation level (WHO 2001).Such recognition that environmental factors influence participation may partly explain the reasons why people with the same severity of disease and level of impairment often function differently in daily life (Keysor et al. 2006). Thus, the assessment of environmental factors will help in determining the need for interventions to address physical, social, or attitudinal barriers.
For version 1, three classifiers knowledgeable to both outcome measures and the ICF independently categorized all of the tools. The classifiers later met to reconcile any disagreement about classification of the tools. When a multidimensional tool covered more than one construct (e.g. activity and participation) they placed it in the category where the tool had the most items. The tools were divided once again into sub-classifications based on the sub-domains within each of the body function/structure, activity and participation areas. Upon classification into the main domains, the tools were further categorized into appropriate subgroups based on the ICF definitions. During the completeion of the version 3 update, new tools were found that did not easily fit into any one of the ICF domains or the QoL category. Currently, we have included these tools in the ‘Body Functions/Structures’ category. While we acknowledge this is not a perfect fit, we considered it to be the best option until new iterations of the ICF are available. Please see Table 2 which outlines these subcategories.
