Keywords
Adult spinal deformity outcomesOutcomes in adult spinal deformitySpinal deformity outcomesLow back pain vs. adult spinal deformityCore outcome set in adult spinal deformityThe Need for Uniform Outcome Assessment
Several multicenter prospective registries have been created to evaluate quality and value of delivered spine care in population of patients with adult spinal deformity (ASD) [1]. Despite these efforts, global surveillance of patients with ASD and comparative effectiveness of treatment has been hindered by lack of a standardized systematic approach towards outcome measurement and reporting. Although meaningful and valuable for its own local purposes at the originating site, data elements, which are not completely consistent between registries, may not be used for comparison or aggregate analysis as different outcomes, measurement instruments, and risk stratification variables are used. This inconsistency limits the application of research findings into clinical practice, and prevents clinician investigators from informing policy makers about efficacy of various treatment strategies [2, 3]. One way to address these issues is by developing and implementing an agreed standardized set of treatment outcomes, including a set of (risk) factors that influence these outcomes, also known as a core outcome set (COS). This is a minimum standard set that is recommended for measuring and reporting outcomes in all clinical studies and national outcome registries. Different COSs for several health conditions have already been published and are currently being implemented in routine clinical practice (e.g., for traumatic brain injury and spine trauma) [4, 5].
In a landmark paper, Porter et al. suggested that progress in routine outcome measurement is hampered by the “let a thousand flowers bloom” approach –with each organization reinventing the wheel, tweaking existing measures and risk factors, or inventing one’s own [6]. To circumvent this variability, multiple international initiatives have been introduced to bring together researchers who are interested in outcome standardization. These initiatives, including OMERACT (Outcome Measures for Rheumatology Clinical Trials), COMET (Core Outcome Measures in Effectiveness Trials), and ICHOM (International Consortium for Health Outcomes Measurement), provide methodological guidelines for the development of COSs, and add much needed framework for outcome reporting and classification [7, 8].
Low Back Pain Versus ASD
In the case of low back pain (LBP), multiple core outcome sets have been developed using formal consensus methods, such as ICHOM low back pain (LBP) standard outcome set and COS developed by Chiarotto et al. in 2015 [9, 10]. Core areas recommended for inclusion in these outcome sets differ based on the target user, which has implications for the framework used to develop and report each one. For example, the ICHOM LBP set is based on ‘what matters to patients’, and is aimed at routine outcome measurements to assess the quality of delivered care as part of value-based healthcare initiatives, and for research purposes, whereas the COS was developed for use in clinical trials in low back pain. Currently these COSs have been implemented in multiple (inter-) national spine registries, and have been reported across clinical trials and observational research studies [9–12]. The introduction of these COSs has been essential to the development of a more standardized and systematic approach in measuring and reporting of outcomes in patients with LBP, which is a frequently observed symptom in patients with ASD. However, patients with ASD have significantly other clinical and radiological characteristics than patients with low back pain. For example, neurological and pulmonary function, which is observed to be affected before and after ASD surgery [13, 14], is not included in current standard sets for LBP. Both outcomes could be considered for inclusion in a COS specifically developed for ASD, which makes it necessary to develop a separate COS for ASD that is aligned with the LBP core sets.
Basic Concepts in Outcome Measurement
For the development of a core outcome set, it is important that a universally accepted conceptual framework is used that covers human functioning. The International Classification of Functioning, Disability and Health (ICF) framework, adopted by the World Health Organization, provides the necessary universal language for health outcome measures. More specifically, the ICF framework is intended to describe functional states associated with various health conditions according to a hierarchical classification system [15–17]. Items of (patient-reported and clinician-based) outcome measures can be linked to one or more ICF health outcome domains, depending on the number of meaningful concepts contained in that item (e.g., item 4 of the patient-reported Oswestry Disability Index version 2.1a, “pain does not prevent me from walking any distance”, refers to the outcome domains “b280 sensation of pain” and “d450 walking”) [18]. Notably, many different health outcomes are currently measured ranging from very generic patient-reported outcome measures (PROMs) to highly disease- and treatment-specific clinician-based measures; each of them have their own purpose for which they were developed.
Prior to the discussion of the process of reaching agreement on a standardized collection of outcomes, it is important to emphasize a basic concept in outcome measurement. A distinction exists between outcome domains, which are defined as the concept or construct to be measured (i.e., pain, walking, self-image) and measurement instruments used to measure these outcome domains (i.e., Visual Analogue Scale, Oswestry Disability Index, and Scoliosis Research Society-22r). A certain outcome domain can be measured using different instruments, which can be categorized into clinician-based or PROMs. For example, the outcome domain “walking” can be measured by using a “6-minute walk” test, which is a clinician-based outcome measurement instrument, or by using a PROM, such as the Oswestry Disability Index. Furthermore, outcome measures can be either ‘subjective’ – i.e., relying on the interpretation and evaluation by either the patient or by the clinician – or more ‘objective’, which means independent of the opinion of the observer – e.g., the 6-minute walk test. Note that ‘objective’ measures do not necessarily mean they are of better methodological quality (e.g., validity or reliability).
Risk Factors and Case-Mix Variables
Benchmarking outcomes is valuable for identifying best practices and in increasing the quality of care [19]. True effects – i.e., improved outcomes that are related to improved quality of care – should be distinguished from improved outcomes that are caused by factors related to the population operated on (e.g., less comorbidity, better pre-operative health), and should not be related to the treatment quality itself. As such, adjustment for patient-related influencing risk factors and case mix variables is required. For example, without this adjustment to the outcomes, patients with more comorbidity would appear to have worse outcomes. When developing a COS it is important not only to reach consensus on the outcomes but also on the pre-treatment risk factors and case-mix variables that could possibly influence the outcome.
What Is a Core Outcome Set?
Benefits
The issue of inconsistent outcome reporting across studies and outcome registries could be addressed by the development and implementation of a COS. This is a consensus-based minimum set of outcome domains, measurement instruments, risk factors and case-mix variables, which should be measured and reported in all clinical studies for a specific disease or disorder [3, 20]. These sets; however, do not imply that outcomes reported should be restricted to those agreed upon in a COS, but rather it is expected that the COS will be collected and reported in future clinical studies and other outcomes or influencing factors can be added that might be of specific interest to researchers, clinicians and institutions. The core outcomes allow for adequate pooling of data and proper comparisons across future clinical studies, systematic reviews, and so-called (national) outcome registries to improve the quality of daily clinical practice.
How to Develop a Core Outcome Set?
Guidelines on COS Development
The COMET initiative, launched in 2010, provides guidelines for the development and application of COSs [21]. The overall aim of COMET is to bring together researchers interested in the development, application and promotion of a COS using rigorous consensus methods. This facilitates the exchange of ideas and information to improve health service users and foster methodological research in the area of standardizing outcomes. Data on current ongoing COS efforts are included in a free, open access repository which is updated periodically [22]. Over the years, there has been a sustained growth in use of the website and database suggesting the gain of interest in the development and implementation of COSs [8].
Scope
Before developing a COS, it is important to agree upon and clearly describe the scope, including details of the health condition, population and type of intervention . It is recommended to check for existing or ongoing development of a COS. Consultation of the database of COMET [www.comet-initiative.org] is recommended to have an overview of the scope and initiatives in the specific area to avoid unnecessary duplication of effort and minimize waste.
Identification of Existing Knowledge
A systematic review of the outcomes (outcome domains and measurement instruments ) reported in clinical studies in the area of interest could be used to provide a clear rationale and to explore whether a need exists to develop a COS. In this systematic review current strengths, weaknesses and gaps in outcome assessment in the area of interest could be highlighted and review results could generate a list of potential outcomes to be included in the future COS. To display current outcome domains and measures as reported in eligible studies and to relate to them the concept of human functioning , the ICF framework is frequently used. For example, in a systematic review that provided the preparatory stage of the COS ASD development, outcome domains related to mobility and pain were identified, and were found to be represented by currently used PROMs in the surgical literature of ASD [18]. A gap was identified regarding neurological and pulmonary function, which are known to be affected in the surgical treatment of complex ASD [13, 14]. Both neurological and pulmonary function should be considered for inclusion in the COS, albeit that measuring and reporting of both outcomes have been limited to date.
Stakeholders
Ideally, a COS should include outcomes that are relevant and important to key stakeholders, including patients, health care professionals, researchers, health care policy makers, industry representatives. This means that all relevant stakeholders should be represented when developing a COS using consensus-based methods. The decision on the number of stakeholders from each group is dependent upon the particular scope of the COS as well as practical feasibility considerations. To reach formal consensus, it is recommended to use a method for consensus and to include an uneven number of stakeholders to avoid a tie. If the aim is to develop a COS for global implementation, geographic balance of the consensus panel should be considered as well as the importance of a certain outcome, which might be dependent on cultural and geographical differences.
Consensus Methods
In the development of a COS, the existing knowledge derived from the systematic review in the preparatory stage should be enriched by practice-based knowledge provided by a formal consensus procedure with experts in the specific area of interest. Multiple consensus methods are available, including the (modified) Delphi method [23], Nominal Group Techniques [7], focus groups [24], and individual interviews [25]. The choice of method should be balanced between considerations concerning practicality and ensuring that a diverse range of opinions is heard [22]. A preferred consensus-based method for the development of a COS is the modified Delphi methodology [21]; an iterative process designed to reach formal consensus among a large group of experts using a series of rounds in which participants vote on or rate statements. In the case of a COS: e.g., in- or exclusion of certain outcomes. A predefined threshold for consensus is determined – e.g., 75% equal votes. The Delphi process is anonymous to make sure all participants have an equal say. After each round, stakeholders receive anonymous feedback report of the group’s ratings to reflect on these outcomes and rate outcomes again where no consensus was reached. Often in the final round, stakeholders are invited to meet face-to-face to review the Delphi findings and discuss the COS in a formal setting led by an independent moderator to reach a final consensus. In general, formal consensus is reached on outcome domains (i.e., ‘what’ should be measured), after which consensus is reached on the measurement instruments (‘how’ to measure the outcome domains), as well as time-points for assessment.
Challenges in the Development of a COS
The development of a COS is an important step towards a standardized and systematic outcome measurement reporting for a specific health condition. It will not achieve its goal if reports of the COS are not complete and transparent. The Core Outcome Set–STAndards for Reporting (COS-STAR) statement is a published checklist of 18 items considered essential for transparent and complete reporting in all COS studies [26]. This checklist supports the completeness in the report of a COS, which future users can use to judge whether the recommended standard set is relevant for their population of interest. Consideration should be given in advance to other challenges when developing a COS. Pre-selection of stakeholders (by personal invitation) to participate in the development of a COS may increase the risk of selection bias, and may result in a COS that is not applicable to all cultures. As such, participation of a patient panel and validation in international patient-focus groups needs to be considered. Moreover, inconclusive or undefined concepts can be encountered during identification of potential core outcome domains for which no measurement instruments exist and are yet to be developed. After completion of a COS sufficient attention should be given to dissemination and implementation.
Core Outcome Set for ASD
Routine outcome measurement is challenging but when implemented successfully it has many benefits. For example, research has shown that asking providers to measure and report outcomes alone already improves performance [27]. Understanding and comparing outcomes facilitates continuous learning and improvement of own strategies through learning from best practices. Continuous improvement and informed decision-making could be important driving forces in improving the quality of healthcare delivery by for example value-based healthcare. This may specifically be beneficial for complex clinical entities such as ASD where randomized controlled trials (RCTs) are often not practical, ethically challenging, and for which a tremendous variability in management strategies exists. Given the expanding interest in patient-centered care and increasing treatment costs, a systematic and standardized approach towards routine outcome assessment by means of a COS for ASD is of utmost importance in the era towards value-based healthcare. This will subsequently allow for intentional data pooling and benchmarking of outcomes, and in turn highlight the value of provided (surgical) treatment.
The development of a COS for ASD has been initiated by the Scoliosis Research Society (SRS) and is currently under development. The scope is ASD patients over 25 years of age undergoing surgical or non-surgical treatment. A modified Delphi study is currently being performed. The working group includes a total of 25 stakeholders from across the world that had to meet strict membership requirements of the SRS [28], which includes the ability to demonstrate their interest in spine deformity (e.g., attend annual meetings, and submit and review abstracts), and at least 5 years of experience in spine deformity surgery or research. A project team not participating in the Delphi rounds is guiding and coordinating the efforts of the larger working group of panelists.
Case example: Baseline and pre-operative characteristics
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
