Abstract
In the past few decades, a growing number of clinical registries have been set up across specialties, by research collaborators, professional societies, or as a result of health authority requirement. Multicenter neurosurgery and subspecialty registries cover a large number of patients and provide real-world data, and have been increasingly employed for safety and quality assurance, outcome improvement projects, and research in clinical and cost-effectiveness. This chapter focuses on background, principles, and practical issues of clinical registries, as illustrated by examples. Furthermore, over the past 20–30 years, there has been a rise in pragmatic registry-based observational studies which offer some advantages over the traditional gold standard of randomized control trials. A comparison of the two research methodologies is made. Their relative strengths, limitations, and challenges are highlighted and their complementary roles are emphasized.
Keywords
Multicenter, Registries, Quality, Outcome improvement, Research, Randomized controlled trial, Pragmatic registry-based observational studies
Outline
Introduction 142
History of Clinical Registries 143
How Do Registries Improve Outcomes? 145
Gathering Relevant Epidemiologic and Clinical Data 146
Setting Benchmarks and Standards 146
Postmarketing Surveillance of Medical Devices 146
Clinical Research 147
Establishment of Guidelines 147
Implementation of Guidelines Within Registries 147
Participation in Data Collection May Improve Outcome 148
Comparison of RCT and Registry-Based Research 149
Generalizability 149
Sample Size and Statistical Power 151
Logistics and Cost 151
Longitudinal Data 151
Availability of Results 152
Relative Appropriateness for RCTs and Registry-Based Research for Specific Clinical Questions 152
Detection of Variation in Clinical Practice 152
Registry-Based Research: Special Considerations and Limitations 152
Design of Registry 153
Ethical, Legal, Confidentiality, and Governance Issues 153
Informed Consent 153
Data Collection Methodology 153
Data Quality 154
Study Design Considerations 154
Biases 154
Statistical Considerations 155
Registry-Based Randomized Trials 157
Examples of Clinical Registries in Neurosurgery 157
Patient-Reported Outcomes 163
Conclusion 163
References 163
Introduction
In the past few decades, a growing number of clinical registries have been set up across specialties for various purposes. Owing to their ability to cover a large number of patients and provide real-world data, they have been increasingly employed for outcome improvement projects. The Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network is a collection of research centers created by the Agency for Healthcare Research and Quality (AHRQ) in 2005. According to DEcIDE and AHRQ, a patient registry is “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.” A multicenter registry can improve patient outcomes by “understanding natural history, assessing or monitoring real-world safety and effectiveness, assessing quality of care and provider performance, and assessing cost-effectiveness.” A registry of patient-reported outcomes can also provide a different perspective in health care provision.
This chapter focuses on the aspects of clinical registries relevant to outcome improvement, including their background, principles, and practical issues. Examples are used for illustration; however, it is not the chapter’s objective to include an exhaustive list of neurosurgical registries.
Several national databases in the US collect vast administrative data. These include the Healthcare Cost and Utilization Project Database (HCUP), the HCUP Kids’ Inpatient Database (KIDS), the National Healthcare Quality and Disparities Report Databases, and the Social Security Administration’s Death Master File. These databases provide important epidemiologic data such as prognostic factors for specific neurosurgical diagnoses, both patient-related and nonpatient-related. An example of a patient-related factor is age, and that of a nonpatient-related factor is hospital size. Similarly, other “big data” sources (e.g., medical records system, institutional or organizational databases, large simple trials, health surveys, administrative and health insurance claims databases, death and birth records, and census databases) provide useful population statistics for quality improvement purposes and are sometimes linked to registries for epidemiologic and research purposes. However, such databases tend to include limited clinical information, for example, disease severity, exact surgical procedures, and thus do not provide the same level of details as a disease or patient population group clinical registry. Furthermore, some studies have shown that administrative databases underreport the complication rate. These databases are out of the scope of this chapter and will not be discussed further.
History of Clinical Registries
Clinical registries have a long history, although they only became an important and recognized outcome improvement and research tool in the past two decades. The recent growth is partly due to the collaboration facilitated by advances in information technology and the application of advanced statistical methodologies. The purposes of setting up a clinical registry are multifold. The main reasons are: (a) obtaining demographic and epidemiologic information, and establishing the natural history, for patients with specific conditions or those who have undergone specific treatments; (b) safety and quality assurance and calculation of site-specific risk-adjusted event rates, (c) detection of early medical device failure; and (d) research in clinical and cost-effectiveness. Registries could be created by research collaborators, professional societies, or as a result of a health authority requirement.
The most notable early database is the Framingham Heart Study. The study began in 1948 and included 5209 participants with no prior history of cardiovascular disease. From 1950 to 2017, 3427 articles have been published based on the Framingham data.
The Surveillance, Epidemiology, and End Results (SEER) Program database of the National Cancer Institute was set up in 1973. It tracks data on diagnoses, treatments, and survivals of patients with cancer, including those affecting the central nervous system.
In 1969, the first joint prosthesis registry was set up at the Mayo Clinic (in Rochester, Minnesota), United States. This was a local registry. The first nationwide knee arthroplasty was created in Sweden in 1975. Since then, orthopedic surgeons and related industries have set up multiple implant registries around the world. These registries have allowed postmarketing surveillance of new implantable medical devices and compared these to standard ones. The findings from the registry data have resulted in the discontinuation of inferior prostheses from the market. The implant failure during the postmarketing period seen in these registry studies would not have been detected by a randomized controlled trial (RCT) of conventional size and time frame.
In 1989, the New York State Cardiac Registries were set up. This was in response to the fivefold variation in hospital mortality rates for coronary bypass graft surgery in the state, which was not explained by the inadequate hospital-level information. This registry was established to assess the relative quality of care taking into account the interhospital variations in disease severity. Annual reports contained risk-adjusted data, which were site-specific. Therefore, the expected mortality for a given patient treated by a provider could be calculated as a function of his or her own risk factors. Such reports have led to service improvement initiatives including new chief appointments, dedicated anesthesiologists and intensive care beds, and changes involving clinical management of acute patients.
One of the most important surgical databases is the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The program originally derived from a congressional mandate in 1985 to report surgical outcomes at Veteran Affairs (VA) Hospitals, which were perceived to have higher complication rates than the private sector. During the 1990s, the National VA Surgical Risk Study published annual outcome reports, which resulted in a 40% decrease in postoperative morbidity and mortality over 15 years. In 2004, with the participation of private hospitals, ACS NSQIP was established and open enrollment began. One of the main outcomes collected was the 30-day complication rate. In one study, the proportion of patients with complications was 28% when calculated by NSQIP but 11% by an administrative claims database, suggesting that the registry was more comprehensive in data collection. Another study showed that the NSQIP participating hospitals have improved in surgical quality from 2005 to 2007. Although the database collects generic data across all surgical specialties, they have also generated useful neurosurgical research. So far, there have been over 20 publications related to neurosurgery. They are generally focused on complications following individual neurosurgical operations or particular complication types. Recently, a report examining postneurosurgery complications in 94,621 patients was published. The authors found a reduction of cranial postoperative complication between 2006 and 2013. In addition, they categorized the predictive risk factors, and examined the trend of decreasing ratio of cranial versus spinal neurosurgery during the time period.
Despite the success of NSQIP in improving outcome, the outcomes are limited in terms of significance to neurosurgery and in many cases are not as granular as most clinicians would prefer. For example, cerebrospinal fluid leak is not a routinely recorded complication. Also, the 30-day complication rate is too early for most neurosurgical diagnoses and procedures. In the past two decades, many registries have been established in various neurosurgical subspecialties. Most of them are national and cover most subspecialties including general, oncology, vascular, trauma, hydrocephalus, spine, and pediatric neurosurgery. Selected individual subspecialty registries are discussed below.
In 2012, the American Association of Neurological Surgeons (AANS) launched the National Neurosurgery Quality and Outcomes Database (N 2 QOD), now known as Quality and Outcomes Database (QOD). The database’s primary purpose is to “track the quality of surgical care for the most common neurosurgical procedures, as well as provide practice groups and hospitals with an immediate infrastructure for analyzing and reporting the quality of their neurosurgical care”. By allowing US neurosurgeons, practice groups, and hospital systems to contribute and access aggregate quality and outcomes data through a centralized, nationally coordinated registry, it may facilitate health care decision-making, which might lead to improvement in the quality, effectiveness, and value of health care delivery. The pilot project was a nationwide, prospective longitudinal lumbar spine registry utilizing patient reported outcome instruments which has been completed. The N 2 QOD was shown to be a robust platform for objective data collection.
How Do Registries Improve Outcomes?
Clinical registries may improve patient outcome in various different ways, as summarized below. Some of these effects overlap. In some areas, the quality improvement impact by registries is less clear. The Hydrocephalus Clinical Research Network (HCRN) is used as an example of a well-maintained registry with multiple outcome improvement projects.
Gathering Relevant Epidemiologic and Clinical Data
Well-designed and maintained multicenter registries gather a large amount of relevant clinical data. From these, important epidemiologic statistics such as incidence, prevalence, mortality, and survival can be calculated. The natural history of diseases can also be established. These are especially important for uncommon neurosurgical conditions. In addition to basic statistics, more detailed clinical information such as surgical outcome and quality of care and data related to social, economic, and environmental factors are also collected. Through sophisticated analyses, clinical and nonclinical risk factors can be identified. Furthermore, these data are longitudinal and so can be used to work out time trends of diseases and also make projections. For example, the NSQIP has found an increasing proportion of spinal versus cranial surgery performed by neurosurgeons. All of this information is important in decision-making in health economics and policies.
Setting Benchmarks and Standards
Some registries collect data from the full range of centers, while others do so from a selected number of institutes, for example, academic centers. For the former type group of registries, results can be used as national benchmarks to drive underperforming centers to improve outcome. For example, in response to the publications from the New York State Cardiac Registries, there have been hospital-specific initiatives to improve safety, outcome, and cost-effectiveness. These included a change in the departmental structure and clinical pathway. In other cases, how the publication of results leads to clinical outcome improvement is more elusive. For the latter type of registries, findings are based on data collected from a group of specialized centers. For example the HCRN, which at the time consisted of nine academic centers, has published outcomes of shunting insertion and endoscopic third ventriculostomy for 1184 and 336 patients, respectively. Although the results cannot be used as benchmarks as such, these data are useful for setting achievable standards for complication rates and treatment outcome.
Postmarketing Surveillance of Medical Devices
Surgical implant registries allow postmarketing surveillance, prompt identification, and thus recall of nonefficacious, failing, or even harmful medical devices. For some implants, such surveillance is required by the federal agencies. For a RCT to detect a revision rate of 30% above the mean (e.g., 6.5% instead of 5%) with a power of 80%, 4000 patients have to be followed up for 10 years. The delay is clearly too long. As mentioned before, owing to the implant registries in the orthopaedics, inferior new implants have been discontinued in practice. In 2000, Spine Tango, a European spine registry, was launched under the auspices of the Spine Society of Europe, with a similar objective in its mission.
Clinical Research
Registries lend themselves to be useful and powerful data sources for (a) generating hypotheses and strategies to justify RCTs and (b) providing risk-adjusted analysis of efficacy.
One example is the assessment of the new HCRN protocol to reduce cerebrospinal fluid shunt infection. It provides real-world evidence of no difference in the shunt infection rate between the use of a shunt surgery protocol that involves the use of antibiotic-impregnated catheters (AIC) and one that does not. Such findings together with the previous metaanalysis set the background for any future studies regarding AICs and shunt infection.
Clinical research can also be conducted directly from registry data, usually in the form of comparative effectiveness research. Well-designed registry-based research may have the advantages of collection of real-world longitudinal data, inclusion of patients otherwise not eligible for RCTs, large sample size (thus good statistical power), and higher generalizability. One example is the HCRN prospective cohort study including 1036 patients who underwent de novo shunt insertion, of which 344 experienced malfunction. The large sample size allowed multiple risk factor analysis and in fact cardiac comorbidity was identified as one. The Spine Tango registry showed that in atypical patients outside the spectrum of clinical trials, the clinical outcome of total disc arthroplasty is the same as anterior cervical interbody fusion. The comparison of registry-based research and RCT is discussed further in the next section.
Establishment of Guidelines
Analyses of well-designed registries can provide high-quality evidence for the development of guidelines. Dissemination and adherence to evidence-based guidelines lead to quality improvement.
Implementation of Guidelines Within Registries
Registries provide the ideal infrastructure to implement guidelines. Furthermore, the compliance to guidelines and their impact can be assessed. For example, Kestle et al. showed that the HCRN shunt surgery protocol reduced infection rate from 8.8% to 5.7%. Analyses of the results allowed refinement and simplification of the protocol which included the omission of several steps. The increasing use of AICs was recognized, and therefore this was added into the new protocol. Further longitudinal data confirmed no difference in infection rate between the old and new “bundles” (5.7% vs 6.0%). The development of the optimal guideline is obviously an evolving process, subject to constant reviews and based on up-to-date evidence.
Participation in Data Collection May Improve Outcome
Centers and surgeons participating in registries are more likely to be committed to quality improvement. There has been conflicting evidence whether participation in clinical trials improves patient outcome, and patient outcome improvement is a result of complex interactions between the patient, treatment, and the health care system. The potential mechanisms include improvement of infrastructure, processes of care, and other less obvious ones. Improvement of infrastructure may be in the form of early access to novel technologies, for example, surgical techniques, acquisition of new skills, improved multidisciplinary collaboration, additional training, education of personnel, and staff specialization. For example, following the introduction of a surgical technique for rectal cancer for a trial in the Netherlands, the structural surgical training and quality assurance had a lasting effect on patient survival even after the trial. Similarly, participation in a clinical trial has been shown to improve the professional knowledge, treatment strategy, and individual patient care. In terms of processes of care, some reports showed that research active institutes are more likely to incorporate new evidence into practice, follow guidelines, and optimize the patient pathway, but some showed the opposite. Other mechanisms are less obvious or more difficult to demonstrate. For example, while researches with negative results are often overlooked, they often have important cost implication and can lead to more efficient allocation of resources. Also, institutes with high academic output may attract and retain outstanding clinicians and scientists. Lastly, institutes with high research activity tend to have more collaborative projects with the commercial biotechnology and medical device sectors. This attracts revenue which may turn to expenditure in direct clinical care. Interestingly, there is also evidence that there is an association of research participation and improved outcome in general district hospitals as well as academic centers.
There are less data in regard to registry participation, but it is likely that many of the above mechanisms apply, especially when participation is noncompulsory. Hall et al. showed that there was surgical quality improvement, in terms of risk-adjusted morbidity and mortality, in hospitals participating in the ACS-NSQIP. Such realization should encourage institutes to engage and provide support for participating in registries at the ethical, logistic, and financial levels.
Related posts:
On Developing the Tools and Metrics for a Neurosurgical Quality Program
Evidence-Based Practice of Neurosurgery: Implications for Quality and Safety
The Physician Handoff and Its Role in Quality and Safety
Improving Performance by Improving Education
Quality of Care and Outcomes in Pediatric Neurosurgery: Incorporating Evidence-Based Practice
Use of Risk Model for Assessment of Resident Perception of Surgical Complexity and Evaluation of Procedural Competence
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
