Introduction

Neuromodulation, defined by the International Neuromodulation Society as “the alteration of nerve activity through targeted delivery of a stimulus, such as electrical stimulation or chemical agents, to specific neurological sites in the body” ( ), has been in use for thousands of years ( ). The pain-relieving effect of stimulation delivered via electric fish was a serendipitous finding, based on anecdotal experience that was passed along verbally and then engraved onto stone tablets. As artificial means of generating and controlling electricity were developed, the use of stimulation for a host of maladies, still based on anecdotal evidence and testimonials, was publicized widely, thanks to the invention of the printing press. Justifiable skepticism accompanied enthusiastic marketing, but just as little good evidence existed to support extravagant claims, there was little good evidence to refute them.

Publication of the “gate theory” of pain transmission 50 years ago provided a rationale for the use of electrical stimulation for the treatment of pain ( ), and concurrent development of solid-state electronics made implanted electronic devices feasible. Spinal cord stimulation (SCS) was introduced in this context in the 1960s, supported by published case reports and small case series ( ), and in a wave of technological zealotry this form of stimulation became the most common application of neuromodulation. By the 1990s, the published scientific literature included reports of case series with hundreds of patients accumulated over two decades with favorable results ( ).

Today, the standards for clinical studies demand much more than case series. Indeed, in the current era of evidence-based medicine (EBM), clinical studies are expected to adhere to certain standards. Entire textbooks and university courses are devoted to this subject, about which this chapter will set forth some general principles, along with special considerations for the design of neuromodulation studies.

Study Design: General

Clinical study design should support the inference that treatment and response are, in fact, cause and effect. Mere observation of an association, such as relief of pain by exposure to electricity, does not demonstrate a causal relationship but can generate a hypothesis amenable to testing by clinical trial. A preliminary trial might take the form of a consecutive case series, which can yield useful data on safety and efficacy but can logically demonstrate only associations, not cause and effect. Observed effects might be nonspecific (placebo) or might simply result from regression to the mean ( ). Thus, a study design comparing treatment with control, in parallel groups, is more convincing, that is, of higher quality than observational studies. Early controlled trials relied on alternate allocation of subjects to treatment groups; potential sources of bias using this method, however, were recognized and addressed, first by randomization and then by blinding study subjects and investigators ( ).

Fig. 4.1 shows a typical contemporary randomized controlled trial (RCT) design in the form recommended by the Consolidated Standards of Reporting Trials (CONSORT) group, as developed since the 1990s ( ). The CONSORT checklist includes “25 items selected because empirical evidence indicates that not reporting the information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings” ( ). This chapter highlights those items with particular relevance to neuromodulation.

The Consolidated Standards of Reporting Trials (CONSORT) group recommends this form for a typical randomized, controlled trial design.

Reproduced with permission from CONSORT Group 2017, accessed at: http://www.consort-statement.org/ .

Fig. 4.2 presents the flow chart of a representative RCT ( ), which, although begun before publication of the CONSORT statement, is nonetheless in general conformity with its guidelines. Sample sizes and patient flow as the study progressed are shown plainly within the diagram. An obvious feature of this study, superimposed on basic RCT design, is the inclusion of crossover. In this study, crossover was optional; patients who were not satisfied with the results of their randomized treatment assignment were allowed, after 6 months, to cross over to the alternative, and this decision on their part was considered one of the study outcome measures. This study compared a neuromodulation procedure with a surgical alternative; in general, however, neuromodulation studies lend themselves to crossover trials, comparing one neuromodulation protocol with another delivered via the same device, without any need for additional procedures apart from programming changes. Crossover studies in general can increase statistical power for a given sample size and address some ethical concerns, while potentially raising others ( ).

A representative RCT, begun before CONSORT was reported but diagrammed in general conformity with its guidelines, showing sample sizes and patient flow as the study progressed. Crossover design was an important feature of this study.

Reproduced with permission from North, R.B., Kidd, D.H., Farrokhi, F., Piantadosi, S., 2005. Spinal cord stimulation versus repeated lumbosacral spine surgery for chronic pain: a randomized, controlled trial. Neurosurgery 56, 98–106.

Many choices of comparator or control are possible: first and most fundamental is incorporating a placebo to show that treatment effects are more than nonspecific. The use of placebo as a treatment arm can raise ethical concerns ( ), but such concerns are substantially addressed by a neuromodulation study in which all patients are implanted and placebo therapy is delivered via a mere programming change. As will be detailed here, however, the usefulness of placebo control diminishes in forms of neuromodulation, producing paresthesia, where blinding is impossible.

Second, an active comparator, such as “optimal medical management” or conventional medical management (CMM) ( ), is commonly used as a control for “comparative efficacy studies.” A comparator that simply continues a previously failed therapy, albeit with that treatment “optimized,” is subject to criticism, even on ethical grounds, but ethical concerns can be addressed substantially by crossover. A comparator that is more invasive and potentially more morbid (e.g., reoperation) than the treatment in the other study arm might also be criticized, except when it is the prevailing norm and the effectiveness of the neuromodulation procedure as an alternative therapy remains to be proved. In general, comparative efficacy studies that include treatments in common use are particularly relevant clinically; they help to address risk-benefit and cost issues. On the other hand, some studies of novel waveforms have used traditional/conventional waveforms as comparators to achieve regulatory objectives; as will be detailed here, the implications of these studies for clinical practice are complex, particularly if the same device can be used to deliver more than one waveform.

Study Population

A clinical study is only as meaningful as its design, and the composition of a study population is of particular importance, as this determines external validity (the population to which the study results may be generalized). Criteria for patient inclusion and exclusion may be broad; for example, to support a regulatory goal for SCS, a study might simply include subjects with intractable pain of the trunk and limbs, consistent with current labeling requirements. On the other hand, to address a specific clinical question, selection criteria might be narrow; for example, patients with persistent leg (not exceeded by low back) pain following one or more lumbosacral spine surgeries, now candidates for another surgery, with a confirmatory second opinion. Many nuances exist for selection of patients with specific conditions, for example, in the case of pain, minimum pain intensity, pain rating variability, or lack thereof on serial baseline ratings ( ). Details of these fine points are beyond the scope of this chapter. In general, we should not presume that study results may be generalized or extrapolated beyond the study population or design, but in clinical practice this is often done, especially if the ratio of potential benefit to risk is favorable. High-quality clinical studies simply cannot be performed to address all possible clinical scenarios.

Implantation and adjustment of neuromodulation devices involve technical skill, which is important in clinical practice ( ) and must be considered in clinical study design and interpretation. Favorable results of a study by the inventors and developers of a new procedure, demonstrating its safety and efficacy, cannot be applied to the general population as readily as, for example, a pharmacologic trial that is not dependent on procedural skill ( ). Even multicenter trials, which are assumed to be more representative of general clinical practice than single-center trials, are not necessarily so.

Study Endpoints

The distinction between desired outcomes or goals of treatment and the measures and tools we use to represent these goals is important. For example, pain relief has been the most common goal of some neuromodulation procedures during the past five decades, and indeed during the past 5000 years, but we are unable to measure pain or its relief directly and objectively. Rather, we rely on patient reports and subjective ratings. The lack of an objective, direct measurement tool has always been a problem for the validity of pain studies.

In practice, a major advantage arises when studies measure pain with a numeric pain rating scale (NPRS, to be distinguished from a visual analog scale, or VAS, with which it is often confused)—namely, that results can be communicated via phone. Thus, the need for in-person follow-up or reliance on the vagaries of mail delivery or electronic media, as required by VAS results marked on paper, need not lead to loss of subjects. Logistic considerations such as these can be important when choosing study endpoints.

Applications of neuromodulation beyond pain (e.g., epilepsy, movement disorders, and bladder dysfunction) have primary manifestations that are amenable to objective measurement. Some measures can be collected directly by neuromodulation devices; for example, accelerometers can record patient activity as an indirect measure of pain relief ( ).

Indirect measures of success can be global, such as patient satisfaction (“Would you undergo this treatment again for the same result?”), perhaps a sine qua non for claims of success but not a sufficient sole criterion. Patient global impression of change (PGIC) is another common indirect measure used in pain studies, although its validity as a measure of pain, per se, is limited ( ). Common practice dictates use of at least one direct measure of primary disease manifestation and at least one indirect global measure of health-related quality of life in reporting study outcomes ( ).

Cost is among the many outcome measures that may be studied and is of obvious practical importance. In the field of neuromodulation, the significant upfront device costs add to the expense of clinical trials. Clinical trials in general have become expensive, due substantially to regulatory and institutional review board compliance costs, and major sources of funding, such as the National Institutes of Health in the United States, generally have not been accessible to neuromodulation researchers, who have been advised to turn to industry for support. Accordingly, most neuromodulation studies, like most pharmaceutical studies, have been sponsored by industry.

Study endpoints can address not only clinical benefits but also adverse events; in fact, events such as neurologic injury complicating SCS can be a primary study endpoint ( ). Sample size requirements can prevent researchers from conducting high-quality (prospective, randomized, etc.) studies of uncommon but clinically important complications; thus, studies based on large public health databases can make important contributions to the field.

Surrogate outcome measures are exemplified, in the field of neuromodulation, by concurrence of stimulation paresthesia with areas of pain, which was (and is) an important and necessary condition for pain relief with traditional/conventional SCS. Optimization of this measure, facilitated by computerized, patient-interactive equipment, has been the subject of a number of RCTs (e.g., ). As detailed in Chapter 50, optimizing electrode placement remains technically important but is not clinically important, in and of itself; it certainly is not synonymous with relief of pain, which is the actual goal of treatment.

Achieving a surrogate endpoint within a trial is often an important part of overall clinical study design: a common, almost trivial, example is the use of a screening trial of the treatment under study, such as testing SCS with a temporary electrode or an intrathecal medication with a temporary catheter to identify candidates for a definitive study procedure. Thus a short-term primary endpoint result is used to select a study population in which the same result will be assessed long-term. Initial SCS series for which only the patients receiving permanent implants were reported exemplify this practice. This “enriches” the study population and, thus, can and should be considered part of the patient selection process.

Statistical Considerations

How do we infer results from an outcome data set? Traditionally, in a comparative trial, we seek to reject the null hypothesis – that no difference exists between treatment groups – by demonstrating a statistically significant (and potentially clinically significant) difference between the groups. Detailed discussion of statistical methods is beyond the scope of this chapter, but let us consider the following issues, pertinent to neuromodulation:

A “responder” to a given treatment may be defined by some minimum change in an outcome measure, and this might be readily understood by clinicians. “Responder analysis” has been recommended as a routine part of clinical reporting ( ), and researchers have made efforts to define “minimal clinically important change” for certain disease states, such as low back pain ( ). The initiative that led to the IMMPACT (Methods, Measurement, and Pain Assessment in Clinical Trials) recommendations went so far as to suggest that a responder analysis can simplify sample size planning ( ). For a given sample size, however, reducing continuous data to a simple dichotomy as part of an a priori study plan substantially reduces potential statistical power ( ). A normally distributed continuous variable, such as a change in pain scores (whether expressed as an absolute change or a percentage in NPRS) or in Parkinson’s disease scores is amenable to simple parametric statistical tests, and if the goal of a study is to reject the null hypothesis, use of a normally distributed continuous variable will reduce the required sample size. Identification of “responders” or “remitters” may then be undertaken post hoc and reported as such ( ).

A clinical study, designed to reject the null hypothesis, is not appropriate to all circumstances, however. Noninferiority designs have been used in two noteworthy recent neuromodulation trials, both of which were “pivotal” trials that used previously approved devices as comparators and were conducted to achieve regulatory (US Food and Drug Administration) approval ( ). From the standpoint of public health as well as the principle of equal treatment under the law in the competitive marketplace, it makes sense to allow new therapies to come to market if their risks and benefits are (at a minimum, and within statistical limits) comparable (or superior) to previously approved therapies.

Some researchers believe that noninferiority trials might be unethical because they disregard patients’ interests ( ). This follows from the possibility that a lower success rate and/or a higher rate of complications may be found to be noninferior, within statistical limits, particularly if sample sizes are small, outcome measures are few, and follow-up is limited ( ). This caveat is particularly relevant if the treatments compared in a study are mutually exclusive, such as one drug versus another. In the field of neuromodulation, however, such concerns might be mitigated if the new treatment is simply a new waveform generated by the same device as the comparator waveform.

An example of such a comparison involves an SCS pulse generator that delivers not only conventional waveforms (monotonic, tens to hundreds of pulses per second) but also bursts of several pulses at higher frequency, delivered at a lower frequency ( ). By comparison with previously approved devices, the major potential clinical benefit of the new device is that it will allow patients who do not respond satisfactorily to “traditional/conventional” stimulation to try burst stimulation, with nothing more than a programming change, using the same generator and electrodes. The pivotal study, performed to gain regulatory approval, did not address this directly; rather, a population of SCS candidates selected on the basis of a trial with the old waveform for implantation with the new generator was randomized to initial treatment with one waveform or the other. The primary outcome of this noninferiority trial was the comparative efficacy of the two waveforms after 6 months. Crossover was allowed after 6 months, but crossover results were only a secondary outcome. Fortunately, the results of the study were positive ( ), and regulatory approval was secured; thus, patients can use either waveform going forward.

What if, however, the results of such a study had been negative, that is, if the new waveform proved to be inferior to the old but provided added benefit in patients for whom the old waveform was ineffective? On the other hand, what would happen if a distinct subset of patients preferred and/or achieved superior pain relief with the old waveform? In the aforementioned “burst” study, 20% of patients did prefer the “conventional” old waveform. Fortunately, the study device delivers both waveforms. Were this not the case, however, a mismatch between study design and clinical need would have arisen.

Conventional SCS is successful overall in about 50% to 60% of patients ( ), and in this context a new waveform that adds incrementally to these figures would be welcomed by clinicians and patients alike, even if it failed a head-to-head noninferiority comparison. Consider by way of analogy a new antibiotic for resistant strains of bacteria, applicable to a minority of patients. This would undoubtedly be valuable, even if it failed by comparison with traditional first-line therapy. Sensitivity testing is easily done in vitro for antibiotics, whereas, for SCS, such testing requires implantation of a device in each patient. Thus, a single device that delivers multiple treatments is advantageous.

Is it feasible to study only patients who are refractory to conventional/traditional SCS, and can we conduct an RCT? Long-term failures are sufficiently common that adequate samples should be available, but the devices presently in clinical use have supported novel waveforms only in limited fashion, such as subperception 1-kHz stimulation ( ), and 5-kHz “high-frequency” stimulation ( ), in both cases below the 10-kHz most often reported in recent studies of high-frequency SCS. New devices will be better suited to support such studies.

Neuromodulation raises study questions as well as study opportunities that do not arise with other treatments, such as surgical procedures, for which one operation might be compared with another but for which the study alternatives are mutually exclusive because one irreversibly changes the substrate for the other. As a reversible therapy amenable to therapeutic trial, neuromodulation has more in common with drugs from the standpoint of study design. Neuromodulation also typically involve a minimally invasive procedure, following which multiple adjustments to treatment parameters (stimulation waveform and electrode location for a stimulator, drug dosage for a pump) are possible without further procedures.