Fig. 13.1
Flowchart showing the three major phases of a study and the involvement of the RC in each of these phases
During the execution phase, the RC supports several vital aspects of the study. It provides server infrastructure and a secure upload portal for the acquired data. Once the data acquisition and upload procedure is documented and provided to the participating sites, it supports the investigator with dedicated training on the image acquisition as required, via learning materials, teleconferencing, or on-site training. Operators usually need to undergo a certification procedure, in order to ensure that they understand the scan protocol and are capable of acquiring images of sufficient quality. Typically, as part of the certification process, a healthy volunteer (e.g., a coworker at the study site) is scanned (in longitudinal studies, two consecutive certification scans may be used in order to assess repeatability) and the data uploaded to the RC server for assessment. The RC performs a full quality check (QC) on the data and, if the scan is of sufficient quality, issues a certificate showing that the operator is allowed to scan patients as part of the study. Certification is specific to the member of staff in question and does not cover other workers at the same center. If an operator submits a scan of insufficient quality, the certification procedure must be repeated until successful. Operators may only scan patients in the study after being certified. The procedure to be followed with patients is similar to that during certification, with the operator performing the scans and then uploading the data to the RC and the data being accepted or rejected following QC. If a scan performed on a patient is rejected, it is highly desirable to provide feedback to the operator, listing the grounds for rejection. The sponsor and/or study administrators (if applicable) should also be informed of any rejected scans. If the scan is accepted, the data is stored for subsequent analysis.
The third phase is typically time consuming and work intensive for the RC. At EOS, the complete set of images received during the study must undergo post-processing (e.g., segmentation of multiple retinal layers). The required outcome measures for each patient are listed on a worksheet and may then be forwarded to the sponsor for statistical analysis or, alternatively, analyzed at the RC.
Challenges of Image Analysis
Image analysis and quantification should ideally be free from systematic errors (accurate) and random errors (precise) and should yield the same results if performed multiple times (repeatable). Since these tasks are performed by human analysts (“graders”), errors are inevitable but must be managed and should be minimized. While random errors are limited in scope and relatively easy to control, systematic effects/errors (“bias”) may lead to misleading results. Bias may be caused by a multitude of factors.
Sources of Bias
In order to be clinically efficient, medical professionals are trained to collect and synthesize information about a patient from many sources “on the fly” from the very first glance at the patient, noting their posture, movements, fixation, speed and ability to navigate, and general appearance even before performing a full (ophthalmic) examination (from the lids all the way back to the retina). The sum of these observations will steer the clinician’s attention toward potential diagnoses and prompt them to perform specific further tests of structure (imaging) and function (e.g., perimetry, electrophysiology). While this approach is essential in the clinical setting, it may be a source of bias and thus out of place in the context of systematic image analysis. If image analysis is performed by persons with access to clinical information, their interpretation may be influenced by factors other than the data shown in the images (clinical bias). Bias may also be introduced by differences in training or personal opinions at different sites within multicenter studies. It is also part of human nature that investigators with a vested interest—academic, personal, or other (including financial)—may be biased, either unconsciously or consciously, toward a certain outcome of a study. To attain and maintain precision, accuracy, and repeatability over time and between graders, and in order to minimize bias, image analysis requires objectivity and clearly set criteria.
Advantages of Using a Dedicated Reading Center
RCs aim—through their organizational structure and procedures—to eliminate bias in data assessment while still utilizing the unparalleled pattern recognition abilities of the human brain, as well as computerized analysis tools that make grading easier and quicker. RCs as organizations are independent. Their quality hallmarks are precision, accuracy, repeatability, reliability, and, above all, objectivity, irrespective of the actual findings or final outcome of any specific study. Images are analyzed according to predefined criteria and procedures by consistently trained graders, who only report to the RC and must not have any vested interest of any kind in any specific study they are involved in. Graders are trained to grade only what they can actually see in the specific image being graded. No clinical inferences are made at the point of grading, and graders are completely masked to clinical information, other graders’ data, and typically even their own grading data from previous scans. Graders are geared toward high-quality, rather than high-speed, image analysis. Predefined grading procedures include strategies for quality assurance (multiple primary graders, adjudication by expert clinicians, regular assessment of inter- and intra-grader variability) and also ensure compliance with regulatory standards [8, 9].
Roles of a Reading Center
Study Development
Ideally, the role of an RC begins long before study initiation. RCs can collaborate with the research group or sponsor in developing a study protocol. Once a research question has been formulated and a structural characteristic (or “sign”) of interest identified, appropriate outcome measures need to be found. RCs can help assess which imaging modality is likely to best reflect the structure in question and which equipment and imaging/scan parameters may be suitable in terms of sensitivity, specificity, and repeatability while remaining practically feasible in terms of patient and operator acceptance. For example, high-density raster volume scans may be superior to circular scans in terms of data consistency; however, in patients with neurological disorders and in the presence of tremor, nystagmus, etc., these may not be practicably attainable. RCs can also help develop a grading system, selecting which morphological descriptors to quantify on which scale (continuous, interval, or categorical). Descriptors may include a wide range of morphological characteristics, including 2D/3D size (diameter, thickness, area, volume), shape (e.g., convexity, roundness), signal intensity, structure, texture, or location. Grading turns structural characteristics into meaningful quantitative or qualitative data that can be used in statistical analyses and serve as a basis for clinical decisions. RCs typically have a thorough knowledge of methods used in previous studies. If precedents can be found and applied to the current problem, established outcome measures (based on evidence rather than opinion) can be used; if not, then new structural (and/or functional) outcome measures need to be developed and evaluated. If necessary, RCs can conduct pilot studies, testing new methods on smaller samples under controlled conditions.
Once the methods have been finalized, the RC can define participant eligibility criteria (e.g., minimum/maximum lesion size, optical media transparency, etc.), which will help standardize patient recruitment. To standardize methods all across the study (at multiple national and/or international clinical study sites), protocols are developed for image acquisition (visit timing; equipment criteria; appropriate scan parameters; scan quality criteria; image/data formats, e.g., TIFF, PNG, DICOM, or proprietary, e.g., E2E) and for image/data management (image anonymization, submission to the RC, tracking). To maintain data uniformity, it is essential that all clinical sites comply fully with the study-specific protocols. Likewise, for use within the RC, a protocol is developed for image analysis (grading standard operating procedure [SOP] or independent review charter, grading forms, paper or electronic data entry) and for how the study is to be managed at the RC (study-specific manual of operations). This deals with issues such as quality assurance, grading data management (NB grading data are managed without grader read access and are held separately from clinical data until all grading is concluded and the database is locked), and data analysis.
The RC may help to ensure that the equipment at each participating clinical site is compliant with the requirements documented in the imaging protocol. The RC may also conduct study-specific clinical site photographer training and certification. Duties at clinical sites are shared between site staff, including the principal investigator, study coordinators, and photographers. The RC provides training for these roles as needed. The PI must be trained in order to know what is expected of their staff at their site as well as the necessary administrative routines for managing a site. Study-specific training and certification may also apply to graders at the RC, according to the grading and data entry protocols.
Study Management (Operational Logistics)
Study Initiation
Once the appropriate study-specific protocols have been developed and agreed on, approvals from relevant regulatory bodies collected, and sites approved in terms of equipment and personnel, the study can be initiated.
Patient Recruitment
Clinical sites identify candidates (according to the clinical inclusion/exclusion criteria, e.g., diagnosis, disease severity, age, comorbidities, previous therapies, etc.), perform the imaging as required by the imaging SOP, and send the (anonymized!) image packages to the RC. The RC may assess participant eligibility based on the predefined imaging criteria. When necessary, expert clinicians are asked to arbitrate questionable or borderline cases. Once the decision whether the patient is eligible or not is made, the site is informed.
Follow-up Phase
Once the baseline image sets are collected, during the follow-up phase of longitudinal studies, the RC may manage image data collection, i.e., ensure that clinical sites obtain and submit image sets of suitable quality within the time frames required by the SOP. The RC provides feedback on protocol compliance and provides assistance to on-site clinical staff with protocol interpretation when necessary. The RC also makes sure that protocols are kept up to date and made available to on-site clinical staff. The RC manages the submitted images, including data storage, inventory, and tracking. Approved image sets are graded at the RC according to the grading protocol by trained and certified graders. Grading performance is assessed regularly. Grading data are managed independently from graders and held separately from clinical data.
Quality Assurance
Study Termination
The RC also helps develop protocols for what happens once the projected sample size and duration of the study are reached, including patient end-of-study (EOS) procedures, clinical site staff EOS procedure training, and monitoring clinical staff protocol adherence. Once the last patient has been seen and the last image has been graded, all data collected and graded by the RC will be transferred to the sponsor. The RC may participate in data analysis and reporting in scientific journals or at meetings. The RC also manages image and data archival following study termination.
Related posts:
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
