A Site Network's Experience With eSource Before And During COVID
With 9 locations in Texas, California and Louisiana, Benchmark Research is the world’s leading vaccine network. Benchmark has performed over 1,000 studies on over 40,000 subjects, and has received numerous awards, including being a 5-time recipient of the Best Clinical Trial Site/Network award from the World Vaccine Congress.
Historically, Benchmark used paper charts to collect source data. In 2019, the management team concluded that they should move to electronic source but had not set a date for transition. When the pandemic hit in spring 2020, this desire became an urgency.
“We were wasting hours scanning and uploading documents for remote monitoring,” explained Deborah Copeland, the network’s Chief Operating Officer. “It was untenable and slowed us down. We needed to move to eSource.”
Benchmark put together a team of end users to review all the competitive options. The team selected an eSource vendor for its functionality and ease of use. The network rolled out eSource on a couple pilot studies, then determined that all new studies would occur on the eSource system.
“The biggest obstacle was fear,” explained Ms. Copeland. “You had coordinators who had been collecting source data on paper for 20 years. Fortunately, the eSource system we chose is very intuitive, so once the coordinators got used to it, they didn’t want to go back.”
The benefits of eSource
The network quickly realized several benefits from the transition. From an efficiency standpoint, it freed up significant space and made it easier to find and organize their study data. It facilitated PI oversight because it made the source data available at the PI’s fingertips at all times.
“All information is available at the click of a button, with a more transparent identification function, as opposed to PI needing to rifle through paper source in order to find the information they need,” said Charie Hynes, Clinical Research Coordinator.
It also made remote monitoring much easier. For site staff, it freed up considerable time from not having to scan and upload paper charts; instead, the site could simply invite the study monitors to log in and review the source data directly. For study monitors, it made direct access immediate and frictionless.
“Several monitors are thrilled that they have the information available to them at any time in real time,” said Ms. Hynes. “It has helped for coordinator questions as well because if there is a question about a specific subject or regulatory document the monitor can review the same information with them in real time.”
The biggest benefit to Benchmark Research was the gain in quality. Unlike paper charts, the system has built-in edit checks that enforce quality at the point of data collection. Study designers make use of the system’s flexible instructions, formatting, alerts and branching logic to prevent coordinators from entering incomplete or wrong information. The system also has built-in workflows to help sites keep visits within window, track required re-consents, and facilitate contemporaneous PI review and sign-off of source.
The network’s Quality Assurance department was able to quantify these gains. The team reviewed the number of protocol deviations in their eSource-based studies compared to their paper-based studies. They sampled all visits that occurred on all studies at the network’s three largest sites from 01 January 2022 through 22 March 2022. Because only new studies were put into eSource, these sites still had considerable legacy studies occurring on paper charts during this time, thus providing a control group. Since the same investigators and staff worked on both types of studies, the QA team is confident there is no effect from differences in staffing quality or training.
The QA team used, as their main endpoint, the number of site-generated and source-specific protocol deviations divided by the number of visits conducted. Site-generated deviations are protocol deviations attributable to site error, as opposed to patient noncompliance, which is beyond the site’s control. Source-specific deviations are deviations that stem from data collected in either the paper charts or the eSource system; examples of other deviations excluded are those stemming from lab collection or informed consents, which are done outside of both paper and eSource.
The team found a 38% reduction in the incidence of site-generated and source-specific protocol deviations, as measured by ratio of deviations to site visits. The following table illustrates the findings by site and in total:
A nearly 40% reduction in protocol deviations would be a significant win for the industry as a whole. It represents a material improvement in patient safety and data quality. It also creates significant time savings because it reduces the need to develop and maintain corrective action plans, time that can be re-invested in recruiting and processing more patients.
The final frontier: Eliminating secondary data entry and SDV
For Benchmark, the next leap for both sites and sponsors is to make eSource the eCRF data, either through use of an eSource system that has a fully integrated EDC platform, or through an ODM-compatible API to third party EDC systems. This would eliminate secondary data entry into the EDC and the attendant clean-up from initial data entry errors.
Data entry is a significant drag on site efficiency. Benchmark staffs at least 1 full-time data entry specialist for every 9 coordinators who collect data, implying a 10-12% workload inefficiency for Benchmark. For smaller sites, who lack Benchmark’s scale efficiencies, the inefficiency is likely even higher, perhaps in the 15-20% range.
What would it mean if sponsors could eliminate secondary data entry for the sites on their trials? For Benchmark, Ms. Copeland explains that it would mean they could see more patients with the same overall level of resources, meaning they could achieve their enrollment target faster. For other sites, where coordinators play a role in recruiting, it means more time could be devoted to recruitment. Either way, sponsors would see faster enrollment, improved data quality and greater site engagement.
“For many sites, EDC entry is very difficult, and any sponsor that could eliminate that would become the sponsor of choice for so many sites,” said Ms. Copeland.
How sponsors should execute eSource
Benchmark has some advice for how sponsors can utilize eSource on their trials.
“First, design the templates in a way that works for sites,” explains Tara Pratt, the network’s Technology Manager who designs most of the eSource templates. That means tailoring the template to be a real-time workflow tool for sites, while still capturing the critical eCRF fields required by the sponsor. Ms. Pratt has designed eSource on over 40 studies and has never had a problem designing them in a way to capture sponsor-required eCRF fields.
Second, Ms. Copeland does not anticipate sites will resist or struggle with the technology, especially since the sponsor would be furnishing the templates. “Our eSource system is so intuitive, and so obviously built for sites that even our least technically savvy coordinators could use it,” explained Ms. Copeland.
Third, sponsors can take advantage of the site configurability feature. In our eSource, the centrally designed source templates cannot be modified by receiving sites; however, sites may add their own custom visits, forms or questions on top of the main template. This ensures each site can configure the template to reflect their specific workflows.
“Site configurability has been critical to adoption within Benchmark,” explained Ms. Pratt. “Even though we’re one network, individual sites may have slightly different ways of doing things, so we find that there are small differences across the site templates on the same study.”
The biggest reason to believe in site adoption, of course, is that the system costs would be free to the site and the sites would not have to do data entry. “Any system that is user friendly and that eliminates EDC entry would be adopted by the vast majority of sites in a heartbeat,” said Ms. Copeland. “It’s such an obvious benefit to sites.”