White Paper

Optimizing Rare Disease Outcomes Through eCOA Technology And Training

By Susan M. Dallabrida, Ph.D.

Optimizing Rare Disease Outcomes Through eCOA Technology And Training

The stakes are high in rare disease research. Significant unknowns, a limited and geographically dispersed patient population, and challenges regarding patient/caregiver data collection pose different pressures for clinical researchers. But, sponsors need not fret. There are proven tools ─ including effective data capture strategies ─ to overcome these challenges and accelerate rare disease research. 

Recognizing Rare Disease Drug Development Challenges

In some contexts, the term ‘rare disease’ may be misleading. Although individual rare diseases are truly rare (each affecting <200,000 persons in the US1 or <1 person per 2,000 people in Europe2), collectively, rare diseases affect more patients (350 million) than those with HIV or cancer combined. Even worse, 95% of rare diseases have no approved treatment or cure, highlighting the importance of rare disease clinical research.

Fast-track reviews within FDA and EMA are in place to accelerate breakthrough therapies, but many researchers have difficulty in finding and enrolling enough patients and securing assessments that collect relevant, valid and reliable information about patients’ experiences with their condition and the treatment under consideration.3 Common challenges stem from:

Heterogeneity in Disease

The variations within the specific, rare disease population complicate outcome measure selection and diagnosis.

Heterogeneity in Disease Progression

Many rare disease patients are enrolled at different disease stages. There are challenges to understand each patient’s disease experience and outcomes, since many are unique to the individual and their respective disease progression.

Patient Demographics / Age

Because half of patients with rare diseases in the world are children,4 clinical trials in these diseases require parents, teachers and other caregivers to complete outcome assessments based on their observations, which are subject to variability, bias and inconsistencies that can give an inaccurate representation of the treatment effect. This increases the burden on data managers to determine in advance which outcome assessments will most accurately evaluate a treatment’s effectiveness.

Developing Effective Outcome Measurement Strategies

Clinical outcome assessments (COAs) are considered by the FDA to include “any assessment that may be influenced by human choices, judgment, or motivation and may support either direct or indirect evidence of treatment benefit5.” While treatment efficacy can be measured in many ways, COAs in clinical trials must represent meaningful outcomes for patients regarding the effect of their condition and how the treatment makes them feel, function or survive.

However, selecting and validating COAs for rare diseases can be particularly challenging, often because there are no new or condition-specific validated outcomes instruments (Table 1). And, because disease presentation, course and response to treatment within each disease vary greatly, it can be difficult to recruit a sufficient sample within the small patient populations for initial COA instrument development and validation studies as well as for later data collection.

Sponsors also need to consider that the common age group (especially for children) and presence of physical and/or cognitive disability within rare disease patient populations may warrant innovative modes of administration and alternative forms of COA such as Observer-Reported Outcomes (ObsRO) and/or Performance Outcomes (PerfO) (Table 1).

Table 1. Types of clinical outcome assessments that can be collected in clinical trials

Improving COA Data Quality in Rare Disease Trials

Quality data are essential to ensuring effective and appropriate measurement of treatment outcomes. This becomes even more important in rare disease trials, where the limited availability of eligible clinical trial patients could affect a study’s statistical power. Every patient matters, and the quality of the data that they provide is of the utmost importance.

However, collecting high-quality data in rare disease clinical trials can be difficult for many reasons. Missing data and nonstandard data collection are the largest threats to data quality. Additionally, inter-rater variability and variability in the rater’s understanding of the assessment contribute to poor-quality data. This is particularly true because any rater bias, including the halo effect, stereotyping, perception differences, leniency/stringency, and scale shrinking is exacerbated ― and can be particularly damaging to the quality of rare disease data collected.

Because of its proven ability to reduce missing data, increase event reporting, and improve accuracy in reporting, electronic clinical outcome assessment (eCOA) is the most effective method for capturing validated, consistent assessments to measure rare disease trial outcomes.

eCOA systems are preferred over traditional paper-based methods by clinical trial patients7 and utilize alarms as reminders for patients to complete assessments; this consistently drives more complete data collection than paper COA (Figure 1).  In addition, eCOA systems are typically built with system checks that reduce missing data, delivering more valid data for the trial sponsor’s evaluation.

Overall, these features improve data quality and ultimately, the statistical power of rare disease studies. As a result, fewer patients are required to determine a treatment’s efficacy and safety, enabling trial sponsors to reach go/no-go decisions faster.

Additionally, most eCOA systems upload data to centralized databases, providing near real-time access to data and the ability to monitor patient safety and compliance with the treatment and protocol. With each improvement in data quality and the ability to monitor data in near real-time, eCOA systems support risk mitigation strategies in rare disease trials and deliver reliable, valid data ― both of which reduce the time needed for database lock, keeping studies on time and on budget.

Moreover, data collected through eCOA are accepted by regulatory bodies. Data meet the ALCOA (Attributable, Legible, Contemporaneous, Original, Accurate) standard in the FDA Guidance for Industry Computerized Systems Used in Clinical Investigationsand the EMA reflection paper on the expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials, which not only include ALCOA, but expand on it to include the attributes of: Complete, Consistent, Enduring and Available when needed7.

Ensuring Consistent Data Collection in Rare Disease Studies

Uniform administration of assessments in clinical trials is required to reduce rater variability and minimize data risk with COA implementations. Furthermore, effective assessment training remains a key determinant of whether a therapy attains efficacy and/or safety. In fact, the FDA, EMA and International Society for Pharmacoeconomics and Outcomes Research (ISPOR) recommend that clinician raters, patients and caregivers capturing assessment data receive training in the correct use of the instrument and of the electronic data capture element.

Training site raters and subject/caregivers is vitally important in rare disease studies because inter-rater variability is high due to the widespread geographic dispersity of sites and subjects.  In addition, rare disease research often requires event and severity reporting by caregivers, who do not inherently understand how to complete assessments.  The role of an observer – a neutral reporter, who reports only on what they actually observe – is very different from the role of a caregiver.  Caregivers can spend in excess of 40 hours/week providing care and have substantial physical and emotional tolls that often cause their stress level to be 20-30% higher than the patients in their care.  Training caregivers on how to report on an ObsRO is a critical element to the success of data capture in rare disease studies. 

Training subjects and caregivers also promotes compliance. More than 75% of patients report that the number one factor that would motivate them to complete a daily diary in a clinical trial is training on the importance of their role, what to expect in the study, and the purpose and importance of the questionnaires8.  The FDA PRO Guidance cites training of site raters, patients and caregivers as the leading factor that is imperative to collecting the highest possible quality data in a clinical trial and the further complications that occur in rare disease essentially necessitate such a course of action9.


Successfully developing new rare disease treatments doesn’t have to be rare. Clinical trial sponsors who incorporate effective data capture strategies to optimize outcome measurement reduce clinical data risks, improve data quality and strengthen their risk management initiatives through near real-time data access, all of which produces a more accurate picture of treatment benefit and keeps development plans on track and on budget. Utilizing electronic data capture systems and training site raters, subjects and caregivers are important tools to ensure high quality data capture.

  1. US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER). Rare diseases: common issues in drug development: guidance for industry. August 2015.
  2. European Commission on Public Health. Rare diseases. https://ec.europa.eu/health/rare_diseases/policy_en Accessed August 31, 2017.
  3. Benjamin K, Vernon MK, Patrick DL, Perfetto E, Nestler-Parr S, Burke L. 2017. Patient-reported outcome and observer-reported outcome assessment in rare disease clinical trials: an ISPOR COA emerging good practices task force report. Value in Health 20: 838-55.
  4. (n.d.). Retrieved September 5, 2017, from https://globalgenes.org/rare-diseases-facts-statistics/
  5. Clinical Outcome Assessment (COA): Glossary of Terms. (n.d.). Retrieved September 5, 2017, from https://www.fda.gov/drugs/developmentapprovalprocess/drugdevelopmenttoolsqualificationprogram/ucm370262.htm
  6. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Biologic Evaluation and Research (CBER), Center for Drug Evaluation and Research (CDER), Center for Devices and Radiological Health (CDRH), Center for Food Safety and Nutrition (CFSAN), Center for Veterinary Medicine (CVM), Office of Regulatory Affairs (ORA). Guidance for industry – computerized systems used in clinical trials. April 1999.
  7. http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2010/08/WC500095754.pdf
  8. Data on File, ERT
  9. https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf

As seen in Applied Clinical Trials, October 2017