Article | April 12, 2017

Moving From Paper To Mobile: Validation Of Mobile Electronic PRO/COA

By Chris Watson, Ph.D., Director of Product Strategy, Digital Patient

Moving From Paper To Mobile: Validation Of Mobile ePRO

Patient Reported Outcomes research (PRO) is indispensable in demonstrating the clinical efficiency of any medical treatment or procedure. This type of research, historically conducted using pencil and paper, employs clinically validated outcomes-tools in order to accumulate accurate and dependable patient endpoint data.   

Despite advances in technology, patient endpoint data is often collected using this archaic, and arguably inefficient, paper-based means. Patients are required to complete a questionnaire or consent to an interview with a member of clinical staff, who will then manually transcribe the data for processing. Common questions posed in these situations will concern the patient’s general health and any specific symptoms or impairments they may have experienced. This manual, paper-based process represents a drain on resources. As such, these traditional methods of data collection are cost- and time-inefficient, and, in some cases, unreliable.

Using technological advances, particularly the increasing availability and popularity of web-enabled mobile devices, the industry is beginning to change the way it thinks about validated clinical research. The sophistication of these devices, and their familiarity to patients, means that time and resources can be utilised more efficiently, especially in such cases where the mobile device in question has internet connectivity (although web access is not necessarily a stumbling block if the device has the capability to function on- and offline.) In the arena of clinical outcomes research, the snowballing use of such technologies has sparked a timely debate about the relative efficiency of traditional pencil and paper-based methods compared to more modern, electronic modes of data collection. The technologies in question are becoming increasingly integral to people’s everyday lives, and, because of the range of possibilities this implies – ease of accessibility, cost- and time-efficiency – web-enabled mobile devices play an ever increasing role in the accumulation of patient endpoint data.

Developing efficiency

In order to resolve the key questions and problems posed by this shift, it is necessary to compare the efficacy of delivering a validated PRO instrument in traditional pencil and paper and web-enabled mobile device formats. Only then will it possible to draw meaningful conclusions about the efficiency of either. The latter format, which may involve mobile-phone and/or tablet-based technologies, should be developed under the supervision of the instrument owner. Once in place, the technology should then be presented to a statistically significant user sample drawn from an appropriate research cohort of patients. This is performed by using a crossover design, where the patients’ responses are compared after they complete one of the modalities (either paper or ePRO) and then complete the other modality after an allotted period of time.

Each respondent will also be asked a number of usability questions once they have completed the process. Responses to these questions will be vital in gauging the efficiency and accessibility of each format, and will inform any strategy moving forward.

Testing for equivalency

The International Society of Pharmacoeconomics and Outcomes Research (ISPOR) has established guidelines for determining equivalency between the different routes to PRO. The guidelines suggest that the level of equivalence testing should vary based on how significantly the PRO has been modified.

Minor changes, such as changing from paper to ePRO may only require cognitive debriefing for patients and usability testing.

In case of word changes, for example a change in a direction from ‘please tick’ to ‘please tap’ when moving from paper to ePRO,  or the change of delivery media (for example from audio to visual) only moderate changes are needed, which may require equivalency testing. To assist this, randomized crossover designs can be employed, in which users are randomly assigned to either paper or electronic PRO for their first administration, and then switched to the other format for the second. Another strategy could involve randomized parallel group designs, with users in one group of the study randomly assigned to the paper format, and those in the second arm assigned to ePRO.

Finally, in the case of major changes being needed, the ePRO should be evaluated as if it were an entirely new instrument.

A recent study by Brendan Mulhern, MRes, Centre for Health Economics Research & Evaluation, University of Technology, Sydney, compared the widely used EQ-5D-5L across different modes of administration. This was done in conjunction with the EuroQol Group to ensure that the electronic version was well designed to avoid the pitfalls highlighted above. It found that response rates were higher in the mobile arm, and the scores were equivalent across the modes. Also, the majority of the mobile arm agreed that the mobile version of EQ-5D-5L was easy to complete, the phone was easy to use and that they would complete mobile health measures again.

This study, whose publication is pending, showed that as long as it is designed appropriately, and the end user is constantly considered during the development of the system, then the electronic pathway is surely the future standard for PRO.

The future standard for PRO

It is now accepted that, as the manual, paper-based process for collecting PROs may not only be cost- and time-inefficient, but also unreliable, mobile devices will play an ever increasing role in the accumulation of patient endpoint data due to its ease of accessibility and cost- and time-efficiency. By following the above guidance, mobile ePRO can be easily validated to become a standard tool for clinical data capture.