There is an extended use of Patient Reported Outcome (PRO) measures in clinical trials used to assess health-related quality of life, symptoms, or perception of health status by the patients participating in clinical research. These measures are increasingly being collected via electronic devices and are, as such, referred to as ePROs.
In ophthalmology trials, the types of ePRO used can be general (SF-36, EQ-5D), vision specific (VFQ-25) or condition specific (glaucoma utility index, Retinopathy Treatment Satisfaction Questionnaire). A very wide range of validated questionnaires are available for use in research and selecting the right ePRO in an ophthalmology study is paramount to the success of a clinical trial.
5 keys to properly select ePRO and ClinRO in ophthalmology clinical trials
Ensure the ePRO answers the intended clinical endpoint
PROs have become a vital part of modern ophthalmology with an increasing focus on PROs within high-quality ophthalmic trials. International organizations are now significant contributors to the development, access, and guidance for the best use of PROs. The International Society for Quality of Life Research (ISOQOL) has published recommendations for minimum measurement standards for PROs, intended to promote the appropriate use these measures in patient-centered outcome research and comparative effectiveness research. The minimum standards include documentation of the conceptual and measurement model, evidence for reliability, validity, interpretability of scores, quality translation, and acceptable patient and investigator burden.
A PRO or ePRO can be used to evaluate:
- A primary endpoint: a vision specific measure such as the National Eye Institute Visual Function Questionnaire (NEI VFQ) is used to measure specific visual impairment symptoms and can assess clinical benefits from treatment. The development of the NEI VFQ, although not the first scale to be validated, was first published in 1998 and remains in its shorter form (NEI VFQ-25), the most widely used vision-related PRO measure in the world.
- A secondary endpoint: can complete or show additional benefits to the primary measure, such as a general health utility measure like the EuroQol – 5 Dimension (EQ-5D), which is not disease specific. The EQ-5D is measured in several ophthalmology studies to support clinical ophthalmology evaluations and monitor the patients’ overall wellbeing.
- An exploratory endpoint: are used to frame future research or explore new hypotheses and are not generally rigorously evaluated for statistical significance. With mobile technology, clinical trials can capture up to daily ePRO measures for their patients. This creates potential for new endpoints that have not been previously possible to assess or improving how we assess existing endpoints.
It is important to carefully select the measure that will answer the intended endpoint, whether it be primary, secondary, or exploratory. Additionally, it is paramount to the study success to match the chosen PRO together with the patients’ capabilities to ensure the PROs can be completed in a timely, accurate and complete manner.
A trial with positive ePRO data, especially when the ePRO is a primary endpoint, can help drive marketing authorizations and labeling claims. A study of drugs approved by the FDA between January 2000 and June 2012 looked at approval of ‘PRO labelling’ (i.e. where the label of the drug included claims of treatment benefit based on the results of a PRO). Of the 308 drugs approved during this period, 70 were specifically approved for a PRO based labelling claim. Of those with the PRO labelling claim, 57 were based on studies in which the PRO was the primary outcome.
Use your PROs to power economic evaluations
Incorporating an economic evaluation within a trial provides valuable information for clinical decision making and coverage determinations, particularly when the clinical effectiveness of the interventions is similar but might vary in terms of patient-reported ability to read, drive or work.
Understanding the impact of an investigational drug on patients’ quality of life (QoL), and analyzing its economic consequences, is why many trials include utility measures in their trials. The utility measures help to define and assign value to health states of interest. PROs are frequently used to measure QoL, which is a significant input for cost-effectiveness models.
A PRO, such as the EQ-5D, is a standard measure of QoL for these models. QoL directly contributes to calculating the cost per quality-adjusted life-year (QALY), which measures the number of life-years gained after receiving a treatment along with the quality of their health during that time. The cost of treatment is then compared to the potential gain in QALYs for patients. This provides a convenient yardstick for measuring and comparing health effects of varied interventions across diverse diseases and conditions.
In ophthalmology, it is unusual for interventions to extend a person’s life. Instead, most interventions affect a person’s ability to see and his or her time spent with better vision. Therefore, the QALY outcome tends to be based on improvement in utility and quality of life. The more pervasive use of QALY calculations may help to develop treatments that more positively impact patients’ health-related quality of life.
Get your patients involved in the ePRO selection
Although clinical research stakeholders strive to be patient-centered in their approach, dependence on clinical parameters alone is not enough. As clinical investigations may not necessarily correlate with the patient’s own experience of their disease, PROs help complete this picture in the clinical research settings.
The FDA recommends for the PROs to be specifically targeted and designed for patients with patients:
“Because the purpose of a PRO measure is to capture the patient’s experience, an instrument will not be a credible measure without evidence of its usefulness from the target population of patients. Sponsors should provide documented evidence of patient input during instrument development and of the instrument’s performance in the specific application in which it is used (i.e., population, condition).”
If there can be direct patient engagement from the outset, including patient input into the selection of domains to explore, this will ensure that studies capture outcomes that matter to the patients. In this way, a clinical study is likely to be more successful and have better patient adherence when the protocol is designed in a way to improve patient ease and understanding.
In ophthalmology trials, interviewing patients on their perception of symptoms and the impact it has in their daily life, can help drive ePRO selection towards the ones that will capture the data that is of most value to the patients.
Consider the method of ePRO administration
PROs in ophthalmology trials can either be self-administered by the patient or the responses can be collected through an interview, administered by the investigator or site personnel. They both present advantages and drawbacks that can affect the responses provided by the patients.
Self- administered ePRO presents the following advantages:
- Prevents interview bias and avoids the need for additional training for interviewers.
- There is less coordination required, as patients may complete the questionnaires on their own time and do not require any specific appointment to be made. It allows for patient autonomy.
- For aspects that explore satisfaction or personal feelings, self-administration allows for more privacy compared to an interview setting.
PROs for patients participating in ophthalmic trials may necessitate an interview mode because of potential visibility limitations. If the population skews more elderly, there may be additional dexterity limitations or technology difficulties. However, there are limitations with using an interview mode that include:
- The Interview bias that may impact the validity of the data. The presence of an interviewer can be distracting to respondents, or interviewers can vary in their ability to appear or sound neutral, listen, aid recall, and to record responses. Careful training and monitoring of interviewers can minimize this, but completely removing this bias is difficult.
- As interviews involve social interaction with another person, it can lead to respondents taking social norms into account when responding, resulting in potential social desirability bias (the desire of respondents to present themselves in the best possible light). This can result in the over-reporting of desirable behaviors, and under-reporting of undesirable behaviors.
- Interview requires standardization of the interview process and additional training for the interviewer may be required, which will need to be anticipated in the study timelines and budget.
Interviews do have some advantages over self-administration such as interviewers can control the order of the questions administered, while patients may change the order and adjust their response accordingly in self-administration. Also, an interviewer can re-direct respondents back to the topic of relevance if they stray off it, probe to elicit relevant information, and utilize a range of techniques to prompt memory, while in the self-administration setting the respondent alone has to judge whether the information they have recalled is relevant to the question, and how best to respond.
Selecting the method of questionnaire administration should take into consideration what the preferred method would be for the patient, to encourage the most accurate responses, and how to reduce potential biasing effects. This must happen early in the questionnaire selection process, as it may have important implications in the research methodology and on the validity of the results of research.
Anticipate cultural differences in multi country clinical trials
Before selecting a PRO for use in a large global trial, it is important to anticipate that responses to questionnaires may vary based on the country or cultural differences.
Ensuring that the phrasing or choice of examples used in the questionnaire makes sense for each country or culture is key. In developing a Japan specific NEI-VFQ-25, authors included different choices for the near-vision and distant vision subscales, where items from the original version were endorsed by very few participants. For example, in the ‘Distance vision’ subscale, replacing ‘going out to movies/plays’ with ‘seeing television program’ reduced the rates of missing data and therefore increased the measurement precision.
Occasionally, the structure of questionnaires or even an entire question may be difficult to apply to certain populations. These situations require thoughtful customization work to avoid generating data inconsistencies and statistical comparability risks. In the same Japan example, the ‘Driving’ subscale had a high rate of missing data since most participants did not drive, so the driving question was excluded. Additionally, the participants did not find an expression equivalent to ‘not applicable.’ Re-structuring of the questionnaire to include additional conditional questions was necessary to generate similar responses. Participants had to be asked whether they participated in an activity first, before asking if they had difficulty doing that activity or if they did not do the activity due to vision problems.
Good preparation and planning can help with finding a balance between maintaining cultural relevance and data quality. It can be challenging, especially in cases where question exclusion or restructuring may be desired for specific countries, to ensure data collection and analysis are standardized.
The ISPOR task force developed a 10-step process to translate and adapt PRO instruments to cultural differences. This is a good resource to refer to when planning for a multi-country study.
Missing PRO data or not meeting a clinical endpoint in a trial can have considerable human, scientific and financial impacts. Properly selecting and implementing the right PRO measures will require input from the target population, thoughtful review of the administration method, and consideration of the cultural difference within the targeted population. Even more importantly, suitable ePRO selection may be the key to ensuring acceptance and adoption of the study by the participants who generously complete them.
Celeste Sage, Clinical Operations Project Manager @Kayentis
& Estelle Haenel, Medical Director @Kayentis
