HOW TO DEVELOP ANDEVALUATE A PROINSTRUMENT

An Academic presentation byDr. Nancy Agens, Head, Technical Operations, Pubrica Group: www.pubrica.comEmail: sales@pubrica.com

Today's Discussion

Outline

What is PROAsk the Doctor, Why Ask the PatientWhat Information Do I Want from the Patient? – The PRO Instrument Designing a Valid PRO InstrumentEvaluating PRO InstrumentConclusion

What is PRO?

PRO or Patient-reported outcome is a relatively novel method to evaluate the outcome of any treatment by clinician or researcher (in clinical trial) purely based on the inputs got from the patient alone.

The United States Food and Drug Administration (US-FDA) defines PRO as ‘any report of the patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.’

Ask the Doctor, Why Ask the Patient?

Objective data regarding outcome can be given by the treating physician, by investigations and by the caregiver.

These are Observer-reported outcomes (OROs) and medicine has long relied on these for evidence-based management.

The concept of Health-related Quality of Life , a multi-dimensional approach that records patient’s perceived effect of the illness and treatment on quality of life in his/her physical, mental, emotional, economical, and social space.

PRO data is used to inform and guide patient-centered care, to assess risk and benefit from treatment, to decide on treatment protocols and to guide health policy makers.

What Information Do I Want from thePatient? – The PRO Instrument

The PRO instrument is the means to collect the data related to the PRO concept, in the form of questionnaire and the information and documentation that supports its use.

In other words, PRO instrument is a questionnaire that compiles data related to the PRO concept of interest which can be summarized as a score.

Content validity is the extent to which an instrument measures the concept of interest that is desired from both the developer’s and user’s perspective.

Designing a ValidPRO Instrument

Contd..

Steps involved are: -

1. DEFINING THE CONCEPT

Concept should be clearly defined based on review and literature gap analysis of existing PRO instruments as well as taking inputs from scientific research services, developers and most importantly the patients or the users.

2. CONCEPTUAL FRAMEWORK

Breakdown the concept to sub-concepts and finally to items or the questions that will be included in the questionnaire.

3. END-POINT MODEL

Endpoint is the final measured outcome of the treatment that will be compared, and an endpoint model summarises all the endpoint based on OROs and PROs.

Contd..

4. REVIEW OF PRELIMINARY INSTRUMENT

Literacy review is done to understand the proportion of people who may ormay not have the literacy level to take the questionnaire.

Translation review is done to ensure conceptual equivalence whentranslated to another language.

Cognitive interview is the process where the user is made to take thequestionnaire by thinking aloud to understand the approach of the user tothe questions.

Contd..

5. RELIABILITY TESTING

Measure to estimate reproducibility and consistent true estimation of outcomes under consideration. It is further described below.

6. VALIDATION

It is an ongoing never-ending process of improving the instrument by repeated feedback mechanism.

Evaluating PRO Instrument

As clinical validation data accumulates, a PRO instrument undergoes review and revision to make them robust. It includes the evaluating for the following: -

1. VALIDITY

is the extent to which the scores and their interpretations, supported by theoretical evidence, correlate with the proposed use of the instrument.

a. Content Validity- It is a measure of how much the instrument measures the concept of interest.

(i) Qualitative estimation – include items involved, their recall period, responseoptions, scoring pattern, method of conducting the questionnaire etc.

Contd..

(ii) Quantitative estimation – e.g.

=> Item response theory (IRT) – A powerful tool to assess whether the items used measure the entire spectrum or continuum of the patient responses.

=> Rasch analysis - Only IRT tool to use the total of all the items in the PRO instrument making it a simple model.

b. Construct Validity- It is based on comparison with convergent and divergent groups with known validity and showing that the instrument under study conforms to similar interpretations.

c. Criterion Validity- Comparison with known criteria or known gold standard forthis concept.

Contd..

2. RELIABILITY

a. Reproducibility or test-retest reliability- It is assessed under same conditions in same user in a close span of time to assess any variations in recording results that can be attributed to the instrument.

b. Internal consistency- The items should deliver same scores in similar situations consistently.

c. Floor and ceiling effect – i.e. the instrument’s ability to measure extremes of construct helps choose the proper instrument for the study.

d. Inter-interviewer reliability- there should not be significant difference in scores when interviewed by two different interviewers.

e. Variability – Instrument should have ability to detect changes when outcomes are different, else it is no good.

Conclusion

Adhering to good practices while designing and validating a PRO instrument is primary for it to be effective and receive regulatory approvals.

Readiness to continuously revisit and revise items is needed to do better justice to the concept of interest in the long run.

Contact UsUNITED KINGDOM+44-1143520021INDIA+91-9884350006EMAILsales@pubrica.com