Qualitative COA research

Adelphi Values are leaders in the conduct of qualitative research to obtain patient experience data to support the development and content validity testing of COAs. In 2018, Adelphi Values researchers collected qualitative data from more than 700 participants (patients, caregivers, clinicians) across a wide range of conditions. Our experts have extensive experience in qualitative research techniques for COA development and evaluation including concept elicitation, cognitive debriefing, ethnography and trial experience interviews. In the past 3 years we have conducted over 250 qualitative studies.

We have in-depth understanding of qualitative methodologies and the flexibility in thinking to apply them appropriately and creatively to COA instrument development. We are also fully versed in regulatory expectations for qualitative research and know what aspects are essential and which are ‘nice to have’. While the gold standard approach might remain face-to-face, 1:1 interviews, we also have the experience of conducting qualitative research across a range of modalities (telephone, video call, chatroom, etc), and the lightness of touch to know when the advantages of one approach outweigh the compromises it entails.

Concept elicitation focus groups and interviews

We lead the field in the rigorous conduct of concept elicitation interviews and focus groups to high standards to meet clients’ COA instrument development goals. Our in-depth understanding of what is important in concept elicitation interviewing and qualitative analysis means that we are able to apply our methods to elicit in-depth, high quality data from a range of participant groups including patients, caregivers/parents and clinicians. We have particularly strong expertise in conducting interviews with children, in rare diseases and other special populations where greater creativity can be required.

Hallmarks of our approach to concept elicitation research are the following:

Open-ended enquiry followed by more direct question if topics of interest do not emerge
Use of creative tasks adapted from market research such as drawing tasks, mood boards, timeline tasks, importance rating
Qualitative analysis using Atlas ti software and thematic analysis methods
Use of tables and figures to present in-depth findings in concise and compelling ways

Our concept elicitation research is well accepted by the FDA and has supported many FDA and EMA PRO/COA submissions. Beyond PRO/COA development needs, the understanding of the patient perspective provided by rigorous concept elicitation research can be invaluable to support go/no go decision, target product profile development and other commercial decisions.

Real-time qualitative data collection

With increasing use of mobile digital technology, there is now the opportunity to capture patients’ qualitative experiences in real time, through smart phones and tablets using an ethnographic approach to qualitative data collection. We have successfully used these approaches to supplement traditional interviews and focus groups with qualitative data with greater ecological validity. Previous projects in this space have included collection of photos, audio, videos and text updates via smartphones.

Cognitive debriefing interviews

Adelphi Values have been at the forefront of the development and refinement of cognitive debriefing interviewing as a central activity in the development and testing of any PRO instrument. Typically we employ a mixture of using a ‘think-aloud’ method (asking patients to speak their thoughts aloud as they read and respond to the questionnaire) and following up with specific and detailed debriefing questions to gain insight regarding patient understanding, and the relevance and appropriateness of items, instruction, response options and recall periods. Our skilled interviewers are highly experienced in eliciting feedback from hard to reach groups such as young children and patients with low affect.

Our cognitive debriefing research has been extensively published and is well accepted by both the FDA and EMA. In the past 3 years we have conducted over 100 cognitive debriefing studies involving over 1200 interviews with patient, caregivers and other stakeholders. Adelphi Values have conducted cognitive debriefing studies in over 20 countries, including: United Kingdom, United States, several European countries (most commonly France, Germany, Italy and Spain) and also countries in Asia (specifically Japan).

Feasibility testing

As part of a mixed methods approach to diary development, Adelphi Values advocate performing pilot testing or feasibility testing prior to the cognitive debriefing interview. This involves patients completing the newly developed PRO (usually a daily diary and usually completed on an ePRO device) at home in a ‘real world’ setting for a period of several days prior to their cognitive debriefing interview. We believe this approach provides considerable added value as patients are able to reflect on their actual experience of completing the diary and fitting it into their life, rather than having to think hypothetically – particularly valuable for children who have difficulty thinking beyond the here and now. Moreover, the data can be collected to provide early insight into item response distributions, albeit in a small sample.

Trial experience interviews

Qualitative trial experience interviews with clinical trial or observational study participants provides an opportunity to gather additional information about the patient perspective.

refine a COA instrument through cognitive debriefing as part of a mixed methods approach,
increase understanding of the patient experience of symptoms and/or medication benefits and side effects
obtain patient/caregiver perspectives to aid interpretation of what level of change can be considered meaningful

As leaders in qualitative, patient-centered research, we are ideally placed to help clients conduct qualitative exit interviews to a high standard but in an efficient, cost effective manner. Exit interviews can be particularly valuable to:

Patient recruitment

Efficient patient recruitment is central to the success of any qualitative study and has the biggest impact on timelines of any aspect of the research. Over the past 20 years we have built up an extensive network of trusted patient recruitment partners in over 30 countries, but particularly across the US and Europe. Moreover we increasingly partner with patient advocacy groups or other charitable bodies, particularly to access rare disease populations. Drivers that shape recruitment strategy include, but are not limited to: the characteristics of the target population (e.g. severity of disease, cognitive capacity, and age), the country where the patients are being recruited from (e.g. English or non-English speaking), the format of the interview (e.g. face-to-face or telephone) and the rarity of the patient condition. For the majority of studies, we aim to meet the gold standard of regulatory guidance, with each subject having a documented, clinically confirmed diagnosis. Adelphi Values is also adept at conducting online platform and direct to research site recruitment strategies.

Mixed methods

While qualitative and quantitative research methods are grounded in different epistemological and methodological perspectives, Adelphi Values consider both to be of critical value in the context of patient-centered research and PRO/COA development and psychometric evaluation and that there should be inter-play between the two throughout instrument development. This can take many forms, such as:

Collecting data during feasibility testing prior to cognitive debriefing, allowing early examination of item-level descriptive statistics early in instrument development
Performing qualitative exit interviews during observational studies or clinical trials to provide insights which might contribute to decisions around deletion of items, development of scoring or to aid interpretation of score changes over time through triangulation of data

Interaction between qualitative and quantitative methodologies is the optimal approach to ensuring rigorous instrument development that will meet FDA and EMA requirements.

Relevant publications

Arbuckle, R., Staunton, H., Sully, K., Tomkins, S., Khindri, Sanjeev., Svedsater, H. and Nelsen, L. Use of both qualitative and quantitative methods to estimate meaningful change thresholds for key endpoints in pediatric asthma trials. Value In Health (2018) – in press

Theodore-Oklota C, Bonner N, Spencer H, Arbuckle R, Chen C-Y, Skrinar A. Qualitative Research to Explore the Patient Experience of X-Linked Hypophosphatemia and Evaluate the Suitability of the BPI-SF and WOMAC® as Clinical Trial End Points. Value in Health 2018; 21(8):973-983

Turner-Bowker DM, Lamoureux RE, Stokes J, et al. Informing a priori Sample Size Estimation in Qualitative Concept Elicitation Interview Studies for Clinical Outcome Assessment Instrument Development. Value Health. 2018; 21(7):839-842.

Jacobson TA, Edelman SV, Galipeau N, Shields AL, Mallya UG, Koren A, Davidson MH. Development of a content-valid patient-reported measure to assess the statin intolerance experience. Patient. 2017; 10(3):321-334.

Tatlock S, Rudell K, Panter C, Arbuckle R, Harrold L, Taylor W, Symonds T. What Outcomes are Important for Gout Patients? In-Depth Qualitative Research into the Gout Patient Experience to Determine Optimal Endpoints for Evaluating Therapeutic Interventions. Patient. 2017; 10(1): 65–79