Real-World Evidence and Health Technology Assessment: Past, Present, and Future

The annual CADTH Symposium is a signature conference for the Canadian and global health technology assessment (HTA) community. The event regularly attracts participants from a range of sectors, including government, regional health authorities, hospitals, clinical practice, patient advocacy, academia, and the pharmaceutical, medical device, and digital health care industries.

The program for the 2021 Symposium included a panel presentation on real-world evidence (RWE) and HTA. During the session, several members of CADTH’s leadership team provided an overview of RWE at CADTH and discussed past, current, and future initiatives.

The following is a summary of the panel presentations and you can watch a recording of the panel discussion.

Introduction

Laurie Lambert
Lead, Real-World Evidence, CADTH

“We’ve been talking about real-world evidence with lots of different names for at least the last 20 years and really much longer,” said Laurie Lambert. She noted that over that time there had been a number of issues raised and opinions expressed especially with regard to the value of RWE in relation to randomized clinical trials (RCTs) for the purposes of health care decision-making. She cited examples from a number of prominent medical journals and online sites, including the New England Journal of Medicine and JAMA Network Open. “I think we all know in Canada and elsewhere that we have tended to work in silos and that makes generating and using and talking about real-world evidence very difficult,” Lambert added. “But that was the past.” A new international definition of health technology assessment arrived at collaboratively, outlines a number of different domains that must be assessed to determine the value of new health technologies. Lambert said it is because of this multiplicity of domains that RWE is needed and can play a significant role. She closed by citing an article published by CADTH and Health Canada in 2020 detailing an action plan to integrate the use of RWE across the drug life cycle.

Scientific Advice Program

Amy Sood
Manager, Scientific Advice, CADTH

Amy Sood outlined what CADTH is currently doing with RWE very early in the life cycle of a drug and potential future opportunities. Since January 2015, the Scientific Advice program at CADTH has been providing an opportunity for pharmaceutical companies to engage early with CADTH to inform evidence generation. Scientific Advice is currently offered before the initiation of pivotal trials when adjustments can still be made to trial protocols based on feedback from CADTH. Sood emphasized that the goal of the program is to inform evidence generation from a Canadian HTA perspective, so the evidence generated is useful for the client and meets the needs of a future CADTH Reimbursement Review.

CADTH currently has 3 scientific advice offerings:

• CADTH-only Scientific Advice
• Parallel Scientific Advice with CADTH and Health Canada (with INESSS in an observatory role)
• Parallel Scientific Advice with NICE and CADTH

“We are seeing more and more interest at this early stage regarding feedback on real-world evidence-generation plans,” said Sood. Data from 28 CADTH Scientific Advice evaluations showed that for the proposed clinical evaluation plans, 50% of the projects included questions about RWE generation. Some questions were broad in scope and others were more specific, often related to the use of real-world data to inform indirect comparisons or to further support the interpretation of surrogate outcomes. “We are continuing to accept questions related to real-world evidence as part of CADTH Scientific Advice, and we encourage applicants to consider integrating real-world evidence planning into the briefing book submissions,” said Sood. She added that in addition to the existing opportunity to seek advice on RWE before initiating pivotal trials, there is the potential opportunity to explore how the CADTH Scientific Advice Program could inform RWE generation at additional time points in the life cycle of a drug. This could complement other ongoing initiatives at CADTH to further enhance the life cycle approach to pharmaceuticals. Any updates to the Scientific Advice program will be posted on the Scientific Advice page on the CADTH website.

Pharmaceutical Reviews

Amanda Allard
Director, Pharmaceutical Reviews, CADTH

Amanda Allard described how RWE is used in CADTH Reimbursement Reviews. “What may not be immediately obvious to everybody is that CADTH has been working steadily to evolve the process to involve and incorporate RWE into our reviews, when it is appropriate,” she said.

Allard presented the following examples:

• In late 2017, CADTH launched expanded eligibility criteria for drug reimbursement resubmissions that removed the requirement for sponsors to submit new RCT data.
• In early 2018, CADTH revised its approach and internal templates for initial submissions to allow for greater consideration of non-randomized forms of evidence. Since that time, CADTH has seen resubmission requests that have included a wide range of evidence, some of which were based solely on RWE. Internally, CADTH has also adapted by providing additional guidance and training for our reviewers to facilitate seamless integration of RWE into the review.
• In 2019, CADTH launched a reassessment framework that provides a mechanism for industry to provide new data, including RWE, to support revised reimbursement criteria or conditions. Since its implementation, there has been a gradual increase in interest from sponsors to take advantage of this opportunity, said Allard.
• In September 2021, CADTH provided the opportunity for new data, including RWE, to be submitted during the reconsideration phase of the reimbursement review process, subject to certain conditions.
• CADTH continues to work with stakeholders to address barriers related to coverage with evidence development.

Allard also described 3 scenarios in which RWE could be considered in Reimbursement Reviews: 

• during the initial submission to CADTH
• with time-limited recommendations, and 
• for resubmissions and reassessments.

The first scenario is in the initial submission process. Allard reiterated that CADTH uses RWE when appropriate (e.g., to address gaps in the evidence from the confirmatory studies) in both oncology and non-oncology drug reviews. RWE is not usually included as part of CADTH systematic reviews because it rarely meets the criteria established by the CADTH review protocol. However, RWE is generally included as supportive evidence when it does address important gaps in information that remain from clinical trials included in the systematic review. These gaps are identified through critical appraisals by CADTH clinical reviewers throughout the review process. Gaps in evidence are also identified during consultations with clinical specialists with expertise in the diagnosis and management of the condition for which the drug under review is indicated.

The second scenario in which the use of RWE is under consideration is for time-limited recommendations that are conditional upon evidence generation and reassessment.

The third scenario in which RWE is considered in the Reimbursement Review process is for reassessments and resubmissions. Reassessments are conducted to address questions related to the comparable clinical benefit and/or cost-effectiveness of a drug that is currently reimbursed and when revisions are sought for existing reimbursement conditions based on new clinical or economic evidence. Resubmissions are typically limited to drugs that were not recommended for reimbursement by the expert committee and are not currently reimbursed by the drug programs for the indication of interest. For both types of submissions, RWE could either be part of the submission in support of other new evidence that the sponsor presents or it may be the sole basis for a resubmission or a reassessment. In either case, Allard said the sponsor must complete an eligibility request form and should always include an explanation about why the use of RWE is appropriate to address the evidence gaps, the limitations, uncertainty, and so on.

Shifting to a discussion of RWE from a pharmaceutical review perspective, Allard listed the following instances of when RWE would be most beneficial:

• the evaluation of clinical end points requires longer-term follow-up, which may not be available at the time of submission
• there is uncertainty regarding persistence of the treatment effect
• RCTs are impractical because of a small number of patients
• randomized studies lack relevant comparators
• there is uncertainty regarding the dosages that are actually used in clinical practice
• RCTs have limited external validity and additional nonrandomized studies can provide meaningful insight into the effectiveness of the treatment in the target population.

Since the CADTH review team has encountered some challenges with RWE submissions, Allard provided the following advice to sponsors when including RWE as part of a reimbursement submission:

• Ensure the methods and results for the submitted RWE are complete and accurately reported using best practices for the type of study involved. Lack of standardized reporting and poorly reported RWE can slow the review process.
• Be prepared for questions from CADTH. Delays may occur when sponsors do not have complete access to the data (e.g., this may be the case for academic registry studies).
• Submit RWE with a robust study design and a clear justification of why RWE is appropriate.
• Clearly identify the gaps that RWE is intended to address 

Therapeutic Review of Multiple Myeloma

Amanda Manoharan
Program Development Officer, CADTH

Amanda Manoharan provided an overview of the use of RWE in CADTH’s first Therapeutic Review in oncology that compares the clinical and cost-effectiveness of different treatments for patients with multiple myeloma (MM) who are ineligible for stem cell transplantation or who have relapsed and/or refractory MM. The review uses patient registry data concerning patient characteristics, experiences, and perspectives.

The MM treatment landscape continues to evolve and presents a complex array of therapeutic options to choose from, including many high-cost novel drugs used as either adjunctive therapy with existing drugs or replacements for existing therapeutic drugs. CADTH has been collaborating with an international research team with expertise in MM from the Erasmus School of Health Policy & Management (ESHPM) in the Netherlands. The ESHPM team is updating 2 of its existing network meta-analyses for the newly diagnosed transplant-ineligible patient population and the relapsed and/or refractory MM patient population. The ESHPM team will combine the outcomes of the network meta-analyses with RWE from the Canadian Myeloma Research Group to develop a living economic model on the cost-effectiveness of treatment sequences, starting with newly diagnosed transplant-ineligible patients with MM. 

Manoharan said CADTH is being proactive by developing a living economic model. When novel agents become available in MM, there is an opportunity to incorporate new treatments into the economic model in a timely manner and payers can use this living economic model as they manage their formularies and search for optimal treatment sequences in MM. 

“There is tremendous value in using real-world evidence to fill in gaps in RCTs in oncology, and other disease areas also,” said Manoharan. “In this therapeutic review of multiple myeloma, which is a complex treatment space, having access to real-world evidence allows us to gain insights into possible overall survival benefits, which is a clinically important outcome for decision-making in HTA, as well as the opportunity to assess treatment sequences and also gain a better understanding of treatment regimens that are reflective of Canadian clinical practice.”

Formulary Management: Biologics and Psoriasis Project

Peter Dyrda
Manager, Program and Policy Development, CADTH

Peter Dyrda discussed the use of RWE at CADTH in the context of formulary management, with specific reference to a project that assesses the use of older versus newer biologics in the treatment of plaque psoriasis. Dyrda explained that formulary management is a relatively new area of interest within CADTH. The framework developed for considering projects has 4 categories: improved outcomes, improvement of access, predictability, and meaningful outcomes.

When investigating projects that may fit this framework, Dyrda said attention was drawn to a March 21, 2021 New York Times article by the Drug Pricing Lab in the US that proposed regulated price decreases for biologics after they lose exclusivity for biosimilars to compete more effectively, which prompted a consideration of the impact of this type of policy in the Canadian market relative to the use of biologics and biosimilars in treating psoriasis. Although a great deal of attention has been paid to the overall uptake of biosimilars in Canada and globally once marketed, Dyrda said relatively little attention has been given to the delay in biosimilars entering the market immediately after the loss of exclusivity for biologics. Because of the potential for looking at this particular issue in the context of formulary management, CADTH decided to take a 360-degree view of biologics in the management of psoriasis. This ongoing project assesses both older biologics (pre-2010) and new biologics (post-2015) in terms of their use, efficacy, cost, and how much payers spend on older biologics after the loss of exclusivity. Rather than take a therapeutic review approach, which takes a lot of both time and resources, Dyrda said the aim of the formulary management approach was to be more targeted and fit for purpose in answering policy questions in a timely manner. The project was initiated in June 2021 with the intent to conclude it before the end of the fiscal year. The project used National Prescription Drug Utilization Information System (NPDUIS) data collected from CIHI and private payer data collected from the payers themselves. The analysis is outsourced in part to epidemiologists and then validated by clinicians. The project has used claims data generated passively by both public and private payers that has not been previously analyzed. Dyrda said this type of RWE can tell a lot about health system performance and whether products are being used optimally in the marketplace. 

“Now that we have opportunities to work together with payers [and] work with different datasets, we can turn that data into evidence and then provide them [the payers] with what they need for formulary management,” said Dyrda.

Post-Market Drug Evaluation

Laurie Lambert
Lead, Real-World Evidence, CADTH

In discussing post-market drug evaluation (PMDE), Laurie Lambert started by noting that CADTH has been named as the new host for the work of the Drug Safety and Effectiveness Network (DSEN) and that CADTH has recently put in place a transition team to work with the Canadian Institute for Health Research (CIHR) and Health Canada to ensure the new CADTH PMDE Program is in place by September 2022. Lambert said the guiding principles for PMDE moving forward are relevant to many of the themes raised by earlier speakers during the session, such as the need for flexibility, to be responsive to stakeholders, to be accountable, and to ensure that all evidence is fit for purpose and meets decision-makers’ needs. Elements of the new model for PMDE include consideration of a new funding structure (grants and contracts), prioritization of incoming queries, and greater accountability. The new program will have dedicated leadership within CADTH and increased collaboration between decision-makers and research teams throughout the process to best meet the needs of the decision-maker. By adding contracts to the funding structure, the PMDE network will be able to access additional expertise when needed, which allows it to be more responsive to the needs of decision-makers. The new PMDE process will include a feasibility assessment guided by research priorities and followed by the generation and/or leveraging of RWE about the safety and effectiveness of drugs to support regulatory and formulary listing decisions.

Real-World Evidence Program

CADTH collaborated with CIHR, the Canadian Organization for Rare Disorders (CORD), and Health Canada to cohost the Best Brains Exchange for Drugs for Rare Diseases, a major initiative to help inform the optimal use of RWE for decision-making. This virtual meeting was held over a 2-day period in October 2021 and featured 150 participants and representatives from 10 different types of stakeholders. The meeting participants were divided into 5 multi-stakeholder groups who discussed the optimal use of RWE in the Canadian context. Comments from attendees included:

• They liked the diversity of stakeholders and the varied relevant perspectives around the virtual table.
• They appreciated the depth of knowledge and variety of stakeholders. There was also confirmation among all that the process needs to be patient-focused.
• Many found the honest dialogue and willingness across stakeholders to collaborate and to be solutions-focused very encouraging.
• Many see the importance of learning from one another and not re-creating the wheel — existing structures and processes for RWE should be leveraged and applied more broadly.
• Multi-stakeholder considerations are more complex than initially understood; however, they are also not insurmountable.

Regarding the next steps for strengthening RWE for decision-making about drugs for rare diseases in Canada, many respondents asked that there be an outreach to different stakeholders to follow up and continue the discussion to advance RWE initiatives. Other proposed steps following the meeting included involving HTA early in the evidence-generation process to ensure data meet all needs, developing an RWE plan well before HTA submission, and working to remove barriers to data linkages in existing rare disease databases and registries. Lambert said CADTH’s next steps will be to share 2 or 3 real-world Canadian case studies with groups involved in the Best Brains Exchange to show what good quality real-world data looks like and to publish a report of the Best Brains Exchange in collaboration with CIHR and CORD. Lambert explained that CADTH is also chairing a Steering Committee that is working toward optimizing the use of RWE at a pan-Canadian level by meeting and talking regularly about challenges and solutions. Member organizations currently include:

• Health Canada
• INESSS
• pCPA
• CIHI
• Statistics Canada
• CIHR
• HDRN
• CORD
• IMC-Biotech

“We hope to find some learning projects we can do with each of these pan-Canadian health organizations to try and move forward,” said Lambert. CADTH is also working on RWE at the international level by participating in the International Network of Agencies for Health Technology Assessment (INAHTA) RWE Learning Group and the European-based RWE4Decisions initiative. Through multi-stakeholder discussion with regulators, HTAs, patients, academia, industry, and clinician-researchers, the vision is to reach agreement on what RWE can and should be collected for highly innovative technologies to inform decision-making by health care organizations, clinicians, and patients. CADTH is also involved with the International Society of Pharmacoepidemiology (ISPE) which has created an RWE task force to develop critical appraisal guidance for HTAs. “We need to trust the data if we're going to use it as evidence,” said Lambert. “People trust RCT data, and we need RWE that is also ‘decision grade’.” 

CADTH is launching an RWE learning period during which projects that integrate RWE at key timepoints within the HTA process (Figure 1) will be sought both internally at CADTH and in collaboration with external stakeholders.

Figure 1: Key Timepoints for RWE Learning Projects

HTA = health technology assessment; RWE = real-world evidence.

“We’re open to learning and listening,” Lambert concluded.

Final Thoughts on RWE

Manoharan: Real-world data has tremendous value in oncology because we’re able to have longer-term follow-up data, which means we can also assess potential benefits around overall survival.

Sood: I would like to encourage prospective applicants to the CADTH Scientific Advice Program to plan early and incorporate real-world evidence-generation plans into the briefing book submission for early feedback.

Dyrda: The quality of data will only be as good as what everybody puts into it, so there’s an opportunity for broader sharing of data. When we start looking at administrative data, sometimes we can get a good dataset from CIHI, but it could be missing elements that could be housed with other organizations. I think being able to share variables across groups would provide the most robust dataset, so that we can provide the most accurate analysis and insights to decision-makers and thus would be beneficial to all. As we move forward with this, continuing the dialogue and increasing our collaboration will be key to improve data quality and usefulness.

Allard: If you have real-world evidence that you think can fill some of the evidence gaps in your submission, please include it. The team will assess it and if it’s appropriate, we will summarize and critically appraise it to fill those gaps to provide decision-makers with the best information available.

Lambert: Just do it and do it now!