Processus d’évaluation des technologies de médicaments uniques au sein des différents organismes d’évaluation des technologies de la santé

Détails

État du projet:

Terminé

Gamme de produits:

Examen d’une technologie de la santé

Sous-type de projet :

Analyse de l'environnement

Numéro de projet :

ES0293-000

Context

Health Technology Assessment (HTA) informs health care decision-makers about the value — such as the safety, efficacy, and cost-effectiveness — of new and existing health technologies. HTA can inform clinical decision-making, as well as formulary listing and reimbursement decisions within health care systems. Given the utility of HTA to support evidence-based decision-making, it has been widely adopted and customized across different countries and regions.

HTA is a dynamic and rapidly evolving process, and involves different types of assessments. Although each organization may apply a different process or methodology, a focus on the direct and indirect consequences (spanning medical, economic, social, and ethical implications) of introducing a new health technology into a national or regional health care system remains consistent.¹ HTA can be applied across the full spectrum of health care technologies, such as drugs, diagnostic tests, medical, dental, and surgical devices; and medical procedures. This scan focuses exclusively on the HTA procedures and policies used in the assessment of drugs.

Currently, most HTA organizations appraise new drugs on a “single technology” basis. For the purposes of this scan, “single drug technology assessment” refers to an assessment that results in clinical and reimbursement decisions or recommendations regarding a single drug, single treatment regimen, or single drugs in a combination regimen. Some organizations, such as the National Institute for Health and Care Excellence (NICE) and Canada's Drug Agency, also conduct multiple drug technology assessments (also known as therapeutic or drug class reviews), which refers to comparing drug classes, or drugs within a class, or drugs used to manage a disease, and where recommendations are made for more than one intervention. Multiple drug technology assessments are outside the scope of this scan.

Although the single drug technology assessment is an established approach in many jurisdictions, there remain important differences in the review processes between organizations. The standard processes of a single drug technology assessment generally include a submission for an assessment by the applicant, a clinical and economic review, and deliberation and development of a set of decisions or recommendations made by an expert committee based on the evidence from the clinical and economic review. HTA organizations may also invite stakeholders such as manufacturers, payers, clinical experts, patient groups, and the public to provide feedback at various stages of the review process. As such, there have been efforts to develop standards of good practice in HTA processes, as well as efforts to benchmark HTA organizations along these standards. For example, Drummond et al. developed a set of principles of good practice, and subsequently developed a series of audit questions that are based on these principles, with the aim of reliably benchmarking HTA organizations.¹

In Canada, Canada's Drug Agency — an independent, not-for-profit organization — is responsible for conducting HTA to inform health care decision-making. Canada's Drug Agency operates two pan-Canadian single drug technology assessment processes: the Canada's Drug Agency pan-Canadian Oncology Drug Review (pCODR) and the Canada's Drug Agency Common Drug Review (CDR). The pCODR process specifically assesses oncology drugs, while CDR assesses all the other drugs. Independent expert committees within each of these programs make reimbursement recommendations to, in the case of CDR, federal, provincial, and territorial public drug plans (with the exception of Quebec). In the case of pCODR, recommendations are also made to provincial cancer agencies in addition to federal, provincial, and territorial drug plans.² In Quebec, the Institut national d'excellence en santé et en services sociaux (INESSS) assesses clinical and cost advantages of the technologies, medications, and interventions used in health care in order to make recommendations on the adoption, use, and coverage of these products to the Quebec public drug plan.

Abbreviations

CDEC — Canada's Drug Agency Canadian Drug Expert Committee
CDR — Canada's Drug Agency Common Drug Review
ERG — Evidence Review Group
G-BA — Gemeinsamer Bundesausschuss or Federal Joint Committee
HAS — Haute Autorité de santé or French National Authority for Health
HTA — health technology assessment
INESSS — Institut national d'excellence en santé et en services sociaux
IQWiG — Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen or Institute for Quality and Efficiency in Health Care
NHS — National Health Services
NICE — National Institute for Health and Care Excellence
NIHR — National Institute for Health Research
PAG — Provincial Advisory Group
PBAC — Pharmaceutical Benefits Advisory Committee
pCODR — Canada's Drug Agency pan-Canadian Oncology Drug Review
pERC — Canada's Drug Agency pCODR Expert Review Committee
PHARMAC — Pharmaceutical Management Agency
PTAC — Pharmaceutical and Therapeutics Advisory Committee
RCT — randomized controlled trial
RGAM — Régime général d’assurance medicaments
SMC — Scottish Medicines Consortium

Objective

The objective of this Environmental Scan is to compare the Canada's Drug Agency single drug technology assessment processes with those of other Canadian and international HTA organizations. The scan identifies and compares the structure and process of single drug technology assessment programs across HTA organizations in selected regions, and provides a comparison of the following components of single drug technology assessment: the submission process, the clinical review process, and the deliberative process. The scan focuses on the various approaches to conducting clinical reviews of new drugs or new indications for existing drugs; however, economic review processes are outside the scope of this Environmental Scan. Multiple drug technology assessment (also known as therapeutic review or drug class review) and assessment of diagnostic tests and medical, dental, and surgical devices and procedures are also outside the scope of the scan. Following is a list of the HTA organizations included in the Environmental Scan. Appendix 1 presents a brief overview of each organization and their mandate.

Canada's Drug Agency Common Drug Review (CDR), Canada
Canada's Drug Agency pan-Canadian Oncology Drug Review (pCODR), Canada
Institut national d'excellence en santé et en services sociaux (INESSS), Canada
National Institute for Health and Care Excellence (NICE), United Kingdom
Scottish Medicines Consortium (SMC), United Kingdom
Pharmaceutical Benefits Advisory Committee (PBAC), Australia
The Pharmaceutical Management Agency (PHARMAC), New Zealand
Gemeinsamer Bundesausschuss (G-BA) or the Federal Joint Committee, Germany^a
Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) or Institute for Quality and Efficiency in Health Care, Germany^a
The Haute Autorité de santé (HAS) or French National Authority for Health, France

^aIn Germany, assessment of the benefits and harms of drugs and non-drug interventions and appraisal (that is, decision-making based on the assessment) are usually conducted by separate organizations. G-BA is the highest decision-making body of the joint self-government of physicians, dentists, hospitals, and statutory health insurance (SHI) funds in Germany. The G-BA issues directives for the SHI benefit catalogue, which specifies which health care services are to be reimbursed. IQWiG is Germany’s main HTA agency. IQWiG is generally commissioned by the G-BA or the Federal Ministry of Health for the assessment of the benefits and harms of drugs and non-drug interventions. These assessments include early benefit assessments of new drugs according to the Act on the Reform of the Market for Medicinal Products (Arzneimittelneuordnungsgesetz – AMNOG). IQWiG’s early benefit assessments are solely commissioned by the G-BA. In general, IQWiG is responsible for assessments, and the G-BA for decision-making (appraisal). However, the G-BA itself also conducts assessments. More information on the G-BA, IQWiG, and AMNOG is available at www.english.g-ba.de/, www.iqwig.de/en/home.2724.html, and www.english.g-ba.de/benefitassessment/information/.

Methodology

The information presented in the Environmental Scan was obtained from a limited grey literature search and consultation with key informants from the aforementioned organizations. A survey questionnaire ^b was developed to assist in gathering the information from key informants.

First, grey literature was identified through a focused Internet search up until September 2015. This search focused on the HTA organizations’ websites and utilized publicly available documents detailing the single drug technology assessment processes of the HTA organizations. Secondly, key informants from HTA organizations were requested to validate and provide additional details on the information gathered through the literature search (presented in the form of a survey questionnaire). Please see Appendix 2 for the survey questionnaire. Key informants from the following nine HTA organizations participated in this consultation process: CDR, pCODR, INESSS, SMC, NICE, PBAC, PHARMAC, IQWiG, and G-BA (see Appendix 3).

^bAdapted from Drummond M, Neuman P, Jönsson B, Luce B, Schwartz JS, Siebert U, et al. Can we reliably benchmark health technology assessment organizations? Int J Technol Assess Health Care. 012;28(02):159-65.

Findings

Information on the single drug technology assessment process for each HTA organization is presented in the following sections: Submission Process, Clinical Review Approach, and Deliberative Process.

The following Submission Process section summarizes the information gathered. Additional details are provided in Appendix 4. Unless reference is made to a particular publication, the information was made available by the key informants.

It should be noted that there is limited information available on HAS, as the grey literature search was limited to English language publications. In the case of Germany, the G-BA commissions IQWiG for the assessment of benefits and harms of drug technology portion of the process, and the G-BA is responsible for the appraisal (that is, decision-making). Hence, most of the questions regarding the drug review process were applicable only to IQWiG, and the questions on the submission and deliberative processes were applicable only to the G-BA.

Submission Process

This section presents information on the processes related to the submission of a drug for a single drug technology assessment for each HTA organization. Information such as submission eligibility, prioritization processes, and approximate timelines are included.

Applicants Eligible to Submit for an Assessment

All of the included HTA organizations permit the manufacturer to initiate the single drug technology assessment process, with the exception of NICE. At NICE, the appraisal process only begins once the topic has been referred by the Department of Health (however, the scoping process is initiated at NICE, prior to market authorization, when a potential topic is identified by the National Institute for Health Research Horizon Scanning Research & Intelligence Centre, a relevant company, health care professionals, researchers, or patients).³ Likewise, government ministers can also request INESSS to conduct a single drug technology assessment. The CDR process allows for participating public drug plans to request an assessment.⁴ the Canada's Drug Agency pCODR process also permits provincially recognized clinician-based tumour groups and its Provincial Advisory Group (PAG) to request an assessment. At PHARMAC, anyone — including a patient, a health professional, or a supplier — can make a funding application.⁵ In the case of Germany, it is mandatory for a manufacturer to submit a dossier (evidence) to the G-BA, who then commissions IQWiG to conduct the assessment. (See Table 1, Appendix 4.)

For the purposes of consistency, the following sections of this scan will use the term “applicant” to refer to different types of organizations and groups (such as sponsors, manufacturers, suppliers, patient groups, health professionals, government ministers, and public drug plans) that are eligible to request a single drug technology assessment from these organizations.

Program Funding

Funding information was identified for all organizations except HAS. NICE, PHARMAC, SMC, IQWiG^c, and INESSS are solely publicly funded. The Canada's Drug Agency programs, CDR and pCODR, require a fee from the industry/manufacturer (applicant) per single drug technology assessment and receive public funding. (In the case of the pCODR process, fees would not apply to submissions from tumour groups or PAG; and in the case of the CDR process, fees would not apply to assessment requests from the public drug plans.)^d PBAC also requires applicants to pay a fee, although some drugs may be eligible for fee exemptions (e.g., orphan drugs or temporary supplies) or a fee waiver if the submission involves a component of public interest (such as palliative care, pediatric care, or care for aboriginal peoples), or where payment of the fee would make proceeding with the application financially unviable (for example, if the patient population is not large enough).⁷ Although applicants in Germany (that is, manufacturers) do not pay a fee for the assessment, the procedure is not fully free of charge, as the applicant may optionally pay a fee to participate in a consultation at the G-BA prior to the early benefit assessment. Canada's Drug Agency provides an option for consultation with manufacturers for a fee, known as the Scientific Advice program.^e the Canada's Drug Agency Scientific Advice Program is separate from the CDR and pCODR single drug technology assessment processes. Similar, optional consultation programs may exist at other HTA organizations. However, such optional consultation programs prior to the submission are beyond the scope of this scan. (See Table 1 in Appendix 4.)

Categories of Eligible Drugs

There are variable levels of detail regarding categories of eligibility across organizations. There is limited information available on HAS. Most organizations assess:

new drugs
new indications for existing drugs
new combinations.

Only the pCODR process focuses exclusively on oncology products. Information is summarized in Table 2, Appendix 4, and further details are presented in Table 3, Appendix 4.

Market/Regulatory Approval

Most organizations permit applicants to file submissions in advance of regulatory approval in order to reduce the time between regulatory approval and decision-making (that is, a reimbursement decision or a recommendation for a reimbursement decision). (See Table 4 in Appendix 4 for details.)

Off-Label Indications

The majority of organizations do not undertake single drug technology assessment for off-label indications. However, the pCODR process explicitly accepts submissions for single drug technology assessment for a new indication even when that indication is not under review by the regulatory agency (that is, Health Canada). pCODR states: “If a Submission or Resubmission is made for an Oncology Drug with a New Indication, for a drug that has already received market authorization in Canada, and sufficient clinical and economic evidence exists to make a Submission, it is not required that the indication currently be under review by Health Canada”.⁸ The pCODR process may accept submissions for off-label indications under the specific and limited circumstances described in Table 4 in Appendix 4.⁸ PHARMAC will consider discussing applications regarding unregistered (not approved for marketing) products and indications with the applicant.⁹ It is current practice (at PHARMAC) to undertake an assessment for off-label indications in either or both the intervention or comparator. Unless the Department of Health indicates otherwise, NICE will not publish guidance on the use of off-label drugs that are unlicensed. However, off-label indications may be considered through a separate program that provides evidence summaries on these medicines.^3,10 In the case of Germany, IQWiG (the HTA organization commissioned by G-BA to conduct the assessment) does not undertake single drug technology assessment for off-label indications. However, the Federal Ministry of Health in Germany can appoint an expert panel to assess the scientific and professional aspect of off-label use of a drug. The expert panel acts on behalf of G-BA to examine cases in which an authorized drug can be used to treat illness even though the drug is not (yet) authorized for that indication according to the German Medicinal Products Act. The panel forwards its recommendation on the most up-to-date scientific knowledge regarding the off-label use of the drug to the G-BA. (Table 4, Appendix 4.)

Prioritization

HTA organizations have a prioritization process to manage the workflow of submissions. The CDR and pCODR processes and G-BA conduct single drug technology assessments on a first-come, first-served basis. However, some of these organizations also conduct priority reviews. For the purposes of this scan, a “priority review” refers to a provision in the HTA organization’s process whereby the organization prioritizes a new drug submission for an assessment (if the drug meets certain criteria) over other drugs that were already in the queue for an assessment. CDR, pCODR, and PHARMAC reported a priority review process and criteria; however, CDR’s priority review process is currently on hold.¹¹ SMC and INESSS also reported a priority review process in order to minimize delay between market approval and a reimbursement decision. Significant clinical or economic benefits were some of the criteria to designate a submission for a priority review in these organizations. In the case of NICE, first there are criteria that help the Department of Health decide if the topic should be referred to NICE for the appraisal to begin for the drug product. Following this referral, NICE prioritizes the appraisal work to ensure that guidance can be published as soon after market authorization as possible for the maximum number of products. IQWiG did not report a prioritization process, as submission and assessment deadlines are specified by the law. PBAC does not have prioritization criteria or a priority review process. Each HTA organization’s approach for prioritization is described in Table 5, Appendix 4. There is no information available for HAS.

Submission Intake Process

All organizations accept submissions on an ongoing basis except for INESSS and IQWiG, where a fixed schedule is in place. (See Table 5, Appendix 4.)

Timelines

Information regarding the timeline between the initiation of review and issuance of the draft or final recommendation or guidance varies by organization. From assessment dossier submission or initiation of review, reported timelines are approximately 12 weeks for IQWiG, PHARMAC, and HAS; approximately 18 weeks for SMC; approximately 20 to 23 weeks for INESSS; approximately 25 weeks for the Canada's Drug Agency CDR and pCODR processes; and 35 weeks for NICE and PBAC. (See Table 6, Appendix 4 for details.)

Drug Review Process

The following section relates to the drug review process followed for a single drug technology assessment. The sections that follow describe the clinical review approach, including the processes used to evaluate a new drug, the number of experts involved and the type of expertise they provide, and the stakeholder input requirements. Economic review processes are outside the scope of this Environmental Scan. In Germany, G-BA generally delegates IQWiG to undertake the assessments; alternatively, G-BA may also commission the work to third parties.¹²

Clinical Review Approach

Assessment process for different categories of drugs

INESSS, PBAC, and IQWiG use the same methods across various categories of drugs, such as new drugs, combination products, oncology drugs, or drugs for rare diseases. CDR uses different methods to assess biosimilars and certain combination products.⁴ NICE uses a highly specialized technology process for drugs for rare diseases.¹³ Combination products, drugs for rare diseases, and oncology drugs are assessed differently by SMC. PHARMAC assesses drugs for rare diseases using their “standard methods" but in 2015 invited competitive proposals through a separate process in order to improve access to these medicines.¹⁴ Information was not available for HAS. (See Tables 3 and 6 in Appendix 4.)

Processes used for the evaluation of a new drug

Most organizations require a systematic review to be submitted by the manufacturers as part of the reimbursement dossier (SMC, NICE, PBAC, IQWiG, and HAS). Only the CDR and pCODR processes conduct their own systematic reviews. PHARMAC assesses the systematic reviews provided by the manufacturer and conducts an additional review in-house. In the case of NICE, an independent academic group critiques the systematic review provided by manufacturers. Additionally, in exceptional circumstances, such as when all published or unpublished clinical data are within the company’s control or possession, NICE may not require a systematic literature review.¹⁵ INESSS considers all of the clinical evidence submitted by the applicant, but it does not require a systematic review to be provided by the applicant and it does not conduct a systematic review in-house, either.¹⁶ (See Figure 1, and Table 7, Appendix 4.)

For the purposes of this scan, a systematic review refers to a literature review focused on a research question that aims to identify, appraise, select, and synthesize all high-quality research evidence relevant to that question.

Figure 1: Processes Used by HTA Organizations for Evaluating a New Drug

Processes Used by HTA Organizations for Evaluating a New Drug

CDR = Common Drug Review; HAS = Haute Autorité de santé; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium; SR = systematic review.
Note: This question was not applicable to G-BA. This figure is informed by Table 7 in Appendix 4.

The following are the processes used by SMC, PBAC, IQWiG, NICE, and PHARMAC to appraise the quality of evidence provided by the applicant. In the case of SMC, NICE, PBAC, and IQWiG, a systematic review must be submitted by the applicant. In the case of PHARMAC, an in-house systematic review is conducted, in addition to the systematic review that is required from the applicant. HAS also requires that a systematic review be submitted by the applicant but information on its processes to appraise the quality of the systematic review is not available.

NICE: “ERG [The Evidence Review Group] prepares a report on the clinical and cost effectiveness of the technology consistent with NICE's Guide to the methods of technology appraisal. The report is based on a review of the company’s evidence submission and advice from the ERG's clinical advisers. The ERG prepares the report in accordance with the National Institute for Health Research (NIHR) HTA programme quality criteria, the scope of work as identified in the service level agreement between the Department of Health, the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) and NICE, and will use an agreed report template. The ERG is responsible for the content and quality of the report.”³
SMC: The SMC uses clinical and economic checklists to critique the quality of provided information using critical appraisal methodology.¹⁷ Appraisal of the clinical evidence includes information on the methodology, efficacy, and safety of key studies, as appraised by the pharmacy assessor, and an overall summary of the strengths and weaknesses of the applicant’s (manufacturer’s) case, as well as any issues and uncertainties.¹⁷ Applicants must provide details of studies that provide evidence of the clinical benefits and clinical adverse effects relative to active comparator(s) used in clinical practice. Active-controlled studies are considered the most relevant. However, if active-controlled studies are not available, details of placebo-controlled or uncontrolled studies are also accepted. Placebo-controlled and uncontrolled studies can also be included if they provide evidence of relevant clinical benefits not demonstrated in active-controlled studies. Applicants must also describe any limitations of the methodology and conduct in the key clinical efficacy or clinical effectiveness studies affecting the quality of the evidence of clinical benefits and adverse effects with the medicine in the indication(s) under review. These must be relative to relevant active comparator(s), with respect to the proposed positioning of the product within the submission, if relevant.¹⁷
PBAC: Applicants (manufacturers) are required to present the clinical evidence of the proposed drug in comparison with the current treatment options, preferably in the form of systematic reviews of direct randomized controlled trials, although non-randomized studies and indirect comparisons are acceptable. The evaluation groups will then critically appraise the evidence provided, including the literature search strategies, whether the inclusion/exclusion criteria are appropriate in terms of the indication, the risk of bias assessment, and the statistical methods used in the analyses.
PHARMAC: The Pharmacology and Therapeutics Advisory Committee (PTAC) gives advice on the quality of evidence provided by the applicants. Applicants are recommended to use key clinical data sources when estimating relative treatment effects, including published randomized controlled trials (RCTs) and meta-analyses. Other possible sources include observational studies, unpublished trial data, expert opinion, and case reports. Information on the incidence and descriptions of adverse drug reactions should include data collected from observational longitudinal clinical studies, RCTs, case reports on adverse drug reactions and expected/unexpected side effects, and post-marketing surveillance data. In evaluating therapeutic effectiveness and safety, PHARMAC places greater weight on well-designed RCTs than other data sources, with particular importance given to head-to-head comparison RCTs between the proposed product and principal comparators. Where possible, applicants should critically appraise and grade the evidence using the methods described in PHARMAC’s Prescription for Pharmacoeconomic Analysis (PFPA). PHARMAC recommends the use of the Graphic Appraisal Tool for Epidemiology (or GATE) for the critical appraisal of clinical trials, and the use of the Scottish Intercollegiate Guidelines Network (or SIGN) to grade clinical evidence. This evidence is evaluated using PHARMAC’s nine “Decision Criteria.” These criteria include:
- the health needs of all eligible people within New Zealand
- the particular needs of Maori and Pacific peoples
- the availability and suitability of existing medicines, therapeutic medical devices, and related products and related items
- the clinical benefits and risks of pharmaceuticals
- the cost-effectiveness of meeting health needs by funding pharmaceuticals, rather than by using other publicly funded health and disability support services
- the budgetary impact (in terms of the pharmaceutical budget and the government's overall health budget) of any changes to the Pharmaceutical Schedule
- the direct cost to health service users
- the government's priorities for health funding, as set out in any objectives notified by the Crown to PHARMAC, in PHARMAC's Funding Agreement or elsewhere
- any other criteria that PHARMAC thinks are relevant.
PHARMAC will carry out the necessary consultation whenever it intends to take any “other criteria” into account.⁹ As of July 2016, the nine Decision Criteria will be replaced by Factors for Consideration.¹⁸
IQWiG: Applicants (manufacturers) are required to present the clinical evidence of the proposed drug in comparison with a treatment option (defined by the G-BA) in the form of a systematic review, preferably using direct RCTs, although non-randomized studies and indirect comparisons are acceptable. The dossier, which also includes full study reports for all studies conducted by the manufacturer, is assessed for completeness of evidence. IQWiG critically appraises the methods used (including search strategies) and the evidence provided in the dossier. In the report, IQWiG presents its own assessment and its own conclusions as a systematic review. This assessment is primarily based on the data presented in the dossier but may be supplemented by analysis conducted by IQWiG (for example, meta-analyses excluding irrelevant studies) and IQWiG’s own literature search and or/study selection.

NICE, SMC, PBAC, and IQWiG also have a publicly available (published) methods guideline for their appraisal process.^17,19,20

Some of the HTA organizations conduct their own in-house research in addition to the systematic review submitted by the applicant. This may take the form of a literature search, meta-analyses, or sensitivity analyses. HAS, INESSS, IQWiG, PBAC, and PHARMAC conduct a literature search. IQWiG, PBAC, and PHARMAC also conduct a meta-analysis. INESSS, IQWiG, PBAC, and PHARMAC conduct a sensitivity analysis, as well. PHARMAC conducts an in-house systematic review in addition to those submitted by the applicant. (See Table 7, Appendix 4.)

Expert Engagement

Clinical expert involvement

All organizations involve clinical experts in the clinical review process by seeking their independent expertise and opinion, and by involving clinical experts in the multidisciplinary committee that conducts the clinical review, among other activities. (See Table 8, Appendix 4.)

Stakeholder Input

Submissions of evidence from stakeholders

Most organizations encourage or require submissions of evidence from other stakeholders in addition to the applicant’s submission (CDR, pCODR, INESSS, NICE, SMC, IQWiG). All of these HTA organizations permit submissions from patient advocacy groups. Only CDR and pCODR, NICE, INESSS, and IQWiG permit submissions from individual patients (although in PHARMAC’s case, a patient may be the applicant). However, CDR and pCODR accepts input from individual patients and caregivers only in the limited instances where no Canadian patient group exists.²¹ pCODR also accepts input from the PAG and, as of February 1, 2016, a pilot initiative was launched that would accept input from registered clinicians.²² IQWiG sends a questionnaire via the G-BA to patients, asking for the following perspectives: general description of the disease, currently available treatment options, therapeutic need, relevant end points, and relevant subgroups. Responses were mixed on evidence submission from clinicians or professional societies, manufacturers, and drug reimbursement bodies. However, it should be noted that manufacturers are eligible to submit or request an assessment at most HTA organizations (Table 1, Appendix 4), and they submit evidence at the time of requesting an assessment. PBAC and PHARMAC do not accept submissions from other stakeholders. Information on HAS is not available. (See Table 9, Appendix 4.)

Stakeholder comments on the draft report

CDR, pCODR, NICE, SMC, PBAC, and PHARMAC permit stakeholders to comment on the draft report, and all six organizations permit the manufacturer to comment. The pCODR process also permits feedback from patient advocacy groups and the PAG. Additionally, as of February 1, 2016, pCODR accepts feedback from registered clinicians who provided input at the start of the submission.²² At NICE, the manufacturer can comment on the accuracy of the report before the committee meeting. PHARMAC invites comments from any interested party. In the case of Germany, the results of the benefit assessment are published online by G-BA, and pharmaceutical companies, federations, and experts are able to submit written and verbal statements on the results. Only INESSS does not permit stakeholders to comment on the draft report. (See Table 10, Appendix 4.)

Deliberative Process

The deliberative process for single drug technology assessments within each HTA organization involves the consideration of evidence by committees which then develop recommendation decisions. The following section addresses the committee structure, deliberative framework, public or patient involvement, availability of information concerning decision-making, the level and nature of stakeholder input allowed on the draft recommendations or guidance, and the appeal process for the various HTA organizations.

Committee Structure

Size and composition of committees

Committee sizes range from 12 members (PHARMAC) up to 30 members (SMC). (See Table 11, Appendix 4.)

There is variation across the HTA organizations considered with respect to the experts who comprise the committees. Most organizations included physicians and pharmacists in the committee; there is less consistency across the different organizations with regard to nurses, researchers, patients, managers, the public, and ethicists.

Deliberation processes for different drug types

INESSS has a separate program specified for oncology (wherein the standing committee works for efficacy and safety review in collaboration with an oncology committee); however, the processes for other organizations are common across drug types. (See Table 12, Appendix 4.)

Deliberation

Deliberative framework documentation

All organizations use a documented, fully published deliberative framework for decision-making, with the exception of HAS, for which there is no information available. (See Table 13, Appendix 4.)

Public involvement and availability of information relating to decisions

The committee meetings of most HTA organizations, where single drug technology assessment deliberations take place, are not accessible to the public. NICE permits pre-registered interested parties to observe the public part of the committee meeting, but the remainder of the meeting takes place in private.³ In cases where it might not be possible to avoid referring to confidential or commercially sensitive information, meetings may be entirely closed.³ For SMC, the discussion is public, but a vote is held in private. For G-BA, deliberations of the plenum are usually public, but the final deliberation is always private. All organizations made supporting documents publicly available. (See Table 14, Appendix 4.)

Conditional reimbursement and coverage-with-evidence development schemes

For the purposes of this scan, “conditional reimbursement” refers to the practice of making recommendations that are contingent upon a specific set of criteria being met.

CDR and pCODR apply the following categories for recommending conditional reimbursement (see also Table 15, Appendix 4):

The drug expert committees (the Canada's Drug Agency Canadian Drug Expert Committee [CDEC] and the pCODR Expert Review Committee [pERC]) may recommend that a drug be reimbursed with clinical criteria and/or conditions, in addition to recommending that a drug be reimbursed or not be reimbursed. The drug expert committees recommend clinical criteria to provide additional characteristics to identify the patient population for whom reimbursement is being recommended. These are typically provided in addition to any clinical characteristics specified in the Health Canada–approved indication. CDEC recommends conditions to provide guidance to the participating jurisdictions on implementation and operational issues related to the drug under review. Such conditions may include a recommendation to reimburse at a reduced price or listing the drug in a similar manner to one or more appropriate comparators, among others.²³

For INESSS, the following categories are applied:

Products recommended without restriction are listed in the “Regular Drug” section of the Régime général d’assurance medicaments (RGAM) Formulary.
Products with restrictions are listed in the “Exception Drug” section of the RGAM Formulary and of the Drug Formulary for Institutions, specifying the criteria for use recognized by INESSS. The criteria used for reimbursement are also recommended by the committee.
Products with good efficacy and security are recommended to be listed with conditions if negotiations/agreements can improve cost-effectiveness.
Products can be recommended not to be listed.

For NICE, the overall goal is to maximize health within limited resources; therefore, the committee can make recommendations:

for a product that reflects its market authorization
for a product that is more restrictive than its market authorization (e.g., based on patient subgroups or employing treatment-starting, continuation, and stopping rules)
that a product be used only in research
that a product be recommended, but that further research must also be completed
that a product not be recommended.

It should be noted that the NICE committee is not able to make recommendations to the National Health Services (NHS) on the pricing of technologies but can consider a patient access scheme subject to the arrangements detailed in the technology appraisal process guide(s).

For PBAC, medicines and medicinal products can be listed as:

“unrestricted benefits, which have no restrictions on their therapeutic uses for the purposes of subsidy”¹⁹
“restricted benefits, which can only be prescribed for specific therapeutic uses”¹⁹
“authority required (STREAMLINED) benefits, which are restricted and require the recording of a streamlined authority code”¹⁹
“authority required benefits, which are restricted and can only be prescribed with previous approval from the Australian Government Department of Human Services or the Australian Government Department of Veterans’ Affairs.”¹⁹

The requested restriction will be assessed based on clinical evidence, such as the patient population, stage of disease, and line of treatment. The Australian Department of Health will negotiate the wording of the restriction and the price of the drug with the sponsor. Under certain circumstances, PBAC makes recommendations on conditional reimbursement, such as risk-sharing arrangements (which are agreements between the sponsor and the government to cap the maximum financial expenditure to submissions’ estimates, with a 100% rebate thereafter). Additionally, a “pay-for-performance” arrangement can be made whereby the sponsor rebates to the government the cost of treatment based on patients’ responses.

For PHARMAC, PTAC can recommend drugs subject to certain conditions, such as for certain subpopulations, entry and exit criteria, or a drug’s pricing relationship to other products in the same class. Products may be funded with restrictions on prescriber type, or with requirement for prior approval for individual patients according to clinical criteria, including indications and response to treatment. Prices are agreed upon in negotiated contracts that may feature rebates, budget caps, time limits, and any other commercial terms.

For the G-BA, there is no conditional reimbursement regulation for drugs. However, the decision on additional benefit (while the drug is already in the market) may be limited by time (typically one to three years) in order to re-review the evidence when additional studies are finished.

Some of the organizations operate coverage-with-evidence development schemes in order to address uncertainty regarding payment decisions. INESSS is in the early stages of introducing such programs. Currently, CDR and pCODR, and the G-BA, do not have a provision for such schemes. NICE can recommend a technology as an option if research is also being conducted and in a case where evidence is weak or uncertain.²⁴ PBAC has recently recommended crizotinib for the treatment of patients with anaplastic lymphoma kinase-positive advanced non–small cell lung cancer under a managed entry scheme as a mechanism to address the uncertainty related to the magnitude of clinical benefit, while providing early access to those patients for whom there is a high clinical need. This framework includes a mechanism for payment of a rebate, with interest to the Australian authorities should crizotinib fail to deliver on its claimed benefits when the data become available.²⁵

Stakeholder Input in Deliberative Process

Stakeholder involvement with draft recommendations/guidance

The majority of HTA organizations permit stakeholders to comment on draft recommendations or guidance; this is largely restricted to the manufacturer. NICE has implemented a specific team that supports and develops public involvement across NICE's work program; a Public Involvement Programme’s — or PIP — Public Involvement Adviser is assigned to each appraisal and supports patient and caregiver consultee organizations.³ Additionally, NICE recommendations have public consultation, allowing manufacturers, patient organizations, research organizations, and anyone else to comment. PHARMAC encourages detailed comment on draft recommendations. pCODR allows stakeholders (patient groups, clinicians, and the PAG) to provide feedback on a pERC initial recommendation. The G-BA, in addition to permitting payers and patient groups to comment, also allows physicians and hospital representatives to comment. (See Table 16, Appendix 4.)

Appeals or procedural reviews

The right for stakeholders to appeal against a recommendation or ruling varies by organization. Appeals are permitted by NICE and SMC (both convene a separate appeals panel); resubmissions are permitted by INESSS, PHARMAC, and PBAC (wherein new evidence may be submitted); provisions for a review is available for the CDR and pCODR processes (if there is concern that procedures have not been properly followed); and appeals are not permitted by G-BA. Information is not available for HAS. Following are the individual organizational procedures concerning the appeal and resubmission processes:

NICE: “…an Appeal Panel consisting of the following 5 members is formed:
- Appeal Panel Chair
- a non-executive director of NICE
- a person with experience of the life sciences industry
- a lay member who may have experience of being a patient or carer or member of an organisation that represents patients and carers
- a person with experience of the NHS.”³
SMC: “If a medicine has been considered by SMC more than once (e.g. a submission and resubmission) then the IRP [Independent review Panel] timescale will refer to the most recent submission…The panel asked to undertake the independent review will be appointed by the SMC, on advice from the Chairman and Secretariat, and will be constituted as follows: Chair, who may be appointed from either of the categories…3 members (where possible) appointed from the SMC/NDC background (who, by reason of absence, have not previously been involved in the particular submission, including former members of SMC or NDC); 4 members (where possible) appointed from Scottish NHS Board Board Area Drug and Therapeutics Committees (or their successors/equivalents) and/or other respected experts in the relevant scientific field, who need not necessarily be working in Scotland.”²⁶
INESSS: The manufacturer can submit new information in writing within 10 working days following the decision by the minister in the event of a first refusal or a first listing as an exception medication. In this case, this information will be evaluated by the Comité scientifique d'évaluation des médicaments aux fins d'inscription (CSEMI) of INESSS as part of the next update, even if the deadline for the presentation of listing applications has passed.
PBAC: PBAC recommendations are not subject to appeal or merits review. Applicants may make a resubmission to PBAC if a previous application has failed to result in a listing recommendation, or if the sponsor wishes to broaden the subsidized indications or vary the listing restriction. In a case where no new evidence or analysis is available, a sponsor whose application for the listing of a medicine or new indication has been rejected by PBAC may seek an independent review, conducted by a single expert reviewer, whose findings, together with any comments by the sponsor, would be presented to PBAC for consideration.
CADTH: CDR and pCODR include provisions for reconsideration and procedural review, respectively. In the case of CDR, a manufacturer may file for reconsideration if
- “CDR and/or the Canadian Drug Expert Committee failed to act fairly and in accordance with its procedures in conducting the review, and/or
- the Canadian Drug Expert Committee recommendation is not supported by the evidence that had been submitted or the evidence identified in the CDR review report(s).”⁴
  
  In the case of pCODR, a procedural review may be requested “on the basis that:
- pCODR failed to act in accordance with its procedures in conducting the review [as described in the pCODR Procedures]⁸
- pERC failed to apply its deliberative framework in formulating its recommendation [as outlined in the pERC Deliberative Framework²⁷].”⁸

^cIQWiG is funded by levies for in-patient and outpatient health care services — that is, contributions from persons insured in statutory health insurance, or SHI, funds (applies to nearly 90% of the population).
^dIn the case of the CDR process, all submissions and resubmissions that receive a Health Canada Notice of Compliance or Notice of Compliance with conditions on or after September 1, 2014 are subject to an application fee.⁴ In the case of the pCODR process, an application fee will apply to a drug manufacturer submitting an application for a submission or resubmission of oncology drugs/indications received on or after April 1, 2015.⁶
^eMore information on the Canada's Drug Agency Scientific Advice Program is available at /scientific-advice.

Conclusions

Canada’s single drug technology assessment processes are unique in comparison with the other international organizations in that Canada's Drug Agency has two separate assessment processes — one for oncology drugs (pCODR) and the other for non-oncology drugs (CDR). Whereas both the G-BA and IQWiG are responsible for portions of the single drug technology assessment process in Germany, the other jurisdictions included in this scan have only one HTA organization responsible for the assessment of drugs.

With regard to the submission process, the CDR and pCODR processes are similar to those of the other HTA organizations in that they permit manufacturers to initiate the assessment process — with the exception of NICE, where topics are typically identified by the NIHR Horizon Scanning Centre and the appraisal process begins once a topic has been referred by Department of Health. PHARMAC permits health care service providers and public drug plans to request an assessment, which is similar to the pCODR and CDR processes, respectively. The CDR and pCODR processes are the only ones that utilize both applicant fees and public funding, whereas INESSS, NICE, PHARMAC, SMC, and IQWiG are solely publicly funded, and PBAC requires the applicant to pay a fee. Regarding the categories of drugs that are eligible for assessment within each organization, most HTAs assess new drugs, drugs with new indications, and new combinations. Similar to the CDR and pCODR processes, most organizations conduct the assessments on a first-come, first-served basis. However, some of these organizations also conduct a priority review of submissions when the drugs meet certain criteria. The approximate timelines for completion of the single drug technology assessment process varies between organizations. The timeline between the initiation of review and issuance of the draft/final recommendation or guidance varies by organization; that is, approximately 12 weeks for IQWiG, PHARMAC, and HAS; approximately 18 weeks for SMC; approximately 20 to 23 weeks for INESSS; approximately 25 weeks for the Canada's Drug Agency CDR and pCODR processes; and 35 weeks for NICE and PBAC.

The review processes within each HTA organization vary. For evaluating the comparative clinical benefit of a drug, CDR and pCODR conduct systematic reviews in-house, whereas most other organizations require a systematic review to be submitted by the applicant (SMC, NICE, PBAC, IQWiG, HAS). In the case of PHARMAC, a systematic review is conducted by both the HTA organization and the applicant. INESSS does not require a systematic review to be provided by the applicant, but rather considers all of the clinical evidence submitted to support the application. This is supplemented by INESSS conducting an in-house literature search. HAS, IQWiG, INESSS, PHARMAC, and PBAC may conduct additional in-house research to support the drug review process, and such research may take the form of literature searches, meta-analyses, and sensitivity analyses.

Every organization involves clinical experts in their review process. Stakeholder involvement in the drug review process varies among organizations. All except PBAC and PHARMAC require or allow submission of evidence from patient advocacy groups, and some permit evidence from individual patients, professional societies, and manufacturers. The CDR and pCODR processes also allow stakeholders, such as manufacturers, to comment on draft or final reports resulting from the drug review process, which is similar to G-BA, NICE, PBAC, PHARMAC, and SMC.

Regarding the deliberation process, committee size and composition varies among HTA organizations. CDR and pCODR are comparable to other organizations in having physician and pharmacists as committee members, but there is less consistency across organizations concerning other members, such as nurses, researchers, patients, the public, and ethicists. All organizations have published deliberative frameworks for decision-making. Most HTA organizations, including Canada's Drug Agency, allow stakeholders, such as the manufacturer, to comment on draft recommendations. Most HTA organizations provide an opportunity, in one form or another, for reconsideration of the review process or the draft recommendation or guidance.

This Environmental Scan illustrates that the Canada's Drug Agency HTA processes are similar to, as well as different from, other comparable HTA organizations. Differences in these processes could also be a result of differences in the health care systems in the jurisdictions where the HTA organizations operate. The type of processes applied by an HTA organization will likely have implications on the organization’s resources and have an impact on its stakeholders. For example, Canada's Drug Agency relies on conducting in-house systematic reviews, which could be a more resource-intensive process compared with the “appraisal” type reviews conducted at other HTA organizations (that is, where the applicant submits the systematic review). Further assessment of the value and impact of varying processes to Canada's Drug Agency stakeholders will potentially help Canada's Drug Agency identify further efficiencies — that is, by identifying areas where resources can be streamlined, while maintaining the quality standards of its programs.

References

Drummond M, Neumann P, Jönsson B, Luce B, Schwartz JS, Siebert U, et al. Can we reliably benchmark health technology assessment organizations? Int J Technol Assess Health Care. 2012;28(02):159-65.
CADTH.ca [Internet]. Ottawa: Canada's Drug Agency. About Canada's Drug Agency; 2015 [cited 2015 Dec 7]. Available from: /about-cadth
Guide to the processes of technology appraisal [Internet]. London: National Institute for Health and Care Excellence; 2014 Sep 2. [cited 2015 May 12]. (Process and methods guides). Available from: https://www.nice.org.uk/process/pmg19/chapter/acknowledgements
CADTH.ca [Internet]. Ottawa: Canada's Drug Agency. Procedure for the Canada's Drug Agency Common Drug Review; 2014 Aug [cited 2015 May 12]. Available from:/media/cdr/process/CDR_Procedure.pdf
PHARMAC [Internet]. Wellington, New Zealand: PHARMAC. New funding applications; 2016 [cited 2016 Jun 7]. Available from: https://www.pharmac.govt.nz/medicines/how-medicines-are-funded/new-funding-applications/
Guidelines for manufacturers on application fees for the Canada's Drug Agency pan-Canadian oncology drug review [Internet]. Ottawa: Canada's Drug Agency; 2015 Apr. [cited 2015 Dec 7]. Available from: /sites/default/files/pcodr/Communications/Guidelines%20for%20Manufacturers%20on%20Application%20Fees%20for%20pCODR_final.pdf
Part 5: exemptions and waivers. In: National health (pharmaceuticals and vaccines - cost recovery) regulations 2009 [Internet]. Canberra, Australia: Australian Government; 2009 [cited 2015 Dec 7]. Available from: https://www.comlaw.gov.au/Details/F2009L04013
pan-Canadian Oncology Drug Review (pCODR). pCODR procedures [Internet]. Ottawa: Canada's Drug Agency; 2016. [cited 2016 Aug 7]. Available from: /sites/default/files/pcodr/pCODR's%20Drug%20Review%20Process/pcodr-procedures.pdf
Guidelines for funding applications to PHARMAC [Internet]. Wellington, New Zealand: PHARMAC; 2015. [cited 2015 May 12]. Available from:http://www.pharmac.health.nz/assets/funding-applications-guidelines.pdf
Evidence summaries: unlicensed and off-label medicines - integrated process statement [Internet]. London: National Institute for Health and Care Excellence; 2013 May. [cited 2016 Jun 7]. (Process and methods guides; no. 14). Available from: https://www.nice.org.uk/article/pmg14/chapter/1%20Introduction Last updated: July 2014.
CDR update -- Issue 109 [Internet]. Ottawa: Canada's Drug Agency; 2015 Feb 26. [cited 2016 Jan 6]. (CDR update newsletter). Available from: /cdr-update-issue-109
The benefit assessment of pharmaceuticals in accordance with the German Social Code, book five (SGB V), section 35a [Internet]. Berlin, Germany: Gemeinsamer Bundesausschuss; 2015. [cited 2015 May 12]. Available from: http://www.english.g-ba.de/benefitassessment/information/
NICE highly specialised technologies guidance [Internet]. London: National Institute for Health and Care Excellence; 2013. [cited 2016 Jun 7]. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-highly-specialised-technologies-guidance
PHARMAC in negotiations over rare disorders medicines [Internet]. Wellington, New Zealand: PHARMAC; 2015 Feb 23. [cited 2015 May 12]. Available from: http://www.pharmac.health.nz/news/media-2015-02-26-pharmac-in-negotiations-rare-disorders/
Single technology appraisal: user guide for company evidence submission template [Internet]. London: National Institute for Health and Care Excellence; 2015 Jan. [cited 2016 Jun 7]. (Process and methods guides; no. 24). Available from: https://www.nice.org.uk/article/pmg24/chapter/Instructions%20for%20companies
Selecting medication for coverage in Quebec: a responsible, transparent process. Information on the scientific evaluation process of medication [Internet]. Quebec (QC): Conseil du médicament; 2007 Feb. [cited 2015 May 12]. Available from: http://www.inesss.qc.ca/fileadmin/doc/INESSS/DocuAdmin/Selecting-medication-coverage.pdf
Submission guidance and templates for submission [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2015. [cited 2015 May 12]. Available from: https://www.scottishmedicines.org.uk/Submission_Process/Submission_guidance_and_forms/Submission_process_-_Submission_guidance_and_templates_for_submission
Changing the way we make decisions [Internet]. Wellington, New Zealand: PHARMAC; 2016. [cited 2016 Jun 7]. Available from: https://www.pharmac.govt.nz/medicines/how-medicines-are-funded/factors-for-consideration/
Pharmaceutical Benefits Advisory Committee. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee [Internet]. Version 4.5. Canberra, Australia: Australian Government, Department of Health; 2015 Jul. [cited 2015 Oct 28]. Available from: http://www.pbac.pbs.gov.au/content/information/printable-files/pbacg-book.pdf
General methods [Internet]. Version 4.2. Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG); 2015 Apr 22. 229 p. [cited 2016 Jun 7]. Available from: https://www.iqwig.de/download/IQWiG_General_Methods_Version_%204-2.pdf
Expanding the Canada's Drug Agency Drug Review process to receive patient input submissions from individual patients and caregivers. CDR update - Issue 113 [Internet]. Ottawa: Canada's Drug Agency; 2015 Oct 6. [cited 2015 Dec 16]. (CDR update newsletter). Available from: /cdr-update-issue-113
New pilot initiative: increasing opportunities for clinician input and feedback in Canada's Drug Agency pan-Canadian Oncology Drug Review process [Internet]. Ottawa: Canada's Drug Agency; 2016 Feb 1. [cited 2016 Jun 7]. Available from:/sites/default/files/pcodr/Communications/pcodr-communication-51.pdf
Recommendation framework for Canada's Drug Agency Common Drug Review and pan-Canadian Oncology Drug Review programs: guidance for the Canada's Drug Agency Drug Expert Committees [Internet]. Ottawa: Canada's Drug Agency; 2016 Mar. [cited 2016 Jul 12]. Available from: /media/cdr/templates/pre-sub-phase/CDR_pCODR_recommendations_framework.pdf
Guide to the methods of technology appraisal [Internet]. London: National Institute for Health and Care Excellence; 2013 Apr 4. [cited 2015 May 12]. (Process and methods guides). Available from: https://www.nice.org.uk/process/pmg9/chapter/foreword
The Pharmaceutical Benefits Scheme [Internet]. Canberra, Australia: Australian Government, Department of Health. Crizotinib; 200 mg capsule, 60 and 250 mg capsule, 60; Xalkori®; 2015 Mar 6 [cited 2015 Oct 28]. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2014-11/crizotinib-psd-11-2014
Procedure for reassessment of SMC decisions - overview [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2016. [cited 2016 Jun 7]. Available from: https://www.scottishmedicines.org.uk/About_SMC/Policy_statements/Procedure_for_Reassessment_of_SMC_Decisions_Overview
pan-Canadian Oncology Drug Review (pCODR). pCODR Expert Review Committee deliberative framework [Internet]. Ottawa: Canada's Drug Agency; 2016 Mar. [cited 2016 Jun 7]. Available from: /sites/default/files/pcodr/The%20pCODR%20Expert%20Review%20Committee%20(pERC)/pcodr_perc_deliberative_frame.pdf
Working with SMC - a guide for manufacturers [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2014 Jul. [cited 2015 May 12]. Available from: https://www.scottishmedicines.org.uk/files/submissionprocess/Working_with_SMC_July_2014.pdf
Pharmaceutical Benefits Scheme [Internet]. Canberra, Australia: Australian Government Department of Health. Cost recovery, fees and charges - frequently asked questions; 2015 May 27 [cited 2015 Dec 7]. Available from: http://www.pbs.gov.au/info/industry/listing/elements/fees-and-charges
La commission de la transparence (CT). Évaluation des médicaments en vue de leur remboursement [Evaluation of medicines for reimbursement] [Internet]. Paris: Haute Autorité de Santé; 2014 Mar. [cited 2015 Oct 28]. Available from: http://www.has-sante.fr/portail/upload/docs/application/pdf/2014-03/commission_transparence_2014_v4.pdf
Guidelines to manufacturers on medicines without SMC remit [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2016. [cited 2016 Jun 7]. Available from: https://www.scottishmedicines.org.uk/Submission_Process/Submission_guidance_and_forms/Templates-Guidance-for-Submission
The Pharmaceutical Benefits Advisory Committee [Internet]. Canberra, Australia: Australian Government, Department of Health. Types of submissions; 2015 Jul 1 [cited 2016 Jun 7]. Available from: https://pbac.pbs.gov.au/information/types-of-submissions.html
Regelment interieur de la Commission de la Transparence [Internet]. Paris: Haute Autorité de santé (HAS); 2015 Feb 18. [cited 2015 May 12]. Available from: http://www.has-sante.fr/portail/upload/docs/application/pdf/2013-01/ri_ct_version_07112012_vf_f.pdf
The drug evaluation process [Internet]. Quebec (QC): INESSS; 2015. [cited 2015 May 12]. Available from: http://www.inesss.qc.ca/fileadmin/doc/INESSS/Inscription_medicaments/Processus/Evaluation-medic_EN_28082013.pdf
Pricing & reimbursement of drugs and HTA policies in France [Internet]. Paris: Haute Autorité de Santé; 2014 Mar. [cited 2015 Oct 28]. Available from: http://www.has-sante.fr/portail/upload/docs/application/pdf/2014-03/pricing_reimbursement_of_drugs_and_hta_policies_in_france.pdf
Submission guidance and forms: clinical experts [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2015. [cited 2015 Feb 11]. Available from: https://www.scottishmedicines.org.uk/Submission_Process/Submission_guidance_and_forms/Clinical_Experts
Canadian Drug Expert Committee terms of reference [Internet]. Ottawa: Canada's Drug Agency; 2014. [cited 2015 May 12]. Available from: /media/corporate/corp_committees/CDEC_TOR_e.pdf
Technology appraisal committee [Internet]. London: National Institute for Health and Care Excellence; 2015. [cited 2015 Oct 28]. Available from: http://www.nice.org.uk/get-involved/meetings-in-public/technology-appraisal-committee
Consumer Advisory Committee (CAC) [Internet]. Wellington, New Zealand: PHARMAC; 2015 Feb 23. [cited 2015 May 12]. Available from: http://www.pharmac.health.nz/about/committees/
pCODR Expert Review Committee terms of reference [Internet]. Toronto (ON): pan-Canadian Oncology Drug Review (pCODR); 2014 May. [cited 2016 Jun 7]. Available from: /sites/default/files/pcodr/The%20pCODR%20Expert%20Review%20Committee%20%28pERC%29/pcodr_expertreviewcom_tor.pdf
Working with SMC - a guide for manufacturers [Internet]. Glasgow, United Kingdom: Scottish Medicines Consortium; 2015 Aug. [cited 2015 Oct 28]. Available from: https://www.scottishmedicines.org.uk/files/submissionprocess/Working_with_SMC_July_2014.pdf
Decision criteria consultation [Internet]. Wellington, New Zealand: PHARMAC; 2015 Mar 20. [cited 2015 May 12]. Available from: http://www.pharmac.health.nz/about/operating-policies-and-procedures/decision-criteria-consultation/
des Gemeinsamen Bundesausschusses. Verfahrensordnung [Rules of procedure] [Internet]. Berlin, Germany: Gemeinsamen Bundesausschusses; 2015 Apr 16. Report No.: 84a. [cited 2015 Oct 28]. Available from: https://www.g-ba.de/downloads/62-492-1002/VerfO_2014-12-18_iK-2015-04-16.pdf

Appendix 1: Overview of HTA Organizations

Common Drug Review (CDR): Canada's Drug Agency CDR is a pan-Canadian process that evaluates clinical, economic, and patient evidence of drugs in order to make reimbursement recommendations to federal, provincial, and territorial public drug plans (with the exception of Quebec). For more information:/cdr.
Pan Canadian Oncology Drug Review (pCODR): Canada's Drug Agency pCODR process evaluates clinical, economic, and patient evidence of oncology drugs in order to make reimbursement recommendations to federal, provincial, and territorial public drug plans (with the exception of Quebec), as well as to provincial cancer agencies. For more information: /pcodr/about-pcodr.
Institut national d'excellence en santé et en services sociaux (INESSS): INESSS assesses clinical and cost advantages of the technologies, medications, and interventions used in health care in order to make recommendations on the adoption, use, and coverage of these products to the Quebec public plan. For more information: https://www.inesss.qc.ca/en/about-us/about-the-institut.html.
National Institute for Health and Care Excellence (NICE): Through single technology appraisal, NICE provides guidance to the National Health Service (NHS) in England on both the clinical and cost-effectiveness of health technologies. These appraisals are carried out following referral from the Department of Health. For more information: https://www.nice.org.uk/article/pmg19/chapter/foreword.
Scottish Medicines Consortium (SMC): SMC analyzes information on the health benefits and cost-effectiveness of medicines, supplied by manufacturers, in order to provide advice to NHS Scotland regarding which medicines provide good value for money. For more information: https://www.scottishmedicines.org.uk/About_SMC/What_we_do/index.
Pharmaceutical Benefits Advisory Committee (PBAC): PBAC, an independent committee appointed by the Australian government, recommends medicines to be listed on the Pharmaceutical Benefits Scheme after considering the medicine’s clinical and cost-effectiveness, and safety. For more information: http://www.pbs.gov.au/info/industry/listing/participants/pbac.
Pharmaceutical Management Agency (PHARMAC): PHARMAC is a New Zealand government agency that decides which medicines are to be subsidized by the District Health Boards in order to provide the best health outcomes within the funding provided. For more information: http://www.pharmac.health.nz/assets/factsheet-01-introduction-to-pharmac.pd.
Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG): IQWiG is an independent institute commissioned by Germany’s Federal Joint Committee (G-BA) to examine the added benefits of a medicine and to make recommendations. IQWiG does not decide whether a medicine will be reimbursed by health insurance funds, as this is the decision of the G-BA. For more information: https://www.iqwig.de/en/about-us/responsibilities-and-objectives-of-iqwig/contracting-agencies-and-funding.2951.html.
Gemeinsamer Bundesausschuss (G-BA): The Federal Joint Committee, or G-BA, is the highest decision-making body of the joint self-government of physicians, dentists, hospitals, and health insurance funds in Germany. It issues directives for the benefit catalogue of the statutory health insurance funds (GKV) for more than 70 million insured persons and thus specifies which services in medical care are reimbursed by the GKV. For more information: http://www.english.g-ba.de/.
Haute Autorité de santé (HAS): HAS is an independent public body, set up by the French government, which performs a wide range of activities. The Transparency Committee assesses medicines and makes recommendations on their inclusion on the list of reimbursable drugs. For more information: http://www.has-sante.fr/portail/jcms/c_412210/en/commission-de-la-transparence.

Appendix 2: Review Approaches for Single Drug Technology Assessments Across HTA Organizations — Survey Questionnaire

DRUG SUBMISSION INTAKE PROCESS AND TIMELINE

Scope of Review, Prioritization, and Timeline

Who is eligible to submit for an assessment of a drug? (Please provide a list; e.g., manufacturer, public drug plan, other health care service provider such as hospital associations.)
Do applicants pay a fee for the drug assessment conducted by the HTA organization or is the program (within the organization) solely publicly funded?
□ Applicants pay the fee □ Solely publicly funded □ Both
What categories of drugs are eligible for drug assessment by the HTA organization?^a (Please provide a list; e.g., new drugs, new combination products, new indication, oncology products, biosimilars, and therapeutic vaccines).

What categories of drugs are explicitly excluded? (Please provide a list.)
Does the HTA organization conduct assessment of drugs that have not yet received market authorization or regulatory approval in the country/region? (Please provide details.)
Does the HTA organization conduct drug assessment for off-label indications?
□ Yes □ No

If yes, please describe.
Does the organization have a prioritization process (for review) when an eligible drug is submitted for assessment?^a
□ Yes □ No

If yes, what are the criteria for prioritization? (Please provide details.)
Is the intake process for drug assessment submissions fixed (i.e., submissions are accepted at certain times of the year only) or are submissions accepted on an ongoing basis?
□ Fixed □ Ongoing
Does your organization have different assessment methods for following categories of drugs?
Combination products     □ Yes     □ No     □ Not Applicable
Biosimilars     □ Yes     □ No     □ Not Applicable
Drugs for rare disease     □ Yes     □ No     □ Not Applicable
Oncology     □ Yes     □ No     □ Not Applicable
What are the approximate timelines for conducting drug assessment at your organization; i.e., timeline between initiation of review and issuance of final recommendation/guidance?^a (Please provide details.)

DRUG REVIEW PROCESS

Clinical Review Approach

Please select which of the following best describes the process used by the HTA organization for the evaluation of a new drug.^a
□ Option A: Systematic review is not a requirement
□ Option B: Systematic review conducted by the HTA organization
□ Option C: Systematic review submitted by the applicant
□ Option D: Option B and C

Note: A systematic review is a literature review focused on a research question that tries to identify, appraise, select, and synthesize all high-quality research evidence relevant to that question.

Please answer the following question ONLY if you selected Option C or D.

a. What is the organization’s process to appraise the quality of the systematic review conducted by the applicant, such as the process to appraise the methodological quality of the SR? (Please provide details.)

b. Does your organization have a guideline (methods) for such appraisals (of the systematic review conducted by the applicant)?
□ Yes □ No

If yes, is the guideline publicly available?
□ Yes □ No

If yes, please provide hyperlink and summarize.

c. Does your organization conduct additional in-house research (i.e., in addition to the SRs submitted by the applicant)?
□ Yes □ No

If yes, does it typically include the one or more of the following? (Please select all that apply.)
□ Literature search
□ Meta-analysis
□ Sensitivity analysis

Expert Engagement

Are clinical experts involved in the drug assessment (clinical review)?^a □ Yes □ No

If yes, what stages of the review are they involved in and what is the level of their involvement? (Please provide details.)

Stakeholder Input

Does the organization’s process encourage or require submissions of evidence from stakeholders (other than the applicant)?^a
□ Yes     □ No

If yes, which stakeholders are allowed to provide such submissions? (Please select all that apply.)
□ Patient advocacy groups
□ Individual patients
□ Clinicians or professional societies (of health care professionals and/or regulatory bodies)
□ Manufacturers
□ Drug reimbursement bodies (i.e,. public drug plans)
Does the organization’s process allow stakeholders to comment on the draft report?^a
□ Yes     □ No

If yes, which of the following stakeholders are allowed to comment? (Select all that apply.)
□ Manufacturer
□ Payer
□ Patient group
□ General public

DELIBERATIVE AND RECOMMENDATION/GUIDANCE FRAMEWORKS

Committee Structure

How many members comprise the committee?
Which is the following experts and/or representatives comprise the committee?^a (Please select all that apply.)
□ Physician
□ Pharmacist
□ Nurses
□ Patients
□ Public representative
□ Ethicist
□ (Research) Methodological experts

Are other members brought in for the deliberation (e.g., specialists with expertise specific to disease/therapeutic area)?
Does the same committee make recommendations for all drug types (or drug classes) or are there separate process/programs for different drug types (e.g., separate program for oncology drugs)? (Please provide details.)

Deliberative Framework

Does the committee have a well-documented deliberative framework to make recommendations/decisions?
□ Yes     □ No

If yes, is the information publicly available?
□ Yes     □ No

If yes, please provide a hyperlink to the document.
Are the meetings of the committee held in public?^a
□ Yes □ No
Are the supporting information and the basis of the recommendations made publicly available?^a
Does the committee recommend, or operate, conditional reimbursement?^a (Please provide details.)
Note: For the purpose of this survey, “conditional reimbursement” refers to the practice of making recommendations that are contingent upon a specific set of criteria being met. Examples may include the following: clinical criteria used to identify a sub-population of patients within the approved indication, pricing considerations (e.g., the need for a reduction in price), or reimbursement limits (e.g., starting and stopping criteria).
□ Yes     □ No

If yes, please provide details.
Does the committee recommend, or operate, coverage-with-evidence-development schemes? (Please provide details.)
□ Yes □ No

If yes, please provide details.

Stakeholder Input

Does the organization’s process allow stakeholders to comment on the draft recommendation/guidance?^a
□ Yes     □ No

If yes, which of the following stakeholders are allowed to comment? (Please select all that apply.)
□ Manufacturer
□ Payer
□ Patient groups
□ General public
Does the organization’s process allow stakeholders to appeal against recommendations/decisions?^a

If yes, which of the following stakeholders are allowed to appeal? (Select all that apply.)
□ Manufacturer
□ Payer
□ Patient groups
□ General public

If yes, are such appeals considered by the same committee or a different committee? (Please provide details.)

HTA = health technology assessment; SR = systematic review.
^a Questions adapted from: Drummond M, Neuman P, Jönsson B, Luce B, Schwartz JS, Siebert U, et al. Can we reliably benchmark health technology assessment organizations? Int J Technol Assess Health Care. 2012;28(02):159-65.

Appendix 3: List of Key Informants

Canada's Drug Agency Common Drug Review (CDR) process, Canada	Mr. Brendan McIntosh Lead, CDR Recommendations and Procedures, Canada's Drug Agency
Canada's Drug Agency pan-Canadian Oncology Drug Review (pCODR) process, Canada	Ms. Alexandra Chambers Director, pCODR Ms. Helen Mai Policy and Strategy Analyst, pCODR
Institut national d'excellence en santé et en services sociaux (INESSS), Canada	Ms. Sylvie Bouchard Director of Medication, INESSS
National Institute for Health and Care Excellence (NICE), United Kingdom	Ms. Zoe Garrett Technical Adviser — Centre for Health Technology Evaluation, NICE
Scottish Medicines Consortium (SMC), United Kingdom	Dr. Jan Jones Principal Pharmacist, SMC
Pharmaceutical Benefits Advisory Committee (PBAC), Australia	A/Prof. Tracy Merlin Managing Director, Adelaide Health Technology Assessment, The University of Adelaide Dr. Shuhong Wang Team Leader of Pharmaceutical Evaluation, Adelaide Health Technology Assessment, The University of Adelaide
Pharmaceutical Management Agency (PHARMAC), New Zealand	Ms. Sarah Fitt Director of Operations, PHARMAC Mr. James Harris Manager, Health Economics, PHARMAC
Gemeinsamer Bundesausschuss (G-BA) or the Federal Joint Committee, Germany	Dr. Matthias Perleth Department Head, Methodological Advice G-BA
Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) or Institute for Quality and Efficiency in Health Care, Germany	Dr. Ruth Schwarzer International Affairs, IQWiG Dr. Thomas Kaiser Head, Drug Assessment, IQWiG

Appendix 4: Findings

Table 1: Submission Process: Eligible Applicants and Program Funding

	Organization Eligible to Submit or Request for a Single Drug Technology Assessment			Type of Program Funding (Applicants Pay a Fee, Publicly Funded, or Both)
HTA Organization	Manufacturer	Public Drug Plan / Ministry of Health	Other Health Care Service Provider
CDR, Canada's Drug Agency⁴	Yes	Yes	No	Both^a
pCODR, Canada's Drug Agency^27b	Yes	Yes^b	Yes^b	Both^b
INESSS¹⁶	Yes	Yes	No	Solely publicly funded^c
NICE^3d	Yes^d	Yes^d	Yes^d	Solely publicly funded
SMC²⁸	Yes	No	No	Solely publicly funded^c
PBAC^19,29	Yes	No	No^c	Applicant pays a fee^c,e
PHARMAC⁹	Yes	Yes^c	Yes	Solely publicly funded^c
G-BA¹²	Yes	No	No	Solely publicly funded^c,f
HAS³⁰	Yes	No	No	Information not available

Canada's Drug Agency = Canadian Agency for Drugs and Technologies in Health; CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium.
^aProgram fees would not apply to assessment requests from the public drug plans.
^bProvincial Advisory Group (comprised of ministries of health and cancer agencies) and provincial tumour groups may make a submission to pCODR. Additionally, submissions from provincial tumour groups are exempt from fees.
^cBased on survey results.
^dIn the case of NICE, a topic can be suggested by a range of sources, but in order for an appraisal to begin there must be a referral from the Department of Health.³
^eAfter 2010, PBAC also requires applicants to pay a fee, although some drugs may be eligible for fee exemptions (e.g., orphan drugs or temporary supplies) or a fee waiver if the submission involves a component of public interest (such as palliative care, pediatric care, or care for aboriginal peoples) or where payment of the fee would make proceeding with the application financially unviable (for example, if the patient population is not large enough).⁷
^fNote for IQWiG: IQWiG is funded by levies for in-patient and outpatient health care services; i.e., contributions from persons insured in statutory health insurance funds (applies to nearly 90% of the population).
Note: This table was informed by survey questions 1 and 2 in Appendix 2 (Survey Questions: 1. Who is eligible to submit for an assessment of a drug? [Please provide a list; e.g., manufacturer, public drug plan, other health care service provider such as hospital associations.] 2. Do applicants pay a fee for the drug assessment conducted by the HTA organization or is the program [within the organization] solely publicly funded? [Applicants pay the fee/ Solely publicly funded/ Both]).

Table 2: Submission Process: Drug Categories Eligible for an Assessment — Overview

HTA Organization	New Drug (Non-Oncology)	New Indication (Non-Oncology)	New Combination (Non-Oncology)	New Oncology Drug (Including Indication with Combination)	Orphan Drug	Vaccine
CDR, Canada's Drug Agency⁴	Yes	Yes	Yes	No	Yes	Yes^a,b
pCODR, Canada's Drug Agency²⁷	No	No	No	Yes	Yes^c	Yes^b,c
INESSS¹⁶	Yes^a	Yes^a	Yes^a	Yes^a	Yes^a	No^a,d
NICE³	Yes	Yes	Yes	Yes	Yes	No
SMC²⁸	Yes	Yes	Yes	Yes	Yes	No
PBAC¹⁹	Yes	Yes	Yes	Yes	Yes	Yes
PHARMAC⁹	Yes	Yes	Yes	Yes	Yes	Yes^a
IQWiG²⁰	Yes	Yes^a	Yes^a	Yes^a	Yes^a	No^a

Table 3: Submission Process: Categories of Drugs Eligible for Single Drug Technology Assessment by the HTA Organization

HTA Organization	Eligible Categories	Categories Explicitly Excluded
CDR, Canada's Drug Agency⁴	“New drug: A new active substance that has not been previously marketed in Canada.”⁴ “Drug with a new indication: A drug previously reviewed by CDR that has received an NOC or NOC/c for a new indication; or A drug marketed before the establishment of CDR that has received an NOC or NOC/c for a new indication.”⁴ “New combination product: Two or more drugs that have not been previously marketed in Canada in that combination. It may consist of: two or more new drugs two or more previously marketed drugs a combination of new drug(s) and previously marketed drug(s).”⁴ “Subsequent entry biologic: Biologic drug demonstrating a high degree of similarity to an already authorized biologic drug (i.e., a reference product).”⁴	Oncology drugs Submissions should be made directly to public drug plans for the following items: line extensions of marketed products, including new dosage forms with the same route of administration and new strengths of the same dosage form generic products⁴
pCODR, Canada's Drug Agency⁸	New oncology drugs and oncology drugs with new indications that have not received a NOC or NOC/c from Health Canada (i.e., pre-NOC or pre-NOC/c submissions). New oncology drugs and oncology drugs with new indications that have received a NOC or NOC/c (i.e., post-NOC or post-NOC/c submissions). Resubmission for new oncology drugs or for an oncology drug with a new indication(s). Note: “if a Submission or Resubmission is made for an Oncology Drug with a New Indication, for a drug that already has received market authorization in Canada, and sufficient clinical and economic evidence exists to make a Submission, it is not required that the indication currently be under review by Health Canada.”⁸	“Non-oncology drugs, including supportive treatments that may be used in the care of patients with cancer. A new indication for which an NOC or NOC/c has not been received and that has been reviewed and rejected by Health Canada.”⁸
INESSS¹⁶	Drugs used in hospitals and in the public program are reviewed by INESSS. This includes oncology drugs, new drugs, drugs with a new indication, new combination products, subsequent entry biologic products, line extension of marketed products, generic drugs, blood sugar test strips, some vitamins, wound dressings, etc.^a	“Any indication for using a drug must target a health condition, i.e., prevent, relieve, heal or postpone a difficult health condition. The exclusion of indications for the treatment of alopecia and baldness as well as indications for use for aesthetic or cosmetic purposes is in accordance with this premise.”¹⁶ Drugs to treat obesity, to stimulate appetite, or to treat cachexy, and oxygen are excluded. Natural health products (of which the generic name is not already listed) are also excluded.^a
NICE	“Health technologies referred to the NICE technology appraisals programme include: medicinal products, medical devices, diagnostic techniques, surgical procedures or other therapeutic techniques, therapeutic technologies other than medicinal products, systems of care, screening tools.”³	“Population screening — falls under the remit of the UK National Screening Committee.”³ “Vaccination — generally falls under the remit of the Joint Committee on Vaccination and Immunisation. However, NICE does consider therapeutic vaccines.”³ “HIV technology or therapy — falls under the remit of the British HIV Association. However, there may be situations when the Department of Health considers that a NICE appraisal of an HIV technology or therapy would be helpful to the NHS and these will be dealt with on a case-by-case basis.”³
SMC	SMC provides advice on “newly licensed medicines, all new formulations of existing medicines and all new indications for established medicines…The SMC remit is confined to prescription-only medicines (PoMs).”¹⁷ “For the assessment of ultra-orphan medicines, a different decision-making framework will be used, similar to that used by the National Institute of Health and care excellence (NICE) for highly specialised technologies (HST).”²⁸	Exclusion criteria apply for medicine that is: “…not a Prescription Only Medicine (PoM). …used in immunisation and guidance on its use is issued by the Joint Committee on Vaccination and Immunisation. …used in diagnosis not treatment. …classified as a blood product …a parenteral preparation for fluid and electrolyte imbalance or parenteral nutrition. …used as an intervention in surgical procedures/wound management…[or] for the acute treatment of poisoning. …a medicine used in tropical diseases. …a branded generic product and costs the same or less than the originator product which has previously been accepted by SMC for the same indication. …a biosimilar medicine and the reference product has been accepted by SMC/HIS for the same indication(s) and in the same population or was initially licensed and available prior to 31 January 2002.”^28,31
PBAC¹⁹	“Submissions…list a new medicine (including a new fixed combination product, a new nutritional product, a new vaccine or a new orphan medicine)”¹⁹ “Minor submissions to list new forms of previously listed products or changes to the conditions of use: do not require an economic evaluation are not evaluated by the Pharmaceutical Evaluation Section or presented to the ESC before consideration by PBAC. Major submissions to new listings, included orphan medicines and significant changes to existing listings: require an economic evaluation are evaluated by the Pharmaceutical Evaluation Section and presented to the ESC before consideration by PBAC.”³²	None^a
PHARMAC⁹	“ funding of new pharmaceuticals for use in the community; funding of new hospital pharmaceuticals; changing access to an already listed pharmaceutical (e.g., for new uses or patient groups); funding of generic or biosimilar pharmaceuticals where an application to fund the pharmaceutical has not previously been considered by PHARMAC; funding new formulations or strengths of already funded pharmaceuticals; and funding combination products (products that consist of two or more pharmaceuticals).”⁹	None^a
IQWiG	“Within the framework of the Act on the Reform of the Market for Medicinal Products (AMNOG), at the beginning of 2011, the Institute’s responsibilities were extended to the assessment of the benefit of drugs with new active ingredients shortly after market entry.”²⁰ The Institute’s responsibilities include the assessment of the added benefit of drugs with new active ingredients /new combinations of active ingredients, or with a new indication at market entry, as well as orphan drugs. Orphan drugs have a special status, since their added benefit is already proven through market authorization and thus IQWiG only assesses information provided by the manufacturers on the number of patients affected by the rare disease and the cost of treatment. However, if the orphan drug’s revenue exceeded €50 million over the past 12 months, the drug also undergoes an early benefit assessment using the same method as for other drugs. The assessment is conducted by IQWiG after commissioning by the GB-A.^a	Drugs on the market before January 1, 2011^a Drugs exempted from reimbursement (e.g., lifestyle drugs)^a Drugs with a reimbursement volume by the SHI of less than €1 million per year^a

CDR = Canada's Drug Agency Common Drug Review; ESC = Economic Sub-Committee; G-BA = Gemeinsamer Bundesausschuss; HIS = Healthcare Improvement Scotland; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; NHS = National Health Services; NICE = National Institute for Health and Care Excellence; NOC = Notice of Compliance; NOC/c = Notice of Compliance with Conditions; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SHI = statutory health insurance; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey question 3 in Appendix 2 [Survey question 3: What categories of drugs are eligible for drug assessment by the HTA organization? (Please provide a list; e.g., new drugs, new combination products, new indication, oncology products, biosimilars, and therapeutic vaccines). What categories of drugs are explicitly excluded? (Please provide a list)]. This question was not applicable to Gemeinsamer Bundesausschuss (G-BA). No information was available for the Haute Autorité de santé (HAS).

Table 4: Submission Process: Appraisal Status According to Market/Regulatory Approval

HTA Organization	HTA organization conduct assessments for drugs/ technologies that have not yet received market authorization or regulatory approval in the country/region (Yes, with details, or No )
CDR, Canada's Drug Agency	“When Health Canada is highly likely to issue an NOC or NOC/c for the indications to be reviewed by CDR within 90 calendar days, a submission may be filed on a pre-NOC basis for a new drug, drug with a new indication, new combination product, new combination product (funded components), or an SEB.”⁴
pCODR, Canada's Drug Agency	Pre-NOC or Pre-NOC-c submissions are accepted under the pCODR program.⁸ The pCODR program may consider reviewing submissions that are made for a cancer drug with an “off-label” indication where sufficient clinical and economic evidence exists for a drug that already has received market authorization in Canada. In these cases, the pCODR program consults with the Provincial Advisory Group to work though specific considerations to determine appropriateness for national review. These considerations may include, but are not limited to, unmet need/therapeutic gap, burden of illness, quality of care, among others. This process was put in place in order to ensure that there are no gaps in therapy for certain population groups diagnosed with cancer and to promote greater evidence-based informed care.^a The pCODR Procedures state: “…if a Submission or Resubmission is made for an Oncology Drug with a New Indication, for a drug that already has received market authorization in Canada, and sufficient clinical and economic evidence exists to make a Submission, it is not required that the indication currently be under review by Health Canada.”⁸
INESSS	No^a
NICE	“Unless the Department of Health specifically indicates otherwise, NICE will not publish guidance on the use of a technology for indications that have not been given regulatory approval in the UK (that is, for unlicensed or 'off-label' use outside the terms of the technology's marketing authorisation).”³ Off-label indications may be considered through a separate programme and evidence summaries may be provided.¹⁰
SMC	SMC asks that submitters provide “details of the licence status of the product for the indication(s) detailed in the submission, including dates of granted or expected marketing approval.”¹⁷ SMC also asks the company to submit as soon as possible after an EMA positive opinion is received.^a
PBAC	“Although marketing approval and registration on the Australian Register of Therapeutic Goods (ARTG) are prerequisites for PBS listing, PBAC accepts submissions before finalisation of marketing approval, provided that a corresponding application has been lodged with the Therapeutic Goods Administration (TGA).”¹⁹
PHARMAC	Yes,⁹ applications can be made independently of current registration status.^a
G-BA	No.^a Note: In the case of Germany, the early benefit assessment is not a decision on reimbursement. AMNOG assessments are therefore not initiated in advance of regulatory approval but only at the time of market access.
HAS	Drugs can be reviewed before market authorization is granted if a favourable opinion from the Committee Medicaments Human Use (CHMP) is given.³³

AMNOG = Act on the Reform of the Market for Medicinal Products (Arzneimittelneuordnungsgesetz); CDR = Common Drug Review; EMA = European Medicines Agency; G-BA = Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; NICE = National Institute for Health and Care Excellence; NOC = Notice of Compliance; NOC/c = Notice of Compliance with Conditions; PBAC = Pharmaceutical Benefits Advisory Committee; PBS = Pharmaceutical Benefits Scheme; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SEB = subsequent entry biologic; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey question 4 in Appendix 2 (Survey question 4: Does the HTA organization conduct assessment of drugs that have not yet received market authorization or regulatory approval in the country/region?). This question was not applicable to Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG).

Table 5: Submission Process: Overview of Prioritization and Intake Process

HTA Organization	Prioritization Process (for review) in Place When an Eligible Drug is Submitted for a Single Drug Technology Assessment		Intake Process (fixed schedule or on an ongoing basis)
HTA Organization	Yes/No	If yes, details of the criteria for prioritization
CDR, Canada's Drug Agency⁴	Yes (but process currently on hold)	“The process used by Canada's Drug Agency in the assessment of a CDR priority review request depends on whether the applicant is requesting priority review based on clinical criteria or based on economic criteria…Priority review status based on clinical criteria will be determined by Canada's Drug Agency in consultation with the Canadian Drug Expert Committee chair, one of the two Canadian Drug Expert Committee public members, external clinical expert(s) (as required), as well as representatives from the drug plans; priority review status based on economic criteria will be determined by Canada's Drug Agency in consultation with representatives from the drug plans.”⁴ “A submission or resubmission may be may be granted priority review status based on clinical criteria if all of the following criteria are demonstrated: The drug is indicated or anticipated to be indicated for an immediately life-threatening or other serious disease The drug addresses an unmet medical need The drug offers substantial improvement in clinically important outcome measures of efficacy and effectiveness, when compared with other appropriate comparators. A submission or resubmission may be granted priority review status based on economic criteria if the following criterion is demonstrated: For the drug under review, the projected combined cost savings for the drug plans is an average of at least $7.5 million per year for the first three years the product is marketed in Canada, when compared with appropriate comparators.”⁴ “The final decision regarding priority review status will be determined by Canada's Drug Agency. There is no provision for requesting reconsideration of the decision.”⁴ It should be noted that the above-specified process is currently on hold.¹¹	Ongoing
pCODR, Canada's Drug Agency⁸	Yes	“The pCODR process screens Submissions in the order they are received, that is, on a “first-come, first-served” basis, and reviews Submissions based on the order in which they are deemed complete. Exceptions to this order of review are as follows: a) If, as a result of a procedural review (see the pCODR Procedural Review Guidelines), it is determined that additional work on a Submission is required, work on the Submission will be given priority within pCODR and the Submission will be given priority placement on the pERC meeting agenda at which it will be re-deliberated”⁸…a) At the time a Submission is made, Submitters may request that a Submission be assessed to determine whether or not it meets priority review criteria. This request will be considered by a three-person panel consisting of the pERC Chair, the pERC Vice Chair and one additional pERC Member, according to the following priority review criteria: a New Oncology Drug, or Oncology Drug with a New Indication and employed for the active treatment of cancer, where use offers potential substantial improvements on significant outcomes, such as (but not limited to): improved overall survival in the adjuvant setting; or elimination or substantial reduction of treatment side effects associated with standard of care; or measurable and substantial improvements in quality of life over other available therapies in Canada OR a New Oncology Drug, or Oncology Drug with a New Indication and employed for the active treatment of cancer, where no other comparable drug/treatment is currently marketed in Canada”⁸	Ongoing
INESSS³⁴	Yes	Some registration applications under the List of Medications covered by the basic prescription drug insurance plan or the List of Medications may be considered for priority evaluation, which means the product can be registered prior to a scheduled update. There are two motives for priority evaluation: the therapeutic and the economic motive. The therapeutic motive is invoked if “delaying the evaluation may lead patients who need the drug to suffer an irreversible and rapid progression of their illness that could cause great harm,”³⁴ and no other therapeutic option in the form of a drug is available on the List of Medications covered by the basic prescription drug insurance plan or on the List of Medications – Institutions. In regard to the economic motive, in accordance with the Politique du médicament (policy on medication), a drug product may also be considered for priority evaluation if registering it could result in sizable cost savings for the public plan. In order for a drug product to be considered for priority registration under an economic motive, it must meet the yearly pre-set minimum for potential savings of $200,000 for each month of early registration.^a	Fixed^a
NICE³	Yes	First, there are criteria that help the Department of Health decide if the topic should be referred (after NICE has completed the scoping process but before the appraisal can start):^a “The prioritisation criteria are: Is the technology likely to result in a significant health benefit, taken across the NHS as a whole, if given to all patients for whom it is indicated?; Is the technology likely to result in a significant impact on other health-related Government policies?; Is the technology likely to have a significant impact on NHS resources if given to all patients for whom it is indicated?; Is there significant inappropriate variation in the use of the technology across the country?; Is NICE likely to be able to add value by issuing national guidance? For example, without such guidance is there likely to be significant controversy over the interpretation or significance of the available evidence on clinical and cost effectiveness?”³ Second, there is the prioritization into the work program following referral. This is done based on NICE’s ability to be timely with the guidance; i.e., guidance is published as soon after market authorization as possible for the maximum number of products.^a	Ongoing
SMC¹⁷	Yes	SMC takes into account various factors in their prioritization process, including the date the product is due to become available in the UK, service need, patient need, and whether or not the submission is for a first-in-class medicine.^a	Ongoing
PBAC¹⁹	No	N/A	Ongoing
PHARMAC⁹	Yes	In consideration of PHARMAC’s Decision Criteria (or from the July 2016 Factors for Consideration), as well as the quality and completeness of the application, proposals may be assessed in any order.^a After assessment, all “Applications are prioritised against other funding options (either listing of new pharmaceuticals or widening access to pharmaceuticals that are already listed), whether received via Application or via PHARMAC initiated proposals. The overall aim is to identify potential amendments to the Pharmaceutical Schedule that would provide the greatest possible health benefits and help us to meet our Statutory Objective. PHARMAC conducts regular prioritisation reviews of all outstanding Applications.”⁹	Ongoing
IQWiG	No^a	Submission and assessment deadlines are specified by the law.^a	Fixed ^{a, b}
HAS	XX	XX	Ongoing

Table 6: Drug Review Process: Assessment Methods by Drug Category and Approximate Timelines for Appraisal

HTA Organization	HTA Organization has Different Assessment Methods for Following Categories of Drugs (Yes/No)						Approximate Timelines for Conducting Single Drug Technology Assessment at the Organization (i.e., Timeline Between the Initiation of Review and Issuance of the Final Recommendation/Guidance)
HTA Organization	Combination Products		Biosimilars		Drugs for Rare Diseases	Oncology	Timelines	Detail
CDR, Canada's Drug Agency⁴	Yes		Yes		No	N/A	25 weeks	Performance target maximum of 180 calendar days; i.e., from date of acceptance for review to date of issuance of embargoed CDEC recommendation
pCODR, Canada's Drug Agency⁸	N/A					N/A^b	25 weeks	Performance target maximum of 180 calendar days; i.e., from date of submissions deemed complete to date of issuance of initial pERC recommendation
INESSS	No^a	No^a		No^a		No^a	20 to 23 weeks^a	The period between submission date and recommendation to the health minister is between 20 and 23 weeks^a
NICE^3,13	No	No		Yes		No	35 weeks	The appraisal process takes approximately 35 weeks
SMC¹⁷	Yes^a	No^c		Yes^a		Yes^a	18 weeks	8 weeks for assessment and provisional advice by the New Drugs Committee, 6 weeks for advice by the SMC, 4 weeks for the advice to be made public; the timeline is extended by 1 to 3 months for end-of-life and orphan medicines^a
PBAC¹⁹	No	No		No		No	35 weeks	Submission to be made 17 weeks before the PBAC meeting, publication of public summary document to take place 18 weeks after the PBAC meeting
PHARMAC⁹	No	No		No		No	12 weeks^a	The minimum time from receipt of a proposal to prioritization and a response to the applicant is 12 weeks^a
IQWiG^12,20	No^a	No^a		No^a,d		No^a	12 weeks	The assessment must be completed no later than 3 months after the relevant date for submission
HAS³⁵	XX	XX		XX		XX	12 weeks	Duration of single technology appraisal is 90 days

CDEC = Canadian Drug Expert Committee; CDR = Common Drug Review; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; N/A = not applicable; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; pERC= pCODR Expert Review Committee; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium; XX = information not available.
^aBased on survey results.
^bIt should be noted that pCODR assesses only oncology drugs.
^cMost biosimilars are beyond the SMC remit. See Table 3.
^dIn Germany, by law, the added benefit of orphan drugs is already proven through market authorization. IQWiG’s responsibility is usually limited to an assessment of information provided by the manufacturers on the number of patients affected by the rare disease and the cost of treatment. If an orphan drug’s revenue exceeded €50 million over the past 12 months, the orphan drug also undergoes an early benefit assessment.
^ePHARMAC assesses drugs for rare diseases using its “standard methods" but in 2015 invited competitive proposals through a separate process in order to improve access to these medicines.¹⁴
Note: This table was informed by survey questions 8 and 9 in Appendix 2 (Survey question 8: Does your organization have different assessment methods for following categories of drugs? Combination products [Yes / No / Not Applicable]; Biosimilars [Yes / No / Not Applicable]; Drugs for rare diseases [Yes / No / Not Applicable]; Oncology [Yes / No / Not Applicable]. Question 9: What are the approximate timelines for conducting drug assessment at your organization; i.e., timeline between the initiation of review and issuance of the final recommendation/guidance? [Please provide details.])) These questions were not applicable to Gemeinsamer Bundesausschuss (G-BA).

Table 7: Drug Review Process: HTA Organization’s Process for Clinical Review

HTA Organization	Process Used by the HTA Organization for the Evaluation of a New Drug			If Systematic Review Submitted by the Applicant (See Section 2.1.1 for HTA organization’s process to appraise the quality of the systematic review conducted by the applicant.)		HTA Organization Conducts Additional In-House Research (i.e., In Addition to the SRs Submitted by the Applicant or When an SR is Not Required)	Type of Additional Research Conducted by the HTA Organization
HTA Organization	SR Conducted by the HTA Organization in-House	SR Submitted by Applicant	SR is Not Required	HTA Organization Has a Guideline (Methods) for Such Appraisals	Guideline is Publicly Available		Literature Search	Meta-Analysis	Sensitivity Analysis
CDR, Canada's Drug Agency⁴	☑			N/A
pCODR, Canada's Drug Agency⁸	☑			N/A
INESSS¹⁶			☑^a	N/A		Yes^b	Yes^b	No^b	Yes^b
NICE³		☑^c		Yes	Yes	No	N/A
SMC^17,28		☑		Yes	Yes	No	N/A
PBAC¹⁹		☑		Yes	Yes	Yes^b	Yes^b	Yes^b	Yes^b
PHARMAC^a	☑	☑		No^b	N/A^b	Yes^b	Yes^b	Yes^b	Yes^b
IQWiG²⁰		☑		Yes	Yes	Yes	Yes	Yes^b	Yes^b
HAS³³		☑		XX	XX	Yes	Yes	No	No

CDR = Common Drug Review; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; N/A = not applicable; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium; SR = systematic review; XX = information not available.
^aINESSS considers all of the clinical evidence submitted by the applicant, but it does not require a systematic review to be provided by the applicant and it does not conduct a systematic review in-house, either.
^bBased on survey results.
^cIn exceptional circumstances, such as when all published or unpublished clinical data are within the company’s control or possession, a systematic literature review may not be required. If a systematic literature search is not included in the submission, the company must confirm that no other additional relevant studies have been done outside its organization. NICE requires the medical director of the company to sign a statement confirming that all clinical trial data necessary to address the remit and scope of the technology appraisal as issued by the Department of Health and NICE, within the company's or any of its associated companies' possession, custody, or control in the UK or elsewhere in the world, have been disclosed to NICE.¹⁵
Note: This table was informed by survey question 10 in Appendix 2 (Survey question: Please select which of the following best describes the process used by the HTA organization for the evaluation of a new drug. [Option A: Systematic review is not a requirement; Option B: Systematic review conducted by the HTA organization; Option C: Systematic review submitted by the applicant or Option D: a combination of Option B and C] Please answer the following question ONLY if you selected Option C or D: What is the organization’s process to appraise the quality of the systematic review conducted by the applicant such as process to appraise the methodological quality of the SR? [Please provide details.] Does your organization have a guideline [methods] for such appraisals [of the systematic review conducted by the applicant]? [Yes/No] If yes, is the guideline publicly available? [Yes/No] If yes, please provide hyperlink and summarize. Does your organization conduct additional in-house research [i.e., in addition to the SRs submitted by the applicant]? [Yes/No] If yes, does it typically include the one or more of the following [please select all that apply]? [Literature search / Meta-analysis / Sensitivity analysis]) This question was not applicable to Gemeinsamer Bundesausschuss (G-BA).

Table 8: Drug Review Process: Clinical Experts Involved in the Single Drug Technology Assessment (Clinical Review)

HTA Organization	Stages of the Review in Which Clinical Experts Are Involved, and Their Level of Involvement
CDR, Canada's Drug Agency	CDR clinical review team invites external clinical experts.⁴ Canada's Drug Agency establishes “a review team, based on the nature of the submission or resubmission, and in consideration of the proposed team members’ qualifications, expertise, and compliance with the Canada's Drug Agency Common Drug Review Conflict of Interest Guidelines.”⁴
pCODR, Canada's Drug Agency	“The Methods Team and the Clinical Guidance Panel develop a review plan, also known as the protocol, for the review of the Submission.”⁸ For each review, the Clinical Guidance Panel is comprised of three (3) oncologist members who are experts in their field. Input on the protocol may be provided by PAG, pERC members, the Economic Guidance Panel, and the pCODR secretariat. The review team considers the patient-important outcomes and issues identified through patient advocacy group input when developing the protocol.^a The Clinical Guidance Panel (CGP) reviews the systematic review conducted by the Methods Team. They are responsible for writing the discussion and conclusion section in the clinical report reflecting on the systematic review and its context within Canada. In addition, they also address any issues raised by the PAG on the implementation feasibility of the drug under review. The CGP also provides clinical context for the economic report, ensuring that assumptions used in the economic model are relevant in a Canadian setting.⁸
INESSS	First stage of review: scientific staff of INESSS, outside experts (composed of scientists, clinicians, pharmacists, managers, the public, methodologists, ethicists; the number is linked to the complexity of the submission), and one or two representatives from the Comité scientifique d'évaluation des médicaments aux fins d'inscription (CSEMI) of INESSS.^a Second stage of review: the results of the first evaluation are presented to 17 members of the permanent committee to be discussed and deliberated and to formulate the recommendations.^a In oncology, an additional stage is taken between stages 1 and 2, and involve an additional 15 members of CEPO implicated in the deliberation of therapeutic value (first criterion to be evaluated).^a
NICE	“The Chair of the Appraisal Committee and the NICE project team select clinical experts and patient experts from nominations by non-company consultees and commentators. They attend the Appraisal Committee meeting as individuals to answer questions to help clarify issues about the submitted evidence and to provide their views and experiences of the technology and/or condition. Before they attend the meeting, all experts must either submit a written statement (using a template) or indicate they agree with the submission made by their nominating organization.”³
SMC	The SMC maintains a multidisciplinary network of clinical experts. After a submission is received, the SMC secretariat request responses to generic questions from 5 clinical experts. Further follow-up questions are provided no later than 21 days before NDC.³⁶ Clinical experts are also involved in the assessment of end-of-life and orphan medicines in the Patient & Clinician Engagement (PACE) step. This involves a roundtable discussion between clinicians and patient groups on the added value of a medicine beyond the existing clinical and economic analysis.^a
PBAC	“In formulating its conclusions, PBAC may seek expert opinion from relevant professional bodies and/or appropriate specialists, and may meet with representatives of relevant medical professional organizations and colleges. PBAC may also seek input from appropriate consumer bodies. As a routine, PBAC seeks advice from the Australian Technical Advisory Group on Immunisation in relation to vaccines, the Nutritional Products Working Party in relation to nutritional products and the Antimicrobial Resistance Subcommittee in relation to the development of resistance to new antimicrobial agents.”¹⁹
PHARMAC	The PTAC and PTAC subcommittees are the primary clinical advisory committees⁹ These committees, after a detailed review of the application and the quality of the evidence provided, will make a recommendation to PHARMAC (taking into consideration the Decision Criteria [or Factors for Consideration from July 2016])^a
IQWiG	“Medical expertise is primarily involved on the basis of a questionnaire sent to external experts at the beginning of the assessment. In its assessment the Institute considers the external experts’ feedback... External experts may if necessary be drawn upon to clarify specific questions arising during the course of the assessment.”²⁰
HAS	The Chairman of the Board may call one or more external experts, called rapporteurs, who are competent in the area of assessment in question, and they inform the Commission. Rapporteurs shall prepare a written report, which is sent to every member of the Commission. The rapporteurs are invited to present their report to the Commission and answer questions from its members. They assist in the deliberations or voting of the Commission.³³

CDR = Common Drug Review; CEPO = Comité de l’évolution des pratiques en oncologie; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; NDC = New Drugs Committee; NICE = National Institute for Health and Care Excellence; PAG = Provincial Advisory Group; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; pERC = pCODR Expert Review Committee; PHARMAC = Pharmaceutical Management Agency; PTAC = Pharmacology and Therapeutics Advisory Committee; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey question 11 in Appendix 2 (Survey question: Are clinical experts involved in the drug assessment [clinical review]? (Yes/ No) If yes, what stages of the review are they involved in and what is the level of their involvement? [Please provide details.]) This question was not applicable to Gemeinsamer Bundesausschuss (G-BA).

Table 9: Drug Review Process: Encourage or Require Submissions of Evidence From Stakeholders

HTA Organization	Encourage or Require Submissions of Evidence From Stakeholders (Other Than the Applicant)	Stakeholders Allowed to Provide Submissions of Evidence?
HTA Organization		Patient Advocacy Groups	Individual Patients	Clinicians or Professional Societies (of Health Care Professionals and/or Regulatory Bodies)	Manufacturers^a	Public Payer
CDR, Canada's Drug Agency⁴	Yes	Yes	Yes^b,c	No	Yes	No
pCODR, Canada's Drug Agency⁸	Yes	Yes	Yes^b,c	Yes^d	Yes	Yes
INESSS¹⁶	Yes	Yes	Yes^b	Yes	Yes^b	No
NICE³	Yes	Yes	Yes^e	Yes	Yes	Yes^b,f
SMC²⁸	Yes	Yes	No	Yes^b,g	N/A	No
PBAC¹⁹	No	N/A	N/A	N/A	N/A	N/A
PHARMAC⁹	No^b,h	N/A	N/A	N/A	N/A	N/A
IQWiG²⁰	Yes^b,i	Yes^b	Yes^b	Yes^b	No	No

CDR = Common Drug Review; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; N/A = not applicable; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium.
^aApplicable only to HTA organizations where manufacturers are not eligible to make submissions for an assessment (e.g., NICE), or where entities other than a manufacturer can also apply for a single technology assessment (e.g., CDR and pCODR programs where public drug plan and health care service providers are also eligible to request an assessment, respectively; and INESSS where the Ministry of Health is also eligible to submit for an assessment, in addition to the manufacturer).
^bBased on survey results.
^cCDR and pCODR will accept submissions from individual patients only in the absence of patient advocacy groups.²¹
^dAs of February 1, 2016, pCODR launched a pilot initiative that would accept input from registered clinicians.²²
^eThose who attend committee meetings.
^fTwo clinical commissioning groups are included in the matrix of stakeholders and can submit evidence. In addition, the commissioning representatives who attend the Committee meeting submit a statement of evidence.
^gRefers to the Patient & Clinician Engagement statement for end-of-life and orphan medicines.
^hFollowing the initial appraisal, PHARMAC invites detailed comment on draft recommendations from all stakeholders. Also, any clinician or patient may initiate a formal application.
ⁱAt IQWiG, clinical experts interested in being involved in assessments have the opportunity to register in a database. If clinical experts are selected for an assessment, their input is obtained by means of a specific questionnaire. Patient organizations or individual patients also have the opportunity to describe the patients' perspective. The spokesperson of the Gemeinsamer Bundesausschuss (G-BA) Coordination Committee of the Patient Representatives passes on the IQWiG’s questionnaire. It contains questions regarding patients’ perspectives on the general description of the disease, currently available treatment options, the therapeutic need, relevant end points, and relevant subgroups.
Note: This table was informed by survey question 12 in Appendix 2 (Survey question: Does the organization’s process encourage or require submissions of evidence from stakeholders [other than the applicant]? [Yes/ No] If yes, which stakeholders are allowed to provide such submissions? [Please select all that apply.] [Patient advocacy groups / Individual patients / Clinicians or professional societies (of health care professionals and/or regulatory bodies) / Manufacturers / Drug reimbursement bodies (i.e., public drug plans)]) This question was not applicable to Gemeinsamer Bundesausschuss (G-BA.) No information was available for HAS.

Table 10: Drug Review Process: Allow Stakeholders to Comment on the Draft or Final Report

(See Table 16 for process to allow stakeholders to comment on Draft Recommendations/Guidance.)

HTA Organization	Allow Stakeholders to Comment on the Draft/Final Report (Yes/No)	If Yes, Stakeholders Allowed to Comment
HTA Organization		Manufacturer	Public Payer	Patient Groups	General Public
CDR, Canada's Drug Agency⁴	Yes	Yes	No	No	No
pCODR, Canada's Drug Agency⁸	Yes^a	Yes	Yes	Yes	No
INESSS	No^b	N/A
NICE³	Yes	Yes	Yes	Yes	Yes
SMC²⁸	Yes	Yes	No	No	No
PBAC	Yes^b	Yes^b	No^b	No^b	No^b
PHARMAC	Yes^b	Yes^b	Yes^b	Yes^b	Yes^b
IQWiG²⁰	No^c	N/A
G-BA	Yes^b	Yes^b	No^b	No^b	No^b
HAS³³	No	N/A

CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; N/A = not applicable; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium.
^aIncluding tumour groups and any registered clinicians who provided input at the start of the submission.²²
^bBased on survey results.
^cThe final report is open for comments at G-BA.
Note: This table was informed by survey question 13 in Appendix 2 (Survey question: Does the organization’s process allow stakeholders to comment on the draft report? [Yes/ No] If yes, which of the following stakeholders are allowed to comment? [Select all that apply.] Manufacturer / Payer / Patient groups / General public)

Table 11: Deliberative Process: Details of the Committee

HTA Organization	Number of Members Comprising the Committee	Experts and/or Representatives Who Comprise the Committee
HTA Organization	Number of Members Comprising the Committee	Physicians	Pharmacists	Nurses	(Research) Methodological Experts	Patients	Public Representative	Ethicist
CDR, Canada's Drug Agency³⁷	14 members	Yes	Yes	No	Yes	No	Yes	No
pCODR, Canada's Drug Agency⁸	Up to 16 members	Yes	Yes	No	Yes	Yes^a,b	No	Yes
INESSS^16,34	15 to 20 members^a	Yes	Yes	No	Yes^a	No	Yes^a	Yes
NICE³⁸	24 members per committee^a	Yes	Yes	Yes	Yes	Yes	Yes	No
SMC²⁸	Approximately 30 members	Yes	Yes	Yes^a	No^a,c	No	Yes^a	No
PBAC¹⁹	Currently 17 members	Yes	Yes	No	No	No	No	No
PHARMAC³⁹	Currently 12 members^d	Yes	Yes^a	Yes^a	No^a	No^a	No^a	No^a
G-BA	13 members^a	See below ^e
HAS³³	20 full members, 6 alternatives, 8 advisory members	Yes³⁰	Yes	XX	Yes	XX	XX	XX

CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium; XX=information not available.
^aBased on survey results.
^bThe pCODR Expert Review Committee (pERC) includes two patient representatives and a patient representative alternate.
^cAn academic health economist sits on SMC.
^dThe core Pharmacology and Therapeutics Advisory Committee (PTAC) has 12 members, with more than 100 subcommittee members [ref Annual Review]. PTAC advises PHARMAC as part of the evidence review process. PHARMAC staff contribute to the deliberative process, which results in a formal recommendation to the PHARMAC Board, comprising Directors appointed by the Minister of Health.
^eThe G-BA plenum includes standing members from the four umbrella organizations included DKG (German Hospital federation), KBV (National Association of Statutory Health Insurance Physicians), KZBV (National Association of Statutory Health Insurance Dentists), and GKV-SV (Federal Association of Statutory Health Insurance Funds) (based on survey results).
Note: This table was informed by survey questions 14 and 15 (in part) in Appendix 2 (Survey question 14: How many members comprise the committee? Survey question 15: Which of the following experts and/or representatives comprise the committee? [Please select all that apply.] Physician / Pharmacist / Nurses/ Patients / Public representative / Ethicist / (Research) Methodological experts]) This question was not applicable to Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG).

Table 12: Deliberative Process: Additional Committee Members and Features of Recommendations

HTA Organization	Other Members Invited to Participate (e.g., Specialists With Expertise Specific to Disease/Therapeutic Area)	Same Committee Makes Recommendations for all Drug Types (or drug class) or There is a Separate Process/Programs for Different Drug Types (e.g., separate program for oncology drugs)
CDR, Canada's Drug Agency	“Non-member specialist expert…may be invited to participate in CDEC meetings to provide their expertise as required. Specialist experts are thought leaders and may be drawn from a variety of fields, such as the clinical setting, methodology, and health economics. These specialist experts will have experience relevant to the use of the particular drug or class of drugs or blood product or treatment of an identified condition.”³⁷	“The CDEC mandate is advisory in nature and is to provide recommendations or advice to Canada's Drug Agency to inform: • decisions regarding the funding of drugs (submitted through the Canada's Drug Agency Common Drug Review process) within the publicly funded health care system in Canada (excluding Quebec) • decisions and strategies regarding the optimal use of drugs (including therapeutic or class reviews) in Canada.”³⁷ There is a different process for oncology drugs.³⁷
pCODR, Canada's Drug Agency	“The pERC Chair may invite members of the Review Team, including Clinical Guidance Panel members or Economic Guidance Panel members and/or External Experts to provide input in person at a pERC meeting...”⁸	pERC provides “cancer drug funding recommendations, including conditions and/or criteria for coverage where appropriate, to the participating provincial/territorial Ministries of Health and provincial cancer agencies, based on Submissions or Resubmissions. Upon request, the pERC also provides advice to jurisdictions on cancer drug products, which may or may not result in a change to a previously issued pERC recommendation. A Request for Advice may be made by the Canada's Drug Agency pCODR Advisory Committee (PAC) or the Provincial Advisory Group (PAG).”⁴⁰
INESSS	At least one outside expert and two members of the scientific staff of INESSS from stage 1 of the review present the results of this review and may answer questions regarding the review. They then leave for the deliberation of the committee.^a	All recommendations are made by the Comité scientifique d'évaluation des médicaments aux fins d'inscription (CSEMI), but differences are observed with the review of oncology drugs. All oncology drugs are evaluated by the CSEMI and the Comité de l’évolution des pratiques en oncologie (CEPO).^a The therapeutic value assessment is performed by members of the CSEMI in collaboration with the CEPO, composed of hematologists, medical oncologists, radiation oncologists, surgeons, and specialist oncology pharmacists.^a With respect to other criteria in the law, members of the CEPO are consulted about clinical assumptions embedded in the pharmacoeconomic analysis and about the ethical and societal aspects.^a
NICE	“The Chair of the Appraisal Committee and the NICE project team select clinical experts and patient experts from nominations by non-company consultees and commentators. They attend the Appraisal Committee meeting as individuals to answer questions to help clarify issues about the submitted evidence and to provide their views and experiences of the technology and/or condition. Before they attend the meeting, all experts must either submit a written statement (using a template) or indicate they agree with the submission made by their nominating organisation.”³ “Two representatives from the company(ies) (normally 1 with health economics expertise and 1 with medical expertise) for the technology(ies) being appraised can attend part 1 of the Appraisal Committee meeting discussions.”³	“NICE allocates Committee members to 1 of 4 standing Appraisal Committees. Members will normally remain in the same Committee for the duration of their membership.”³
SMC	Other members are brought in through the Patient & Clinical Engagement (PACE) step. The PACE step involves a roundtable discussion between patient groups and clinicians to explore the added value of a medicine that is not fully captured in existing clinical and economic analysis.^a	“SMC is helped to reach decisions by its New Drugs Committee (NDC)… “with a purely technical remit to review the clinical and economic data. This committee makes the first assessment of each submission, which may include an e-mail exchange of questions and answers with the submitting company prior to the NDC meeting.”²⁸ The same committee makes the final recommendation for all types of submissions.^a
PBAC	Consumers and health economists also comprise the Committee.¹⁹	“The Pharmaceutical Benefits Advisory Committee (PBAC) is established under the National Health Act 1953 (the Act). Its primary role is to recommend to the Minister for Health which medicines and medicinal preparations should be subsidised by the Australian Government under the Pharmaceutical Benefits Scheme (PBS). In doing this, PBAC is required by the Act to consider both the effectiveness and cost of the proposed medicines and medicinal preparations.”¹⁹
PHARMAC	PTAC has many expert subcommittees in specialist areas.⁹	The PTAC and, depending on the case, PTAC subcommittees, make recommendations to the PHARMAC Board, who then make a decision.⁹ Each subcommittee is grouped by therapeutic area.^a
G-BA	Representatives from other organizations and federations are involved, as required, and experts are brought in as they are needed.^a	The final decision is always made by the plenum of the G-BA. Whereas the preparation process runs across different subcommittees, drugs are not divided by types and classes.^a

CDEC = Canadian Drug Expert Committee; CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; pERC = pCODR Expert Review Committee; PHARMAC = Pharmaceutical Management Agency; PTAC = Pharmacology and Therapeutics Advisory Committee; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey questions 15 (in part) and 16 in Appendix 2. [Survey question 15: Are other members brought in for the deliberation? (e.g., specialists with expertise specific to disease/therapeutic area)? Question 16: Does the same committee make recommendations for all drug types (or drug class) or are there separate process/programs for different drug types (e.g., separate program for oncology drugs)? (Please provide details)] These questions were not applicable to the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). No information was available for the Haute Autorité de santé (HAS).

Table 13: Deliberative Process: Deliberative Framework to Make Recommendations/Decisions

HTA Organization	Publication on Deliberative Framework to Make Recommendations/Decisions
CDR, Canada's Drug Agency	Procedure for the Canada's Drug Agency Common Drug Review. Ottawa: Canada's Drug Agency; 2014 Aug. /media/cdr/process/CDR_Procedure.pdf⁴
pCODR, Canada's Drug Agency	pCODR Expert Review Committee deliberative framework. Ottawa: Canada's Drug Agency; 2016 Mar. /sites/default/files/pcodr/The pCODR Expert Review Committee (pERC)/pcodr_perc_deliberative_frame.pdf²⁷
INESSS	Selecting medication for coverage in Quebec. A responsible, transparent process. Quebec (QC): Conseil du médicament; 2007 Feb. http://www.inesss.qc.ca/fileadmin/doc/INESSS/DocuAdmin/Selecting-medication-coverage.pdf¹⁶
NICE	National Institute for Health and Care Excellence. Guide to the methods of technology appraisal. London: The Institute; 2013 Apr 4. (Process and methods guides). https://www.nice.org.uk/process/pmg9/chapter/foreword ²⁴
SMC	Working with SMC - a guide for manufacturers. Glasgow: Scottish Medicines Consortium; 2015 Aug.https://www.scottishmedicines.org.uk/files/submissionprocess/Working_with_SMC_July_2014.pdf⁴¹
PBAC	Pharmaceutical Benefits Advisory Committee. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee [Internet]. Version 4.5. Canberra, Australia: Australian Government, Department of Health; 2015 Jul. http://www.pbac.pbs.gov.au/content/information/printable-files/pbacg-book.pdf¹⁹
PHARMAC	Decision criteria consultation. Wellington, New Zealand: PHARMAC; 2015 Mar 20.http://www.pharmac.health.nz/about/operating-policies-and-procedures/decision-criteria-consultation/⁴² Consumer Advisory Committee (CAC). Wellington, New Zealand: PHARMAC; 2015 Feb 23. https://www.pharmac.govt.nz/about/committees/³⁹
IQWiG	General methods [Internet]. Version 4.2. Cologne (DE): Institute for Quality and Efficiency in Health Care (IQWiG); 2015 Apr 22. https://www.iqwig.de/download/IQWiG_General_Methods_Version_%204-2.pdf²⁰
G-BA	des Gemeinsamen Bundesausschusses. Verfahrensordnung [Rules of procedure]. Berlin, Germany: Gemeinsamen Bundesausschusses; 2015 Apr 16. Report No.: 84a. https://www.g-ba.de/downloads/62-492-1002/VerfO_2014-12-18_iK-2015-04-16.pdfGerman^a43

CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey question 17 in Appendix 2. [Survey question: Does the committee have a well-documented deliberative framework to make recommendations/decisions? (Yes/No) If yes, is the information publicly available? (Yes/No) If yes, please provide a hyperlink to the document.] No information was available for the Haute Autorité de santé (HAS).

Table 14: Deliberative Process: Details Regarding Public Involvement and Availability of Information Relating to Decisions to the Public.

HTA Organization	Meetings of the Committee Held in Public	Supporting Information and the Basis of the Recommendations Made Publicly Available
CDR, Canada's Drug Agency⁴	No	Yes
pCODR, Canada's Drug Agency⁸	No	Yes
INESSS¹⁶	No	Yes
NICE^3,24a	Yes	Yes
SMC^28b	Yes^c	Yes
PBAC¹⁹	No	Yes
PHARMAC⁹	No	Yes
G-BA^d	Yes^c	Yes^c
HAS³³	No	Yes

CDR = Common Drug Review; G-BA=Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; INESSS = Institut national d'excellence en santé et en services sociaux; IQWiG = Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen; N/A = Not applicable; NICE= National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium.
^a NICE permits pre-registered interested parties to observe the public part of the committee meeting, but the remainder of the meeting takes place in private.³ In cases where it might not be possible to avoid referring to confidential or commercially sensitive information, meetings may be entirely closed.³
^b For SMC, the discussion is public, but a vote is held in private.
^cBased on survey results.
^d For the G-BA, deliberations of the plenum are usually public, but the final deliberation is always private.
Note: This table was informed by survey questions 18 and 19 in Appendix 2. (Survey question 18: Are the meetings of the committee held in public? [Yes/No] Question 19: Are the supporting information and the basis of the recommendations made publicly available? [Yes/No]) This question was not applicable to IQWiG.

Table 15: Deliberative Process: Details Regarding Conditional Reimbursement and Coverage-With-Evidence Development Schemes

HTA Organization	Committee Recommends, or Operates, Conditional Reimbursement (With Details)	Committee Recommends, or Operates, Coverage-With-Evidence-Development Schemes
CDR, Canada's Drug Agency⁴	Yes CADTH’s drug expert committees (the Canada's Drug Agency Canadian Drug Expert Committee [CDEC] and the Canada's Drug Agency pCODR Expert Review Committee [pERC]) may recommend that a drug be reimbursed with clinical criteria and/or conditions, in addition to recommending that a drug be reimbursed, or not be reimbursed.²³ Scenarios that could be considered under the “reimburse with clinical criteria and/or conditions” category include when the drug under review demonstrates: “…comparable or added clinical benefit and acceptable cost/cost-effectiveness relative to one or more appropriate comparators in a subgroup of patients within the approved indication. In such cases, the subgroup is specified through ‘clinical criteria.’ “…comparable clinical benefit and acceptable cost/cost-effectiveness relative to one or more appropriate comparators. In such cases, a condition may include that the drug be listed in a similar manner to one or more appropriate comparators. “…comparable or added clinical benefit, but the cost/cost-effectiveness relative to one or more appropriate comparators is unacceptable. In such cases, a condition may include a reduced price. “…clinical benefit, with a greater degree of uncertainty and an acceptable balance between benefits and harms, in a therapeutic area with significant unmet clinical need. In such cases, if the cost/cost-effectiveness relative to one or more appropriate comparators is unacceptable, a condition may include a reduced price.”²³ Some of the examples of the “clinical criteria” include (but are not limited to): characteristics that identify a patient subgroup (e.g., comorbidity status; severity of disease or disease progression, etc.) or starting and stopping rules (e.g., duration of treatment, response to treatment, etc.)²³ Some of the examples of the “‘conditions” include (but are not limited to); cost considerations (e.g., reduction in price or not to be reimbursed at the submitted price, etc.); reimbursement limits (e.g., number of doses supported by clinical and cost-effectiveness evidence); characteristics of the care setting or reimbursement of the drug in a manner similar to comparator(s) that are reimbursed by the participating jurisdictions at the time of the review; real-world evidence development for scenarios where there is uncertain clinical benefit but significant unmet need or considerations related to companion diagnostics, as applicable.²³	No^a
pCODR, Canada's Drug Agency⁸		No^a
INESSS¹⁶	Yes The following categories apply: products recommended without restriction are listed in the “Regular Drug” section of the RGAM Formulary products with restrictions are listed in the “Exception Drug” section of the RGAM Formulary and of the Drug Formulary for Institutions, specifying the criteria for use recognized by INESSS. The criteria used for reimbursement are also recommended by the committee products with good efficacy and safety are recommended to be listed with condition if negotiations/agreements can improve cost-effectiveness products recommended not to be listed.^a	Yes^a
NICE^3,24	Yes The overall goal is to maximize health within limited resources, therefore the Committee can make recommendations: for a product reflecting its market authorization for a product that is more restrictive than its market authorization (e.g., based on patient subgroups or employing treatment-starting, continuation, and stopping rules) that a product be used only in research that a product be recommended, but that further research is also completed that a product not be recommended. It should be noted that the Committee (at NICE) is not able to make recommendations on the pricing of technologies to the NHS but can consider a patient access scheme subject to the arrangements detailed in the technology appraisal process guide(s).	Yes
SMC²⁸	Yes Criteria may be specified in a restricted acceptance (such as identifying a sub-population, starting and stopping criteria, and specifying a particular specialist who should prescribe the medicine).^a	No
PBAC¹⁹	Yes Medicines and medicinal products can be listed as: unrestricted benefits, which have no restrictions on their therapeutic uses for the purposes of subsidy restricted benefits, which can only be prescribed for specific therapeutic uses authority required (STREAMLINED) benefits, which are restricted and require the recording of a streamlined authority code authority-required benefits, which are restricted and can only be prescribed with previous approval from the Australian Government Department of Human Services or the Australian Government Department of Veterans’ Affairs. The requested restriction will be assessed based on clinical evidence, such as the patient population, stage of disease, and line of treatment. The Australian Department of Health will negotiate the wording of the restriction and the price of the drug with the sponsor. Under certain circumstances, PBAC makes recommendations on conditional reimbursement, such as risk-sharing arrangements (which are agreements between the sponsor and the government to cap the maximum financial expenditure to submissions’ estimates, with a 100% rebate thereafter). Additionally, a “pay-for-performance” arrangement can be made whereby the sponsor rebates to the government the cost of treatment based on patients’ responses.^a	Yes^{a 6}
PHARMAC⁹	Yes PTAC can recommend pharmaceuticals subject to conditions for sub-populations, entry and exit criteria, or a drug’s pricing relationship to similar products in the same class of drugs.^a Products may be funded with restrictions on prescriber type, or with requirement for prior approval for individual patients according to clinical criteria, including indications and response to treatment. Prices are agreed upon in negotiated contracts that may feature rebates, budget caps, time limits, and any other commercial terms.^a	Yes^a
G-BA	No^a While conditional reimbursement is not used in Germany, it is possible that the decision on additional benefit may be limited by time (typically 1 to 3 years) in order to re-review the evidence when additional studies are finished. The drugs are on the market during that time and a change of decision of the G-BA would not affect market availability.^a	No^a

CDR= Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HTA = health technology assessment; INESSS = Institut national d'excellence en santé et en services sociaux; NHS = National Health Services; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; PTAC = Pharmaceutical and Therapeutics Advisory Committee; RGAM = Régime général d’assurance medicaments; SMC = Scottish Medicines Consortium.
^aBased on survey results.
Note: This table was informed by survey question 20 in Appendix 2 (Survey question: Does the committee recommend, or operate, conditional reimbursement? [Yes/No] [Pease provide details.] Does the committee recommend, or operate, coverage-with-evidence-development schemes? [Yes/No] [Please provide details.]) This question was not applicable to Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). No information was available on the Haute Autorité de santé (HAS).

Table 16: Deliberative Process: Stakeholder Involvement With Draft Recommendations/Guidance

HTA Organization	Process Allows Stakeholders to Comment on the Draft Recommendation/ Guidance (See Table 10 for process to allow stakeholders to comment on Draft or Final Report)	If Yes, Stakeholders Allowed to Comment on the Draft/Final Report				Process Allows Stakeholders to Appeal Against Recommendations/ Decisions	If Yes, Stakeholders Allowed to Appeal Against Recommendations/Decisions
HTA Organization		Manufacturer	Payer	Patient Groups	General Public		Manufacturer	Payer	Patient Groups	General Public
CDR, Canada's Drug Agency⁴	Yes	Yes	Yes	No	No	No^c	N/A
pCODR, Canada's Drug Agency⁸	Yes^a,b	Yes	Yes	Yes	No	No^c	N/A
INESSS	No^a	N/A				No^d	N/A
NICE³	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes^a	Yes	No
SMC²⁸	Yes	Yes	No	No	No	Yes	Yes	No	No	No
PBAC¹⁹	No	N/A				No^d	N/A
PHARMAC⁹	Yes	Yes	Yes^a	Yes	Yes	No^a,d	N/A
G-BA	Yes^a	No	Yes^a	Yes^a	No	No^a	N/A
HAS³³	Yes	Yes	No	No	No	XX	XX	XX	XX	XX

CDR = Common Drug Review; G-BA = Gemeinsamer Bundesausschuss; HAS = Haute Autorité de santé; INESSS = Institut national d'excellence en santé et en services sociaux; N/A = not applicable; NICE = National Institute for Health and Care Excellence; PBAC = Pharmaceutical Benefits Advisory Committee; pCODR = pan-Canadian Oncology Drug Review; PHARMAC = Pharmaceutical Management Agency; SMC = Scottish Medicines Consortium; XX = information not available.
^aBased on survey results.
^bIn addition to patient groups and the Provincial Advisory Group, clinicians can provide feedback on a pCODR Expert Review Committee initial recommendation.²²
^cCDR and pCODR programs include provisions for a review. In the case of CDR, a manufacturer may file for reconsideration if “CDR and/or the Canadian Drug Expert Committee failed to act fairly and in accordance with its procedures in conducting the review, and/or the Canadian Drug Expert Committee recommendation is not supported by the evidence that had been submitted or the evidence identified in the CDR review report(s).”⁴ In the case of pCODR, a party (e.g., manufacturer, tumour group, patient group, registered clinician, or the Provincial Advisory Group) that participated in the pCODR review for a specific drug product may make a procedural review request to the Canada's Drug Agency Chief Executive Officer and President. “A procedural review may be requested on the basis that: (i) pCODR failed to act in accordance with its procedures in conducting the review, as described in the pCODR Procedures; or (ii) pERC failed to apply its deliberative framework in formulating its recommendation, as outlined in the pERC Deliberative Framework.”⁸ The grounds relate only to whether or not process was followed and not to the content of the pERC Final Recommendation.
^dINESSS, PBAC, and PHARMAC permit resubmissions with new evidence.
^eThe G-BA also allows physicians and hospital representatives to comment.
Note: This table was informed by survey questions 21 and 22 in Appendix 2. (Survey question 21: Does the organization’s process allow stakeholders to comment on the draft recommendation/guidance? [Yes/No] If yes, which of the following stakeholders are allowed to comment? [Please select all that apply — Manufacturer / Payer / Patient groups / General public]. Question 22: Does the organization’s process allow stakeholders to appeal against recommendations/decisions? [Yes/No] If yes, which of the following stakeholders are allowed to appeal? [Select all that apply — Manufacturer / Payer / Patient groups / General public] If yes, are such appeals considered by the same committee or a different committee? [Please provide details]) This question was not applicable to Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG).

About this Document

Authors: Tara Cowling, Sirjana Pant.

Canada's Drug Agency would like to acknowledge Brendan McIntosh for his contribution to the report

Cite as: Single drug technology assessment processes across health technology assessment organizations. Ottawa: Canada's Drug Agency; 2016 Dec. (Environmental scan; no. 58).

Disclaimer: The information in this document is intended to help Canadian health care decision-makers, health care professionals, health systems leaders, and policy-makers make well-informed decisions and thereby improve the quality of health care services. While patients and others may access this document, the document is made available for informational purposes only and no representations or warranties are made with respect to its fitness for any particular purpose. The information in this document should not be used as a substitute for professional medical advice or as a substitute for the application of clinical judgment in respect of the care of a particular patient or other professional judgment in any decision-making process. The Canadian Agency for Drugs and Technologies in Health (CADTH) does not endorse any information, drugs, therapies, treatments, products, processes, or services.

While care has been taken to ensure that the information prepared by Canada's Drug Agency in this document is accurate, complete, and up-to-date as at the applicable date the material was first published by Canada's Drug Agency, Canada's Drug Agency does not make any guarantees to that effect. Canada's Drug Agency does not guarantee and is not responsible for the quality, currency, propriety, accuracy, or reasonableness of any statements, information, or conclusions contained in any third-party materials used in preparing this document. The views and opinions of third parties published in this document do not necessarily state or reflect those of Canada's Drug Agency.

Canada's Drug Agency is not responsible for any errors, omissions, injury, loss, or damage arising from or relating to the use (or misuse) of any information, statements, or conclusions contained in or implied by the contents of this document or any of the source materials.

This document may contain links to third-party websites. Canada's Drug Agency does not have control over the content of such sites. Use of third-party sites is governed by the third-party website owners’ own terms and conditions set out for such sites. Canada's Drug Agency does not make any guarantee with respect to any information contained on such third-party sites and Canada's Drug Agency is not responsible for any injury, loss, or damage suffered as a result of using such third-party sites. Canada's Drug Agency has no responsibility for the collection, use, and disclosure of personal information by third-party sites.

Subject to the aforementioned limitations, the views expressed herein are those of Canada's Drug Agency and do not necessarily represent the views of Canada’s federal, provincial, or territorial governments, or any third-party supplier of information.

This document is prepared and intended for use in the context of the Canadian health care system. The use of this document outside of Canada is done so at the user’s own risk.

This disclaimer and any questions or matters of any nature arising from or relating to the content or use (or misuse) of this document will be governed by and interpreted in accordance with the laws of the Province of Ontario and the laws of Canada applicable therein, and all proceedings shall be subject to the exclusive jurisdiction of the courts of the Province of Ontario, Canada.

The copyright and other intellectual property rights in this document are owned by Canada's Drug Agency and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to Canada's Drug Agency and its licensors.

About Canada's Drug Agency: Canada's Drug Agency is an independent, not-for-profit organization responsible for providing Canada’s health care decision-makers with objective evidence to help make informed decisions about the optimal use of drugs, medical devices, diagnostics, and procedures in our health care system.

Funding: Canada's Drug Agency receives funding from Canada’s federal, provincial, and territorial governments, with the exception of Quebec.

Dernière mise à jour : 09 décembre 2016