Process and methods guides
Published 04 April 2013

Guide to the methods of technology appraisal 2013

This is an extract from a NICE publication. The complete publication is available at http://publications.nice.org.uk/PMG9

4 Involvement and participation

NICE will normally receive evidence from:

  • an independent academic group

  • manufacturers and sponsors of technologies

  • national patient or carer groups

  • healthcare professional organisations

  • clinical specialists, commissioning experts and patient experts

  • commissioning bodies.

    Detailed information for individual groups participating in an appraisal who wish to submit written or oral evidence is provided in the additional documents listed in section 1.1.3 and is available on the NICE website.

4.1 Independent academic groups

4.1.1 A group of independent experts from 1 of a number of academic centres is commissioned by the NHS National Institute for Health Research (NIHR) through the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) to review and critique the available evidence for each technology under appraisal. Groups commissioned for appraisals in the MTA process are referred to as Assessment Groups, whereas those commissioned for appraisals in the STA process are referred to as Evidence Review Groups. The reports they produce are the responsibility of the authors.

Assessment groups (MTA process)

4.1.2 In the MTA process, the Assessment Group independently synthesises the evidence from published information and the submissions from manufacturers and sponsors about the clinical and cost effectiveness of the technology or technologies. The report focuses on the evidence relevant to the scope (see section 5.1.4).

4.1.3 In addition to a systematic review of the evidence on clinical effectiveness and a review of published cost-effectiveness studies, the assessment report will normally include a cost-effectiveness analysis informed by a review of the clinical-effectiveness evidence. This analysis should conform to the requirements of the reference case (see section 5).

Evidence review groups (STA process)

4.1.4 In the STA process, the Evidence Review Group prepares a report, which assesses the submission provided by the manufacturer or sponsor of the technology (see section 4.2). The Evidence Review Group may recommend that the Institute request additional analysis from the manufacturer or sponsor, and/or may undertake additional exploratory analyses itself.

4.2 Manufacturers and sponsors

4.2.1 Submissions are invited from manufacturers and sponsors (organisations who market the technology under licence) of the technology or technologies being appraised. Manufacturers and sponsors should identify all evidence relevant to the appraisal. This includes a list of all studies known to them, including clinical trials, follow-up studies and evidence from disease registries. They may also include relevant study evidence to which they have privileged access and which is not in the public domain. In particular, when technologies are undergoing appraisal immediately before regulatory approval, sufficient details of the clinical trial evidence should be made available to enable the Institute to conduct the appraisal according to the defined scope.

4.2.2 At the earliest opportunity, the Institute will ask manufacturers or sponsors to make available details of the studies they intend to include in their submissions. When there is unpublished information, the Institute will request the study reports.

4.2.3 In the STA process the manufacturer or sponsor is required to provide a systematic review of the evidence on clinical and cost effectiveness and an assessment of cost effectiveness containing a reference-case analysis based on clinical-effectiveness evidence. This submission forms the principal evidence base to estimate clinical and cost effectiveness. The manufacturer or sponsor must justify any cost-effectiveness analysis that does not fulfil this reference-case requirement.

4.2.4 Further information on the content of manufacturer and sponsor submissions is available in the Institute's document: Single technology appraisal (STA): specification for manufacturer/sponsor submission of evidence.

4.3 Patient and carer groups

4.3.1 The Institute invites submissions from all patient and carer groups involved in the appraisal.

4.3.2 These written submissions may provide perspectives from patients and carers on:

  • the experience of having the condition, or in the case of carers, the experience of caring for someone with the condition

  • the experience of receiving care for the condition in the healthcare system

  • the experience of having specific treatments for the condition

  • the outcomes of treatment that are important to patients or carers (which may differ from the outcomes measured in the relevant clinical studies and the aspects of health included in generic measures of health-related quality of life)

  • the acceptability of different treatments and modes of treatment

  • their preferences for different treatments and modes of treatment

  • their expectations about the risks and benefits of the technology.

4.3.3 The information is best taken directly from people with the condition (or their family or carers) in the form of written accounts of their experiences and points of view. Narrative summaries, preferably with illustrative quotes, addressing the issues listed in section 4.3.2 are acceptable. Standard qualitative research techniques, such as thematic analysis, facilitate the synthesis of evidence of this type. Accounts and experiences may be collected and analysed systematically using these qualitative research techniques, but there is no requirement to present the information in this way. The Institute supports the collection of patient and carer evidence by use of a template, by offering the services of the Public Involvement programme, and by providing some financial support.

4.3.4 The Appraisal Committee is interested in a range of patient and carer perspectives, including majority views and views that may be held by only a few patients even if they contradict the majority. It is therefore important to include a range of views, especially when there are differences of opinion.

4.3.5 In the context of a technology appraisal, the Appraisal Committee is interested in limitations in the published research literature identified by patient groups and in particular the extent to which patient-reported outcome measures, or other end points reported in clinical studies, capture outcomes of importance to patients. They may assess research-based evidence from a different perspective to researchers and clinicians and they may judge the evidence according to different criteria. Additionally, it is helpful to have the perspective of patients or carers about how relevant the clinical outcomes and the standardised generic instruments for measuring health-related quality of life (as specified in the reference case, see section 5.3) are to the disease or condition of the appraisal.

4.4 Healthcare providers and commissioners of health services

4.4.1 The Institute invites submissions from all professional bodies and relevant NHS organisations involved in the appraisal, including:

  • the Royal Colleges of the appropriate clinical disciplines

  • the specialist societies of the appropriate clinical disciplines

  • other appropriate professional bodies and NHS organisations including commissioners of NHS services.

4.4.2 Healthcare professionals and commissioners of health services provide a view of the technology in relation to current clinical practice. This puts into context the evidence derived from pre- and post-licensing studies, which often relates to efficacy and safety under clinical trial conditions rather than effectiveness in routine clinical practice.

4.4.3 The written submissions provide a unique contribution, outlining the professional view of the place of the technology in current clinical practice and in the pathway of care. This includes evidence that relates to some or all of the following:

  • Variations between groups of patients, in particular, differential baseline risk of the condition and potential for different subgroups of patients to benefit.

  • Identifying appropriate outcome and surrogate outcome measures.

  • Significance of side effects or adverse reactions and the clinical benefits.

  • Circumstances in which treatment is delivered, including:

    • the need for concomitant treatments

    • the settings in which treatment is delivered (for example, primary or secondary care, or in specialist clinics)

    • the requirements for additional professional input (for example, community care, specialist nursing or other healthcare professionals).

  • Relevant potential comparators.

  • Information on unpublished evidence. Such information should be accompanied by sufficient details to enable a judgement as to whether it meets the same standards as published evidence and to determine potential sources of bias.

  • Evidence from registries and nationally coordinated clinical audit.

  • Published clinical guidelines produced by specialist societies.

  • The impact of possible guidance on:

    • delivery of care

    • education and training requirements of NHS staff

    • patients who would use the technology.

4.5 Clinical specialists, commissioning experts and patient experts

4.5.1 Three groups of experts – clinical specialists, commissioning experts and patient experts – are selected by the Appraisal Committee Chair from nominations provided by consultees and commentators. These experts provide written evidence and attend the Committee meeting to help in the Committee's discussion of the technology being appraised.

Format of the evidence

4.5.2 The Institute asks all experts attending the Committee meeting to submit, in advance, a brief written personal view of the current management of the condition and the expected role and use of the technology in the NHS, as well as to provide oral commentary during the meeting. The purpose of the written personal view is to make the expert's perspective transparent to those who did not attend the meeting. The purpose of the oral commentary provided by the experts is to explore the evidence that is provided in the written submissions from consultees. During the open part of the meeting, clinical specialists, commissioning experts and patient experts are encouraged to interact fully in the debate with the Committee, including responding to and posing questions. The clinical specialists, commissioning experts and patient experts are asked to withdraw from the meeting before the Committee discusses the content of the guidance.

4.5.3 Views expressed orally by the experts at the Committee meeting can inform the debate in a variety of ways, including the following:

  • Identifying important variations in clinical practice in both the management of the condition and specifically in the current use of the technology. This might include:

    • geographical variations

    • identification of subgroups

    • constraints on local implementation

    • specific issues for implementation that affect patients and carers directly.

  • Identifying the requirements for support to implement any guidance on the technology. This might include:

    • requirements for extra staff or equipment in NHS units

    • education and training requirements for NHS staff and for the patients on how to use the technology

    • special requirements within the community for patients and carers (for example, travel to hospital for treatment)

    • ways in which adherence to treatment can be improved.

  • Giving personal perspectives on the use of the technology and the difficulties encountered, including the important benefits to patients and the range and significance of adverse effects.

  • Providing views on assessing response to treatment and the circumstances in which treatment might be discontinued.

  • Identifying subgroups of patients for whom the benefits and risks of treatment might differ.

  • Responding to queries that arise from:

    • the lead team (which comprises 3 Committee members who make a presentation to introduce the appraisal topic)

    • issues raised by the Chair and other Committee members

    • issues raised by the Evidence Review Group or the Assessment Group

    • issues raised by other experts

    • issues raised by a response given by the manufacturer or sponsor to a question posed.