NICE public health guidance
Issued: February 2008 (last modified: November 2013)
PH10

Smoking cessation services

This is an extract from the guidance. The complete guidance is available at guidance.nice.org.uk/ph10

Appendix B: summary of the methods used to develop this guidance

Introduction

The reports of the reviews, expert reports and economic analyses include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it. The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E and are available from the NICE website.

The guidance development process

The stages of the guidance development process are outlined in the box below:

1. Draft scope

2. Stakeholder meeting

3. Stakeholder comments

4. Final scope and responses published on website

5. Reviews and cost-effectiveness modelling

6. Synopsis report of the evidence (executive summaries and evidence tables) circulated to stakeholders for comment

7. Comments and additional material submitted by stakeholders

8. Review of additional material submitted by stakeholders (screened against inclusion criteria used in reviews)

9. Synopsis, full reviews, supplementary reviews and economic modelling submitted to the PDG

10.The PDG produces draft recommendations

11. Draft recommendations published on website for comment by stakeholders and for field testing

12. The PDG amends recommendations

13. Responses to comments published on website

14. Final guidance published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by the PDG. The overarching question was: 'What is the optimal provision of smoking cessation services, including the provision of nicotine replacement therapy (NRT), for primary care, pharmacies, local authorities and workplaces, with particular reference to manual working groups, pregnant women who smoke and hard to reach communities?' The subsidiary questions were:

  1. What is the aim or objective?

2. What is the content and how does it influence effectiveness?

3. How does the way that the intervention is carried out influence effectiveness?

4. Does effectiveness depend on the job title/position of the person delivering the intervention? What are the significant features of an effective leader?

5. Does the site/setting influence effectiveness?

6. Does the intensity (or length or frequency) influence effectiveness or duration of effect?

7. How does effectiveness vary according to factors such as the age, sex, class or ethnicity of the target audience?

8. How much does the intervention cost (in terms of money, people, time)?

9. What evidence is there on cost effectiveness? Does the intervention offer value for money?

10 What are the facilitators and barriers to implementation?

These questions were refined further in relation to the topic of each review (see reviews for further details).

Reviewing the evidence of effectiveness

Five reviews of effectiveness were conducted.

Identifying the evidence

The following databases were searched for meta-analyses, systematic reviews of randomised controlled trials (RCTs), individual RCTs, systematic reviews of non-RCTs, case-control studies, cohort studies, interrupted time series studies, correlational studies, controlled before-and-after studies, non-analytic studies (for example case reports, case studies) and expert opinion (1900–2007):

  • AMED

  • ASSIA

  • British Nursing Index

  • CINAHL

  • Cochrane Database of Systematic Reviews

  • Cochrane Controlled Trials Register (CENTRAL)

  • Controlled Clinical Trials

  • Database of Abstracts of Reviews of Effects

  • DARE

  • DH-Data

  • EMBASE

  • Google Scholar

  • Health Technology Assessment Database

  • HSTAT

  • King's Fund

  • MEDLINE (Ovid)

  • National Guideline Clearinghouse

  • National Research Register (including CRD ongoing reviews database and unpublished reports)

  • NICE web pages (published appraisals)

  • PsycINFO

  • SIGN Guidelines

  • Sociological Abstracts

  • TRIP.

In addition, for the NHS Stop Smoking Services review, telephone interviews were carried out with 12 people working in tobacco cessation.

These questions were refined further in relation to the topic of each review (see reviews for further details).

Expert report

The expert report on mass media interventions (see appendix A for details) identified both unpublished and published data produced between 1996 and 2006.

Further details of the databases, search terms and strategies are included in the review reports.

Selection criteria

Inclusion and exclusion criteria for each review (see appendix A for details) varied and details can be found online. However, in general:

  • Review 1 included systematic reviews and meta-analyses that focused on the most widely advertised, commercially available smoking cessation treatments in the UK. This included pharmacological and behavioural treatments where there was published research available on their effects.

  • Review 2 included reviews, RCTs and non-randomised studies that evaluated the effectiveness of intensive treatments for smoking within the NHS, in particular, those offered by the NHS Stop Smoking Services.

  • Review 3 included reviews and other studies of selective or indicated interventions that evaluated the effectiveness of workplace policies in England to support smoking cessation.

  • Review 4 included reviews and other studies on mass media and community interventions that both encourage quit attempts and reinforce current and recent attempts to quit smoking among all population groups.

  • Review 5 included reviews and other studies of telephone interventions for smoking cessation where telephone support was a key intervention component, or an adjunct to brief advice, and where it could be evaluated independently of the other intervention components.

  • The expert paper on mass media interventions for smoking cessation included data (both published and unpublished) produced over the last 10 years (1996–2006).

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution:

Study type
  • Meta-analyses, systematic reviews of RCTs or RCTs (including cluster RCTs).

  • Systematic reviews of, or individual, non-RCTs, case-control studies, cohort studies, controlled before-and-after studies, interrupted time series studies, correlation studies.

  • Non-analytical studies (for example, case reports, case series).

  • Expert opinion, formal consensus.

Study quality

++ All or most of the criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The studies were also assessed for their applicability to the UK.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews and the synopsis).

The findings from the reviews, interviews and expert report were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements reflect the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Further details of the databases, search terms and strategies are included in the review reports.

Economic appraisal

The economic appraisal consisted of four economic evaluations and three economic analyses (See appendix A for details).

Review of economic evaluations

Three databases were searched for each cost-effectiveness review: NHS Economic Evaluation Database, Centre for Reviews and Dissemination (CRD) and the internal database results from the original effectiveness review.

The criteria for inclusion of papers were:

  • studies used a defined intervention to assist smoking cessation

  • the study population was smoking at the start of the study (although if drawn from a general population, it is accepted that some people may not smoke)

  • studies reported both the costs and effectiveness of an intervention (although costs and effectiveness did not have to be combined into a single cost-effectiveness ratio).

Ten papers were identified for the mass media economic review, no papers were identified for the economic review of non-NHS interventions, 18 papers were identified for NHS interventions and 10 papers were identified for workplace interventions.

Cost-effectiveness analysis

A cohort simulation model was designed to estimate the costs and quality-adjusted life years (QALYs) associated with smoking cessation. The model was designed to compare different smoking cessation interventions to determine their incremental cost-effectiveness.

To furnish the model with relevant data, the following databases were searched: MEDLINE and MEDLINE In-Process, NHS EED, HEED, CINAHL, HMIC, CRD (internal database) and PubMed. The World Wide Web and references listed in identified articles were also searched for relevant studies. Data were gathered on the following:

  • mortality, by age, gender and smoking status

  • prevalence of each comorbidity by age, gender and smoking status

  • utilities for each comorbidity

  • costs for each comorbidity

  • the annual cessation and cost of each intervention modelled.

The results are reported in: 'Cost effectiveness of interventions for smoking cessation' (Flack et al. 2007a) and 'Cost impact analysis of workplace-based interventions for smoking cessation' (Flack et al. 2007b). These reports are available on the NICE website.

Review of NICE technology appraisal 39: reviewing the evidence of effectiveness

Two effectiveness reviews were conducted to inform the update of NICE technology appraisal 39.

Identifying the evidence

'Cut down to quit with nicotine replacement therapies (NRT) in smoking cessation: systematic review of effectiveness and economic analysis'.

Searches were carried out for systematic reviews and primary studies from 1992–2006 on the following databases: Cochrane reviews, Cochrane Collaboration (via Cochrane Library), Database of Abstracts of Reviews of Effectiveness (DARE), Cochrane Central Register of Controlled Trials (CENTRAL), Health Technology Assessment (HTA) database and ARIF Database, NHS CRD, Bandolier, TRIP, MEDLINE, EMBASE, CINAHL and PsycINFO.

Information was also gathered from theScience Citation Index (Web of Science), National Research Register and citations of relevant studies and reviews. In addition, further information was sought from regional experts, especially Pharmacy Prescribing Unit, Keele University (&MTRAC) and the West Midlands Drug Information Service and from licensing authority and industry documents.

'Clinical and cost-effectiveness of nicotine replacement therapy for new licensed indications and combination therapy: a summary of best evidence'.

Specific searches related to the aims of each report were carried out in the following databases: The Cochrane Library, MEDLINE (1966–2006) and EMBASE (1980–2006). Generic searches were also conducted to gather cost-effectiveness information in: OHE HEED (August 2006) and MEDLINE. Searches for ongoing studies were conducted in the National Research Register.

Further details of the databases, search terms and strategies are included in the review reports.

Selection criteria

Inclusion and exclusion criteria for the Cut down to quit (CDTQ) reviews were:

  • at least one electronic database (for example, MEDLINE) was scrutinised using a stated search strategy

  • RCT studies of CDTQ were reviewed

  • quit rates were quantitatively reviewed and/or meta-analysed.

Inclusion and inclusion criteria for primary studies of CDTQ were:

  • RCTs were undertaken of people who were currently unable or unwilling to quit smoking abruptly

  • interventions included the use of NRT gum or NRT inhalator alone, or as part of combination therapy (for example, motivational support)

  • the comparator was: placebo or no treatment, non-NRT drugs for smoking cessation, psychological interventions (for example, motivational support) for quitting. Where an adjunct therapy was used in the intervention it had to be used with the comparator

  • quit rates had to be provided.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution:

Study type
  • Meta-analyses, systematic reviews of RCTs or RCTs (including cluster RCTs).

  • Systematic reviews of, or individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after studies, interrupted time series studies, correlation studies.

  • Non-analytical studies (for example, case reports, case series).

  • Expert opinion, formal consensus.

Study quality

++ All or most of the criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The studies were also assessed for their applicability to the UK.

Review of NICE technology appraisal 39: economic appraisal

The economic appraisal consisted of two effectiveness reviews.

Review of economic evaluations

'Cut down to quit with nicotine replacement therapies (NRT) in smoking cessation: systematic review of effectiveness and economic analysis'. The following databases were searched: Cochrane Library (Wiley) 2006 Issue 2, MEDLINE (Ovid) 1992–July 2006, MEDLINE In-Process (Ovid) 12 July 2006, EMBASE (Ovid) 1992–week 27 2006, CINAHL (Ovid) 1992–July 2006, PsycINFO (Ovid) 1992–July 2006, Science Citation Index (Web of Science) 1992–July 2006.

In addition, other searches included the research registries of ongoing trials: National Research Register 2006 Issue 2, Current Controlled Trials metaRegister and ClinicalTrials.gov, and citations of relevant studies and reviews. Further information was also sought from experts and from licensing authority and industry documents.

Relevant systematic reviews, randomised controlled trials and existing economic analyses of CDTQ were identified.

RCTs were included if:

  • the populationconsisted of people who were unable or unwilling to attempt to quit smoking in the short term

  • the interventions encompassed a cut-down smoking programme supported by NRT

  • the comparator was a cut-down programme with placebo or other support.

Systematic reviews were included if:

  • at least one electronic database had been searched and RCTs documenting quit rates in NRT smoking reduction programmes were reviewed.

Economic studies were included if they encompassed cost effectiveness or cost-utility analysis of CDTQ programme(s).

'Clinical and cost-effectiveness of nicotine replacement therapy for new licensed indications and combination therapy: a summary of best evidence'.

The following databases were searched: MEDLINE, EMBASE, the Cochrane library and OHE HEED. Studies were included if they covered:

  • NRT

  • the relevant population/combination as a systematic review

  • the relevant population/combination as an RCT (if a systematic review was identified only more recent RCTs were sought).

Cost effectiveness analysis

'Cut down to quit with nicotine replacement therapies (NRT) in smoking cessation: systematic review of effectiveness and economic analysis'.

No existing economic analyses of CDTQ were identified. A 'de novo' decision analytic model was constructed to estimate the cost-effectiveness of making CDTQ with NRT available for people who were unwilling or unable to attempt an abrupt quit. The outcome measure was expected quality-adjusted life years (QALYS). The model also took account of the possibility that some people willing to attempt abrupt quitting might instead switch to CDTQ. People moving from an abrupt quit attempt to CDTQ were assumed to either experience a 'CDTQ success rate' or to retain the abstinence success rate of abrupt quitters.

'Clinical and cost-effectiveness of nicotine replacement therapy for new licensed indications and combination therapy: a summary of best evidence'.

There were no economic analyses specifically addressing the cost effectiveness of NRT for adolescents, pregnant women, breastfeeding women, combination therapy NRT + NRT, combination therapy NRT + bupropion or cardiovascular patients.

It was not possible to undertake any modelling with the resources available. Searches for existing models did not identify any models for adolescents, pregnancy, breastfeeding, cardiovascular disease or combination treatment.

Fieldwork

Fieldwork was carried out to evaluate the relevance and usefulness of NICE guidance for practitioners and the feasibility of implementation. It was conducted with practitioners and commissioners who are involved in smoking cessation services. They included those working in general practice, maternity services, secondary care, community pharmacies, dentistry, strategic health authorities and primary care teams, NHS Stop Smoking Services and regional networks in the NHS and appropriate charities.

The fieldwork comprised:

  • Group discussions carried out in Greater Manchester, Kirklees, Merseyside, Leicester, London and the South East, Birmingham and West Midlands with:

    • smoking cessation teams

    • GPs

    • practice nurses

    • community pharmacists

    • dentists

    • midwives

    • hospital staff

    • SHA directors of public health.

  • In-depth interviews were conducted individually, in pairs and in trios in Greater Manchester, Kirklees, Merseyside, Leicester, London and the South East, Birmingham and the West Midlands and by telephone.

  • The studies were commissioned to ensure there was ample geographical coverage. The main issues arising from the five group discussions and 23 in-depth interviews with individual professionals either singly, in pairs or in trios, are set out in appendix C under 'Fieldwork findings'. The full fieldwork report is available on the NICE website.

How the PDG formulated the recommendations

At its meetings held between May 2006 and September 2007, the PDG considered the evidence of effectiveness, expert reports and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of effect.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope

  • effect size and potential impact on population health and/or reducing inequalities in health

  • cost effectiveness (for the NHS and other public sector organisations)

  • balance of risks and benefits

  • ease of implementation and the anticipated extent of change in practice that would be required.

The PDG also considered whether a recommendation should only be implemented as part of a research programme, where evidence was lacking.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in May 2007. At its meetings in June and September 2007, the PDG considered comments from stakeholders and the results from fieldwork The guidance was signed off by the NICE Guidance Executive in December 2007.