Reducing differences in the uptake of immunisations Appendix B:... PH21

Introduction

The review and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PHIAC meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E and are available online.

Guidance development

The stages involved in developing public health intervention guidance are outlined in the box below.

1. Draft scope released for consultation

2. Stakeholder meeting about the draft scope

3. Stakeholder comments used to revise the scope

4. Final scope and responses to comments published on website

5. Evidence review(s) and economic analysis undertaken

6. Evidence and economic analysis released for consultation

7. Comments and additional material submitted by stakeholders

8. Review of additional material submitted by stakeholders (screened against inclusion criteria used in review/s)

9. Evidence and economic analysis submitted to PHIAC

10. PHIAC produces draft recommendations

11. Draft guidance released for consultation and for field testing

12. PHIAC amends recommendations

13. Final guidance published on website

14. Responses to comments published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by PHIAC to help develop the recommendations. The overarching questions were:

What interventions are effective and cost effective at reducing differences in immunisation uptake in children and young people 19 years or younger?

What are the views and experiences of parents and carers, those receiving and those delivering either immunisations themselves or interventions to increase uptake of immunisations in the UK to children and young people 19 years or younger?

Reviewing the evidence of effectiveness

A review of effectiveness was conducted for each intervention that reduces differences in immunisation uptake.

Identifying the evidence

The following databases were searched for published literature (1 January 1988 to 31 March 2008):

ASSIA
Campbell Collaboration
CINAHL
Cochrane Library (Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effectiveness [DARE])
Embase
Eppi-centre databases
ERIC
Medline
PsycINFO
Sociological Abstracts

The review team contacted relevant external people for additional information and also searched the following websites for relevant studies:

CDC
American Academy of Pediatrics
Canadian Coalition for Immunization awareness and Promotion
Canadian Pediatric Society
Department of Health
DIPEX: personal experiences of health and illness
European Centre for Disease Prevention and Control
Eurosurveillance
Evidence for Social Policy and Practice Co-ordinating Centre
Health Evidence Bulletins Wales
Health Protection Agency
Health Protection Scotland
Immunisation advisory centre
Immunise Australia
Intute (previously OMNI)
National Centre for Immunisation Research and Surveillance
NHS Quality Improvement Scotland
NHS Wales
Public Health Organization of Canada
Scottish Intercollegiate Guidelines Network (SIGN)
US Centers for Disease Control and Prevention
Vaccine Education Center, Philadelphia Children's Hospital
World Health Organization

Further details of the databases, search terms and strategies are included in the review reports.

Selection criteria

Qualitative studies were included in the effectiveness review if:

they took place in the UK
they reported on the knowledge, attitudes, values and beliefs relating to immunisations for children and young people under 19 years
they reported on immunisation uptake rates in people under 19.

Quantitative and economic studies were included in the effectiveness review if they reported on interventions that seek to reduce differences in the uptake of universal or targeted vaccination programmes for children and young people under 19 years.

Studies were excluded if they:

were published in a language other than English
were conducted in developing countries
reported interventions that sought to reduce differences in the uptake of immunisations in people aged 19 or older
explored the setting of national immunisation strategies, policies, priorities and targets
targeted vaccination of young people at occupational risk of infection (for example, vaccination of healthcare workers for hepatitis B and varicella)
targeted vaccination of children and young people travelling to countries with increased prevalence of infectious agents (for example, vaccination for typhoid, rabies or tick-borne encephalitis)
targeted vaccination of children and young people who were clinically at risk of infection with a vaccine-preventable disease as a result of an underlying condition (for example, vaccination of asplenic or immunocompromised people for pneumococcal or Haemophilus influenzae type b infections)
aimed to increase uptake of single vaccines for measles, mumps and rubella
did not report findings from primary research (for example, were secondary reviews of the literature)
were published before 1988
were published as abstracts only or were not held by the British Library.

The evidence in this review was subject to further analysis and revision. This revision was carried out by NICE.

The revised analysis excluded studies if they were not transferrable to the UK context. It also excluded studies if they:

involved the provision of free vaccines either alone or as part of a health insurance package
involved immunisation-linked, provider payments on a capitation or fee-for-service basis
reported baseline coverage levels of less than 70% (with the exception of MMR coverage)
presented post-intervention and control (no-intervention) levels less than 70% (however, if multiple vaccines or age groups were considered and at least one baseline level was greater than 70%, then the study was included)
aimed to increase uptake of human papillomavirus vaccine.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix E). Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the methodology checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are thought very unlikely to alter.

+ Some of the methodology checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

– Few or no methodology checklist criteria have been fulfilled. The conclusions of the study are thought likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data was summarised in the revised analysis and in evidence tables.

The findings from the revised analysis were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by NICE. The statements reflect their judgement of the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

The economic analysis consisted of a review of economic evaluations and a cost-effectiveness analysis which consisted of three economic models.

Review of economic evaluations

The following databases were searched for the period from 1 January 1988 to 31 March 2008:

Health Economics Evaluation Database(HEED)
Econlit (1969–March 2008)
Health Technology Assessment
NHS Economic Evaluation Database (NHS EED).

Cost-effectiveness analysis

An economic model for measles was constructed 'The impact of increasing vaccine coverage on the distribution of disease: measles in the UK'. Further results are reported in: 'An exploration of the cost-effectiveness of interventions to reduce the difference in uptake of childhood immunisations in the UK using threshold analysis'.

Additional economic modelling was undertaken to produce: 'The estimated cost-effectiveness of vaccination in infants born to hepatitis-B-virus-positive mothers'. The three economic modelling reports are available on the NICE website.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in immunisation within primary care, public health and children's services in the NHS.

The fieldwork comprised two studies commissioned to ensure ample geographical coverage:

Seven workshops carried out in Birmingham, Brighton, London and Manchester with a range of practitioners including immunisation coordinators, nurses, paediatricians, GPs and directors of public health.
Thirty telephone interviews with a range of professionals including practice nurses and managers, community and neonatal paediatricians and those involved in PCT information services.

The main issues arising from the two studies are set out in appendix C under 'Fieldwork findings'. The full fieldwork report, 'Reducing differences in the uptake of immunisations (including targeted vaccines) in children and young people aged under 19 years'.

How PHIAC formulated the recommendations

At its meetings in October 2008, January, March and June 2009, PHIAC considered the evidence of effectiveness and the revised analysis, plus the cost effectiveness to determine:

whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement
whether, on balance, the evidence demonstrates that the intervention is effective, ineffective or equivocal
where there is an effect, the typical size of effect.

PHIAC developed draft recommendations through informal consensus, based on the following criteria.

Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.
Effect size and potential impact on the target population's health.
Impact on inequalities in health between different groups of the population.
Cost effectiveness (for the NHS and other public sector organisations).
Balance of risks and benefits.
Ease of implementation and any anticipated changes in practice.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in May 2009. At its meeting in June 2009, PHIAC amended the guidance in light of comments from stakeholders, experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in August 2009.

Reducing differences in the uptake of immunisations

Appendix B: Summary of the methods used to develop this guidance