NICE Technology Appraisals in the NHS in England (Innovation Scorecard) To December 2020

This section of the report aims to provide users with an evidence-based assessment of the quality of the publication outputs by reporting against the nine European Statistical System (ESS) quality dimensions and principles.

In doing so, this meets NHS Digital’s obligation to comply with the UK Statistics Authority (UKSA) Code of Practice for Official Statistics, which is based on three pillars:

• Trustworthiness is about having confidence in the people and organisations that produce statistics and data.
• Quality is about using data and methods that produce assured statistics.
• Value is about producing statistics that support society’s needs for information.

Due to the provisional nature of some of the data included in the innovation scorecard, some figures may be revised from publication to publication as issues are uncovered and resolved. Where a refresh of data occurs, it will be clearly documented in the publications. Users should always use the figures in the latest publication to ensure they are the most up to date figures available.

Accuracy and reliability relates to the proximity between an estimate and the unknown true value.

Statistics in this publication are based on data from:

• English Prescribing Dataset (EPD) published by the NHS Business Services Authority
• Secondary Care Medicines Data (SCMD) published by the NHS Business Services Authority
• Hospital Episode Statistics (HES) and Quality Outcomes Framework (QOF) data from NHS Digital
• Mid-year population estimates from Office for National Statistics
• Defined Daily Doses (DDD) from World Health Organisation Collaborating Centre for Drug Statistics Methodology

Relevance is the degree to which the statistical product meets user needs in both coverage and content.

Medicines which meet the inclusion criteria of the Innovation Scorecard are presented at National (England), NHS Region level, STP, CCG and NHS Trust levels, by calendar quarter.

The medicine groupings are available in a separate dashboard on the web platform tool and users can see both the grouped uptake and the values for each individual medicine within the grouping. As such, medicines which form part of these medicine groupings will not be included in the individual medicine dashboard.

The web platform tool shows use of individual medicines, medicine groupings and medical technologies over time i.e. by calendar quarter, and at different NHS organisation levels i.e. National (England), Regions, STP, CCG and NHS Trust.

Coherence is the degree to which data that is derived from different sources or methods, but refers to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain.

There will be on-going developments with each Innovation Scorecard publication. Users should always use the figures in the latest publication to ensure they are the most up to date figures available. Previous Innovation Scorecard publications can be found at: 

The Prescription Services data presented here differs from that presented in the NHS Digital publications based on the Prescription Cost Analysis system. This is because the PCA database is based on all prescriptions written in England, Wales, Scotland, Northern Ireland and the Isle of Man but dispensed in England only and includes prescriptions written by dentists and hospital doctors.

The data reported as ADDs, DDDs or mgs in this publication will not match other prescribing data published by NHS Digital, which are generally reported as items and cost.

Users can misinterpret the data as relating to numbers of patients, but care should be taken as the data relates to volumes of medicines, not directly to the number of individuals.

Changes to the figures over time need to be interpreted in the context of changes in available medicines and changes in NHS practice. For example, a reduction in items dispensed for a particular medicine may be due to the introduction of alternative medicines, or a change in prescribing behaviour, especially in the length of treatment each item is intended to cover. Additionally, a change in prescribing practice could also be due to drug safety updates as published by the Medicines and Healthcare products Regulatory Agency and its independent advisor the Commission on Human Medicines.

Details of drug safety updates can be found at:

Sales and purchases data reflect when medicines and medical technologies were purchased and not when they were actually used. Sales and purchases may differ between quarters due to economic reasons rather than medical reasons. For example Trusts may bulk buy in one quarter but use the product in several subsequent quarters.

Local level data (Regions, STPs, CCGs, Trusts) will not add up to national data due to unidentified organisations which are included in the national totals but not against local level organisations.

NHS organisations differ widely in the populations they serve so data at National (England), NHS Regions, STPs and CCG levels are standardised by the estimated resident populations.

For hospital Trusts data the number of FCE days of hospital care for the time period under consideration (taken from the Hospital Episode Statistics data) has been used to standardise the data. The values vary significantly, with more specialist hospitals, for example, the Royal National Hospital for Rheumatic Diseases NHS Foundation Trust reporting fewer than 10,000 days of hospital care per year, whereas larger Trusts such as Bart’s Health NHS Trust report over 800,000 days of hospital care per year.

Trust level data should not be compared with the national, Region, STP or CCG data due to the differing data sources and standardisation methodologies applied.

This scorecard covers some highly specialised medicines and technologies so differences in use across organisations are to be expected.

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

This report is published quarterly and reports on data which is approximately six months in arrears. The publication date is determined by the availability of the data and allows adequate time for the compilation of the report including all other publication outputs. The aim with the transition to the SCMD from the NHS BSA is to reduce the lag time in available data to less than 4 months.

New medicines with a positive recommendation on published TAs are made available on the Innovation Scorecard approximately six months in arrears, although the aim is to reduce this to less than 4 months. 

Organisational changes are published in line with the reporting periods of the Innovation Scorecard and are not relative to the publication dates of the releases. This means that for an organisational change which takes effect from April 2021 will only be reflective from the October 2021 release of the Innovation Scorecard onwards and where possible, historic data updated on the new releases.

Similarly, DDD refreshes are also implemented in line with the reporting periods of the Innovation Scorecard, and not relative to the publication dates of the releases.

New and amended DDDs, including those for combined products, are released twice annually i.e. around May and December. The ATC searchable index with DDDs is updated in January and is to be used for prescribing from January onwards. This means that new and updated DDD figures will take effect from the October release of the Innovation Scorecard onwards, with historic data updated accordingly.

This publication has been released in line with the pre-announced publication date and is therefore deemed to be punctual.

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

This publication uses the open data published by the NHS BSA.

The publication may be requested in large print or other formats through the NHS Digital’s contact centre: [email protected] (please include ‘Innovation Scorecard’ in the subject line).

This report provides a high-level summary of the medicines reported on for the current publication as well as some analysis of utilisation comparing the current 12 months with the previous 12 months.

Also available is a key points infographic and frequently asked questions as well as some contextual information where applicable.

Guidance documents on details of the underlying data as well as methodology specifications for the medicine groupings are provided as resources of the publication.

Data is presented on a new web platform tool following an initial release in January 2017. This new tool has been developed in response to a user consultation undertaken in early 2016 specifically to enhance user experience in accessing the data, making it easier for commissioners and users to find information on what medicines are available in their region and allow for easier comparison with other areas. 

User feedback is welcomed to feed ongoing and future developments.

This section describes the processes for finding out about users and uses and their views on the Innovation Scorecard publication.

Comments on the Innovation Scorecard publication can be made through various media:

• NHS Digital general enquiries email [email protected] and/or telephone number 0300 303 5678
• Twitter @nhsdigital

A user survey of the Innovation Scorecard publication in Feb 2016 was conducted to collect feedback on the content and display of the current publication. The results of the survey will be used to ensure the publication remains relevant to users. Some of the recent developments in response to the feedback received include:

• Key points infographic to summarise the contents of the publication  
• A web platform tool to enhance user experience in accessing the data
• Refreshed reports with user friendly commentary and charts
• Guidance documents on the underlying data
• Specification details for the medicine groupings

The Innovation Scorecard strategic group and technical working group consist of a range of stakeholders whose views have been used to continuously develop this publication.

This section describes the effectiveness, efficiency and economy of the statistical output.

The publication uses existing administrative sources. The figures used in this publication are collected as part of the process of reimbursing dispensers for drugs supplied and monitoring of medicine use within hospitals. For hospital dispensing data the Trusts are not compelled to provide the data and do so voluntarily. HES data is from an existing administrative source.

Information about the administrative sources and their use for statistical purposes can be found at: 

This section describes the procedures and policy used to ensure sound confidentiality, security and transparent practices.

The data contained in this publication are Official Statistics. The code of practice for official statistics is adhered to from collecting the data to publishing. Further details can be found at:  

This publication is subject to the standard NHS Digital publication scheme established to fulfil the requirements of the Information Commissioner for government agencies. Further details can be found at:

This publication is subject to a standard NHS Digital risk assessment prior to issue. Disclosure control is implemented where this is deemed to be necessary in accordance with the protocols associated with the underlying data sources. Further details of the risk assessment are available at:

Methodology specification documents on medicine groupings and FAQs are provided in the list of resources for this publication. Also provided is a Guide to underlying data which describes the csv files provided as open data.