Wireless Trials benefits realisation and definition

One of the main factors that underpins a trial is learning how to discuss the impact that the trial has had, which is done via benefits tracking and realisation. This process will be ongoing throughout the trial, and a starter guide is presented below.

Between the 15 successful trialists and over one hundred unsuccessful applicants that took part in NHS England’s wireless trials, a key piece of feedback received was that it is difficult to define and measure the benefits of a trial in a way that captures and expresses the potential of connectivity technology, as the technology itself may only be an enabler of other services rather than delivering direct measurable benefits.

In response to this challenge NHS England proposed use of the below methodology, based on the Infrastructure and Projects Authority (IPA) guide for effective benefits management in major projects, for benefits gathering with wireless trialists.

Trialists utilised three board categories of benefits definition:

• health benefits include any positive outcome of the trial that will improve the quality of care for the patient or improve patient outcomes
• efficiency benefits cover positives that are not directly related to patient outcomes, for example cost savings, or freeing up clinician time
• technical benefits incorporate improvements to network coverage, bandwidth and network reliability

As a part of requirements definition (see stage 1 below), hold a workshop to identify every possible benefit from the deployment.

This should be a session held with representatives of each team involved in and impacted by the trial and aim to document every improvement, no matter how minor.

This should also include any perceived disbenefits, or negative outcomes from the deployment. 

Each benefit should be measurable – if  a way to gather baseline (as is) or comparison (post deployment) figures with either qualitative or quantitative data cannot be identified , then the benefit should not be considered.

Potential measurements can be cash releasing (such as direct savings) or non-cash releasing (such as improving a clinician’s quality of life).

Assign an owner to each benefit that will be responsible for measuring the benefits over the course of the trial.

As a part of requirements gathering (stage 2 in the trial structure), each benefit owner should measure the baseline metrics for each agreed benefit. This can include, for example, technical measurements or staff surveys.

Note that some benefits will introduce a completely new capability, in which case it will not have a relevant baseline, although the outcomes of the capability should have a measurable impact within the organisation.

During deployment to the end of the trial (Stages 3-5), each benefit owner should remeasure these benefits when it is applicable to do so.

After new measurements have been taken, compare the baseline and post deployment figures to quantify the benefits of the trial. These can take a variety of metrics, from money saved over a period to an improved staff satisfaction survey.

These figures can then be used to determine if any further expansion of the trial’s technology is suitable.

More information on benefits management and realisation can be found in the Infrastructure and Projects Authority (IPA) guide for effective benefits management in major projects.