Trial structure walkthrough

Trialists working with NHS England followed a 5 stage trial structure, with milestone documents submitted at the end of each stage. This ensured they were able to conduct their deployment and provide a record of their approach, successes and lessons learned.

If you would like more information on any of the stages or documents referenced below, please contact us at [email protected] and we will be happy to support you and share the relevant documents.

At the start of the trial, the focus should be on defining the scope to form requirements and initial plans. By drawing up clear timelines, cost estimates, risk and issue logs, dependencies and identified benefits (discussed below), a clear picture of the trial’s goals and methodology can be ascertained.  Once the reasoning for a trial has been established, the scope and overall objectives should be defined to provide direction for the trial process. 

As a part of this process, organisations should identify and speak to every team that will be impacted by the trial and allow their insight to inform these initial plans. This will usually include technical, estates and clinical teams, who should understand how the trial will affect them and by how they can provide input and feedback. This should also include a communications plan to ensure effective updates to stakeholders that need to be informed of the trial’s progress.

At this stage, a trial should evaluate solutions and products against requirements and, if the size or nature of the trial requires it, run a competitive procurement to find a technical supplier that is knowledgeable about the technology and capable of delivering the solution.  

When doing this, engagement with local commercial teams is important to ensure awareness of and compliance with relevant local and national financial, procurement, and budgeting guidelines and policies.

Initiation should ultimately assess the feasibility of the trial  with currently available information what is the confidence level that the trial will deliver on its targeted benefits?

Stage 1 should conclude with a readiness check to confirm the trial is clear to proceed. At this stage trials should have:

Has every party that will be affected by this trial been identified and consulted regarding the scope, methodology and timescales?

Have dependencies and resource requirements for the trial been identified?

Are there any practical considerations, such as physical constraints or regulations on infrastructure changes to building, for example cabling or equipment installation on ceilings? Is any work required in clinical areas, and what sign offs, and notice periods are required for this?

Is there sufficient contingency available in time and budget for the trial to continue if there are unexpected delays or spendings? What are the main risks of the trial at this stage?

What is the definition of success for the trial? Ensure that identified benefits are clear and measurable.

What are the next steps if the trial is successful in proving the proof of concept?

Have staff who will be using the new technology been engaged, and do they agree that it will be an improvement over pre-trial ways of working?

Risk assumptions issues and dependencies (RAID) log To be maintained and updated throughout the trial.

Communications plan Including engagement plan and identified stakeholders.

Agreed business case or justification for the project.

Initial project plan This should include expected start/end dates for all key activities, outlining the critical path for the deployment.

High level documentation of the solution.

This should include the building blocks of the technical solution, indicating equipment, connections, and key dependencies and initial security considerations.

Record of technical and health metrics to be tracked and reported by the project metrics should relate to benefits the project is expected to provide and outline who benefits, should include baseline data in some form, and outline frequency and method of reporting.

Once requirements have been properly defined and delivery planned, work closely with any suppliers or third parties to complete procurement of the solution and prepare for deployment.

The first output to be created during this stage are well defined implementation and test plans, covering both technical and user-based testing. This should list the timelines, procedure and success criteria for the testing process.

Once created, the solution should be tested for functionality and user acceptance - at this stage, any concerns around the practical viability of the solution should be resolved before continuing to deployment.

At this point begin producing an implementation plan, this will outline the scope, steps and timescale of the deployment, including people training requirements and business/change management processes.

What clinical, security (cyber and physical if appropriate), or information governance considerations apply to the trial (especially if Patient Identifiable Data is involved)?

Does the trial need user training? Would it help onboarding to create a guide to assist with using the new technology?

Does the trial require user acceptance testing? If so, by whom?

Ensure there is enough time to action any changes that are required from the results of the testing.

Post testing, ask again if staff who will be using the new technology agree that it will be an improvement over pre-trial ways of working? If not, why?

Test plan (Technical and user acceptance testing)

This should cover what test phases were included (for example system, integration, PAT, UAT), are there specific environments used, and the scale and scope of each phase.

Outline implementation plan

This should include the scope and scale of the deployment, people training requirements, business/change management processes and environmental considerations.

Detailed technical documentation

A lower-level design description and technical architecture design of the proposed solution. This should build on the high level design produced previously and include details of how the service will be managed and maintained.

Clinical considerations of the deployment

Documentation covering considerations made for deployment from a clinical perspective. This may include user experience considerations for clinical and/or patient users, estate or building considerations.

During this stage, the trial’s solution is to be deployed as per the implementation plan agreed in stages 1 and 2.

This is the time of most intensive change for any team effected by the trial, with collaboration and communication between the project team, supplier and effected staff an important part of a successful deployment.

What is the contingency plan if the deployment is delayed, or fails? 

When is the least disruptive time to the organisation, particularly to clinical areas and staff, to deploy?

Will any other technical services be impacted or unavailable during the deployment?

Consider a dual running deployment, where both the “as is” and trialled connectivity solutions are running in parallel to mitigate any outage of service risks during deployment.

Capture lessons, issues and resolutions encountered during deployment and rollout.

Milestone documents for stage 3 

Lessons learned log detail the deployment process factually and chronologically, supported with a narrative of what worked well, and lessons learned.

Full implementation plan

Deployment and go-live report Where relevant, include updates of considerations captured previously, such as clinical considerations, security implementations, or information governance methods. The report should outline any deviations from the initial plans or designs, and what necessitated that decision.

Updated benefits realisation plan This should be based on the initial benefits plan and include finalised baseline data, responsible owners, method of collection, and expected date of completion. Where possible include quantified benefits (for example time saved per member of staff, cost savings per year, expected return on investment).

After the deployment has concluded, the focus of the trial should be on supporting the initial operation of the solution and assessing the impacts (both positive and negative) of the trial.

Has the project fallen within expected timescales and budget? If not, what could have been done to avoid this?

What preparation can be done to make benefits remeasuring as easy as possible? This can include engaging with effected groups or planning how to measure certain metrics.

How will findings and outcomes be presented and shared with stakeholders?

Updated RAID Log and Project Plan.

Benefits report: This should compare baseline data with operationally experienced benefits.  This should demonstrate return on investment (if any) and detail what those returns are (cash releasing, non-cash releasing, clinical, and societal). Benefits measurement documents (for example surveys, staff questionnaires)  and any unexpected benefits, not listed in either the initial business case or benefits plan, can be included here.

The main priorities for the end of the trial are handing over the operation of the solution to business as usual teams, quantifying and measuring benefits, and considering if there are any further use cases for the trialed technology.

Was the trial successful? What were the main successes and lessons learned from the deployment?

Do the staff impacted by the trial have everything they need to continue with their normal function?

Is there any opportunity for the trial to be rolled out over a larger remit? Can a technical blueprint be produced?

The recommended milestone documents for stage 5 include:

• closure report
• complete RAID log
• complete lessons learned log
• blueprint (if applicable)

For more information on blueprints across the NHS read blueprinting. You will need a NHS Futures account.  

By following a documented, measured and collaborative approach it is possible for any organisation to run a technology trial to test new solutions and provide justification for connectivity improvements in the NHS.

If you have conducted, or are thinking of conducting, a connectivity trial in your organisation, let us know at [email protected].