Overall lessons learned and recommendations

The trialist organisations that NHS England worked with have shared their lessons learned which we have summarised to assist NHS organisations who are considering undertaking a trial of their own.

Being able to identify when a trial is no longer tenable or unlikely to achieve its desired business outcomes and benefits in its current form can be an important part of the deployment process.

Keep in mind that concluding that a technology or process is not suitable can still constitute a successful trial if the lessons learned assist with future planning and moving closer to a solution that meets the requirements, either within the current trial or as part of a future deployment.

In their trial of mobile connectivity solutions Liverpool Women’s Hospital initially planned to trial use of a Wi-Fi product that would allow improved access to connectivity when delivering care in patients homes. However following challenges with the integration of the solution with existing technologies they decided to remove this technology from their trial and instead focus wholly on a bonded SIM solution to meet the same technical requirements and business outcomes.

Thinking of users first will help avoid implementing a solution that is difficult to use or poorly explained. If this happens, staff could avoid using the new technology and revert to the pre-trial ways of working. At best, this could mean costly reworks with the supplier and at worst, a failed implementation.

This can be mitigated by consulting with the teams that will be using the technology and asking them what their challenges and needs are to identify improvements to the solution’s usability.

In their Real Time Location Services trial, Princess Alexandra NHS Foundation Trust realised that their first solution of having locations of tagged medical devices displayed in a text only list view did not meet user needs. This was because there was no standardised room naming convention at the trust, which meant that the trust had to rework their solution to include a visual map view to locate the devices. For more information on their trial see Implementing RTLS for asset tracking and capacity management. 

The NHS is a challenging environment to introduce new technologies and ways of working. Many of the connectivity trials ran with NHS England encountered delays, however with flexible planning and support the majority were still able to deliver useful outcomes, even if months after their originally planned dates. These were for a variety of reasons, including external challenges such as global hardware shortages, unforeseen challenges with incumbent devices and infrastructure or local challenges like staff availability.

During the trial, make every effort to ensure that planned timelines are accurate estimates, from both internal staff and suppliers, and ensure there is sufficient contingency to account for unforeseen delays.  In their  trial of satellite connectivity Manchester University NHS Foundation Trust encountered significant delays during testing and faced challenges with integration of the new connectivity with existing firewall infrastructure. This meant they were not able to fully test all of the use cases identified, but by building contingency into their planning and the technical resource required when needed they were able to support a live deployment of the technology successfully and address many of the technical questions identified in their use cases.

Special care should be taken to ensure that identifying, capturing, clarifying, and documenting the desired business outcomes, for example a reduction time spent completing a task due to use of a new technology, and tying these to the needs and expectations of stakeholders is done in a thorough and realistic way.

As a part of this process you should engage with every team involved in and effected by the trial and consider their wants and needs. In this way, the solution can be created collaboratively, with as many user requirements as possible accommodated for. This will also help to create a sense of shared ownership of the trial, allowing staff to feel part of the process. As part of their trial of Real time Location Systems Mid Cheshire Hospitals NHS Trust spent additional time and effort at the planning stage to tie the trial of a technical solution to the clearly defined user challenge of finding critical medical equipment and the business outcome of increased efficiency by quicker location of these items by staff. This outcome could then be measured and tested throughout to provide a clear measure of how the trial was meeting business objectives and clearly communicate benefits to stakeholders.

It is often worth looking at the services that a supplier would provide and assessing if it is possible for internal staff to perform them. For example, Sussex Community NHS Foundation Trust determined that they could bring Wi-Fi surveying in house, a function that was previously provided at significant cost by a third-party supplier.

Doing this allowed Sussex to both save money and empower their staff with new skillsets. Read the summary of their trial. 

If using a supplier in the trial, ensure staff take advantage of the opportunity to learn from them, upskilling internal staff to learn how to manage the solution by assigning them to work with the supplier over the course of the trial. This should be discussed early when choosing a supplier.

Every organisation is unique, and running a trial is a fantastic way to assess readiness for larger changes. The trial process will engage many different teams as part of a small-scale deployment, and many of the learnings will be relevant in the future.

To take best advantage of this aim to keep a log of decisions made and their outcomes throughout the trial and make this information available to everyone in the organisation. It is strongly recommended that a trial incorporates project management resource and skills wherever possible to support management of the documentation, tracking of progress and coordinating the different clinical, technical and other teams involved.

A trial of Li-Fi technology by North West Ambulance Service initially faced challenges in implanting and testing the technology due to a lack of project resource, but was able to later introduce a project manager to support and reassess the scope of the trial and deliver a smaller scale deployment which helped inform them come to an informed decision on the trialled technology and conclude the trial.

During the trial process, consider contacting nearby NHS trusts to discuss the trial. Collaborating on ideas with other organisations means that new ideas, lessons learned, and best practise can be shared from both sides.

For example, in their trial, the East of England Ambulance Service NHS Trust reached out to other trusts and ambulance stations in their region before installing antennas in their ambulance bays for smoother patient handover. By doing this, they were able to assess the local need for their solution and receive advice that fed into their requirements definition. Working with NHS England or other regional or national bodies can also show that the trial is of wider interest and application and can help with engagement not just with the wider NHS but with suppliers and public sector partners.

Read the improving ambulance and A&E department connectivity trial.