New and improved data product within the Data Platform Service

The Emergency Care Data Set (ECDS) is the national data set for urgent and emergency care. It replaced the Accident and Emergency Commissioning Data Set (CDS type 010) through a period of phased implementation commencing in October 2017. Whilst collected daily, fixed snapshots of the ECDS data (CDS type 011) are also produced for the monthly and annual ECDS data product. This has been made available through the legacy NHS Digital SQL Server RDS platform since this point.

While this has served the requirements of providing timely, patient level emergency care data to our users, the opportunity to clean and curate the product to the extent which occurred with the historical accident and emergency data product had not been possible, nor its alignment with other existing HES data products..  

Work has now been undertaken to migrate legacy processing activities across to the DPS platform providing opportunity to curate and assure the ECDS data in a similar way to HES through the development of a new supported ECDS Data Product within the Data Platform Service. The first release (tranche 1) is now available within the Data Platform Service. Improvements and benefits of using this curated data product and the plan for releasing pending derivations are given below.

This new ECDS Data Product is now accessible in the Data Access Environment (DAE) within the Data Platform Service (DPS) to meet the organisation wide migration plan to consolidate data assets onto one common data platform. 

Additional curation of submitted fields has been applied to ECDS in DPS, making fields easier to query.

The consistency and transparency of all cleaned and derived fields of ECDS in DPS aligns with our HES data products which is not the case for ECDS RDS.

We have also taken the opportunity to add around 40 additional derived fields to support analyses in ECDS DPS, particularly geographical fields such as LOWER_SUPER_OUTPUT_AREA_2021 and Index of Multiple Deprivation (IMD) that are not available in ECDS RDS.

The patient linkage fields Person_ID and Token_person_ID are now available for all years in ECDS, from 2017/18 onwards.

ECDS DPS is now the supported data product for new user requirements and updates to the Commissioning Data Set in the future.

ECDS data will still be available and updated in ECDS RDS as it is now for all the daily, monthly and annual cuts to enable a managed transition of existing dependent legacy downstream business processing where applicable. During this time, it will continue to be supported for issues relating to processing and availability.

Any new requirements and developments will only be considered and implemented for the new ECDS Data Product in DPS. The curation and improvements, summarised below, will not be applied to ECDS RDS. Users will need to migrate to the DPS platform to utilise these changes or raise new requirements.

We are performing a discovery piece to collate requirements and blockers for users to move onto the new DPS platform. Although there is no current specified date at which we shall decommission ECDS RDS, it is very much the intention this will be switched off.

The release of the new ECDS Data Product in DPS is split into tranches: 

Tranche 1 includes all submitted fields plus a handful of additional key derived (or cleaned) fields. Although all fields are visible in the data product, many of the derived or cleaned fields will show as Null in tranche 1, due to outstanding testing and assurance.

Tranche 2 will release the pending derived and cleaned fields as they are tested and assured, following a priority order.

Tranche 3 will release the remaining, lower priority derivations.

Tranche 1 of the ECDS Data Product in DPS is now available to users within the Data Platform Service (DPS) via the usual internal and external mechanisms. DARS customers can refer to the DARS onboarding roadmap.

The new, updated ECDS Analytical Technical Output Specification (ATOS) for DPS contains details of all the fields, cleaning rules, derivations and any known issues. Users can also see which fields are currently available (tranche 1) and which are pending (tranches 2 and 3) identified by greyed out rows. Legacy ECDS RDS and Flat ECDS are also incorporated into this ATOS.

Again, although there is not currently a set date for the decommissioning of ECDS RDS, we strongly encourage users to undertake any new developments and start planning migration of existing activities onto the new ECDS DPS from this point.

We wanted to retain consistency with finalised published data for years up to and including 2023/24. As such, new derivation and cleaning rules to existing fields have only been applied to data from April 2024 (2024/25) in ECDS DPS. Any new derived fields, however, have been made available all the way back to 2017/18.

The list of improvements below has only been applied in DPS and hence reflects any differences between DPS and RDS.

Tranche 1 release

Roughly 26 name changes have been applied to fields to correct a typo or to align with updated NHS organisational changes, for all years. Details are given in the ECDS ATOS.

Table/view names have been simplified from legacy RDS SQL for easier use and linkage.

Person_ID and Token_Person_ID patient identifier fields are now available for all years.

Cleaning has been applied to the PROVIDER_CODE, ensuring all values are valid and open to help with analytical purposes (from 2024/25 onwards). The methodology for ECDS is the same as for HES apart from one difference which is: where an invalid provider code was submitted with no valid successor or valid replacement code, HES deletes these records, whereas in ECDS they are instead set to 9X9X9. The HES methodology is available.

Due to limitations in DPS not having a ‘time’ field format, all ‘time’ fields (such as ARRIVAL_TIME, SEEN_TIME) are in a ‘timestamp’ format with a valid time along with a default date of 01/01/1970. Users will need to take this into account within analysis and calculations. Unfortunately, it is not possible to remove the date part of these fields in DPS. We understand that this is not ideal and seek to rectify as software updates are available. ‘Date’ fields do not have this issue.

All the above and any other known issues are documented in the ECDS Analytical Technical Output Specification (ATOS). Fields not currently available are identified by greyed out rows.

ECDS to HES APC Linkage

The methodology for linking patients who present at ECDS and are subsequently admitted to Admitted Patient Care (APC) has been updated and applied to the EPIKEY (from 2024/25 onwards). The update is to derive directly from ECDS instead of mapping from legacy HES A&E. Any differences, compared with the previous methodology are negligible, and this update will also apply to HES APC AEKEY field from 2024/25 finalised annual data.

In Tranche 1, EPIKEY is only available for years 2017/18 to 2020/21. The workaround for the later years is as follows until the remaining years become available in Tranche 2:

Part 1a – Identify the first admission episode 

Identify all records within APC data where FAE, Finished Admission Episode = 1 (referred to as APC subset from this point). For the FAE derivation rule, refer to the HES Technical Output Specification.

Part 1b – Match on Person ID

Identify instances of records where Master Patient Service Identifier, Person_ID (or Token_person_ID) is present in ECDS and is also present in the APC subset mentioned above.

Part 1c – Match on activity date

From Part 1b, identify records where the DEPARTURE_DATE in ECDS matches ADMIDATE (Date of admission) in APC.

An additional consideration is where the ADMIDATE is in the next financial year. For example, if ECDS Departure Date is 31/03/2023 and the APC ADMIDATE is 31/03/2023 and EPIEND (Episode end date) is 02/04/2023 then the APC record will be in the next financial year as this goes by the EPIEND.

Users should note that this workaround will, on occasion, return multiple or no matches which the derived EPIKEY/AEKEY fields account for. The linkage methodology for deriving EPIKEY and AEKEY will be made available in the coming weeks.

Tranche 2 release

Roughly 40 new derived fields have been created for all years.

Roughly 32 derivation and cleaning rules have been revised to existing fields to provide consistency, transparency, and timeliness using latest available reference data – from 2024/25 onwards. See Derived Fields section on the ECDS Concepts page for how these are handled differently compared to earlier years.

Details of specific fields and derivation rules are given in the Emergency Care Data Set Analytical Technical Output Specification.

Tranche 3 release

An automated DUPLICATE_FLAG will be available to highlight duplicate records in the data.

Automated mapping from Max to Core ECDS diagnosis codes has been developed within data processing. This gives users the option to report on core diagnoses only (mapped back from the max), or both core and max diagnoses (as submitted) to suit individual analytical requirements. Read the ECDS diagnosis max and core codes for analytical purposes for further guidance.

Later releases from 2026

Whilst the implementation and submission by providers using the ECDS Version 4 schema is ongoing, version 4 fields are not currently available within the ECDS data product. Where this is needed for internal business processes, internal users should raise this via internal data access routes for assessment.

Availability of Same Day Emergency Care (SDEC) for internal users. Due to low coverage this is not yet available for DARS customers until further notice.

Whilst the content of the ECDS RDS data will still be updated, in time it will gradually fall behind with further developments, and new fields that are required for user needs will not be made available in ECDS RDS.

From 2024/25, analytical reports and publications from ECDS RDS could produce slightly different results from the new ECDS in DPS where certain derivations are utilised, risking inconsistencies with counts and user interpretation. However, to ensure alignment during this transition phase in terms of record counts, we will not be removing invalid or duplicate records. We will be deferred until the decommissioning of RDS is complete.