Using DLP

When you log in as a Data Sender the Data Submissions screen is displayed:

The data submissions screen displays the submit data tab and the previous submissions tab.

The Previous Submissions tab contains a table with the following columns:

• status identifies whether a submission has been successful.  Successful submissions are identified by a green tick icon. Failed submissions are identified by a red cross icon. Clicking the red cross will display information about why the submission was unsuccessful. Entries from the previous submissions list cannot be removed, but they can be filtered according to status (success or failed), recipient, date and specification
• recipient(s) identifies the recipient(s) of that submission. Where the submission was sent to more than one recipient a ‘+’ sign followed by the number of recipients is displayed. Hovering the mouse over this text will display a tooltip that lists all recipients of that submission.
• specification identifies the data specification used for the submission
• The last downloaded column displays the date the file was last downloaded by the recipient, or the date it was automatically sent to the recipient via MESH
• Clicking the icon under more info will display information about the submitted file, including submission date, row count and any supporting information included when the submission was made
• validation report available if validation rules are active against the specification and errors are found within the submission

Above the previous submissions table there is the option to Export to CSV. This will export the contents from the previous submission table into a CSV file to give an audit log of previous submissions made.

Submissions to DLP are made using the submit data tab on the data submissions screen. Data senders can drag file(s) into the interface or press the select file(s) button and browse to where their files are. 

Upon dropping files into the interface, a submission queue is formed. Users are required to select a specification for each of the files in the queue. The submission button will not become active until a specification has been selected for each file.

At this stage, users are able to delete rows from the submission queue or to select or drag more files into the queue.

Once submit has been pressed, the queue will begin to upload.  You can click pause processing if you want to make changes to the remaining items in the queue after the current file has completed processing, or to add further files. A progress bar is displayed (as below) to show the status of each submission as it uploads. The orange bar only appears if your file is subject to additional validation rules (see the next section for more information).

Once the queue has been completely processed, the status will update and submissions can be viewed in the previous submissions tracker. View errors can be clicked to show validation errors.

Data Field Checking (DFC) functionality is now active in DLP. As a result, submissions may be subject to additional validation that checks the content of individual fields within the submission such as date formats and text string length. DSCRO specification managers will control what validation rules are applied and define what behaviour is triggered where a submission fails to meet the data checking rule.

The following actions are available to specification managers when applying DFC to new and existing specifications:

• do nothing/no action – file passes through without any action
• warning – file passes through, but validation errors are highlighted
• reject – file is rejected, and validation errors are highlighted

Where warning or reject behaviours have been activated by the DSCRO and validation errors are identified in the submission, a download button will appear in the ‘submit data’ and ‘previous submissions’ screen that providers and DSCROs can use to view the validation report. This report is generated in a compressed .zip file. 

Users can check if DFC/validation rules have been applied to a specification by viewing the specification. The screenshot below shows a specification with five columns (named column1, column2, column3, column4 and column5), with each of these having validation rules applied.

The rule type defines the kind of value expected. The exception to this is where the column is for a date entry – in this case, the rule type will display the date format expected, as in the example below.

Where a max length is specified, this shows the maximum expected length of any entry under that column (including any spaces). For a complete list of the characters allowed for each rule type, please see the table in Appendix A - List of validation rules for each Rule type.

If you submit a value that does not match the expected format (as defined by the rule type and described above), a DFC report is generated.

The validation report (also known as the DFC report) can be downloaded as soon as it is generated using the Download button displayed in this screenshot. 

If the report is not downloaded at that point, it can be accessed later through the previous submissions tab.

Note that it is possible to generate a DFC report even if your submission has not been successfully submitted.

The DFC report is generated as a Comma-Separated Value (CSV) file and provided in a .zip file. It is recommended that you open the CSV file using a text editor such as Notepad, however most systems will use Excel by default. The images below show the same report open in both Excel and Notepad.

While the Excel version displays the information in a more uniform manner, in some instances (e.g. date formats) a Submitted Value will be displayed using Excel’s default format rather than as the actual value submitted. For example, if the value submitted for a date is 1972-05-06, Excel may display this as 06/05/1972 (but it will display as 1972-05-06 in a text editor).

To minimise any impact on performance, DFC reports have been limited to the first 1million failures.

Regardless of how you view the report, the first two rows tell you:

• the unique submission ID
• the specification name
• the sender ODS code
• the recipient ODS code

The rows after detail what issues have occurred and where they can be found. The example below details how the report works.

• column A ‘row Number’ and column B ‘column name’, identify the location of the submitted value that has failed the validation rule. In this case, it’s in row 2 of the column named column1.
• column C ‘data type’ and column D ‘size expected’ show the validation rule that has been applied. In this case, entries must conform to the AlphaNumericSpecials format and be no longer than 10 characters.
• column E ‘allow blanks’, specifies whether a blank value is allowed. Where it says “yes”, not submitting a value means that the validation rule will not be triggered.
• column F ‘submitted value’, shows the actual value submitted. In this example you can see that the submitted value – value1234567890 – is longer than 10 characters, which is why the entry has failed validation.  If you use NHS Identity to access DLP, this column is not included in your DFC report.
• column G ‘DQ result’, shows the action that will be taken if an entry fails validation. In this example, the whole file would be rejected. Alternative options in this field are “warning” (where the submission is allowed to pass through, but a warning is given) and “No action” (where the submission passes through without a warning or being rejected, but a DFC report is still created).
• If a file is rejected, it must be re-submitted once any issues are fixed otherwise it will not be received by the recipient.

To rectify the issue in the example above, the submitted value in row 2 of column 1 would have to be amended so that it was 10 characters or fewer.

Each individual failure will generate an entry in the DFC report – so if a row has three validation errors, it will generate three rows similar to the one in the example, explaining what each error is.

Once all errors have been corrected, the file can be resubmitted to confirm that it now conforms with the validation rules defined for the specification.

Click the Logout button to exit the portal.

If the Logout button is not selected before exiting the portal, the User may be displayed with the following screen when they next try to login.

If this screen is displayed, the page must be refreshed and the user will then be displayed with the normal drop down list of roles to login with.