Data set support and who to contact

Slide 1

Having seen how to use the environment and how to access data, you may still be wondering how to get started with your work in the environment.

As part of the SDE service, there is information such as example code, data dictionaries and online guidance that is available.

Slide 2

Firstly, data dictionaries. If you have worked with large data sets before, particularly healthcare data, then you will know that the terminology and codes such as ICD10 can make analysing data tricky, so we have created data dictionaries for the data sets we provide.

Slide 3

For a given data set, the corresponding data dictionary will list the column names along with a description of what that column is, the data type you can expect to find in that column, and the meaning of codes and abbreviations.

To access the data dictionaries, you should first get in touch with someone in your organisation, as they may already have these stored in a shared area for you to use.

If you can't find the information you're after or if you have suggestions for improvements, contact the SDE service team.

Slide 4

Slide 5

Next up is Summary Notebooks. These are available for each data set and give a high-level overview of the data, such as the number of records over time and how many rows have null values project, to help you determine which columns may be useful for your work and how data sets may be best joined together.

Slide 6

To find them, you should go to the workspace area within your Databricks and then to the shared area where you should find a folder called Summary Notebooks.

Inside this folder, you should find a summary for each dataset that has been made available to your organisation. You will be able to run each notebook to get the most up-to-date summary of the dataset. The Deaths Summary Notebook shown in this video is an example.

Slide 7

Slide 8

We also provide Example Analysis Notebooks. These can be found in your organisation's GitLab in a repo called SDE Example Analysis.

Slide 9

The best way to view them is to clone the repo to your local area within Databricks. We have some user guidance on how to clone a repo into Databricks in the further support section under this video.

There will be an ideal opportunity for you to practice cloning a repo once you have access to the environment.

The aim of these notebooks is to give a range of code examples to help you make full use of the powerful tools and packages available within the SDE. The notebooks range from giving guidance on how to use an absolute versus a relative path to more complex things like machine learning packages.

Slide 10

We hope that the help and guidance mentioned so far will give you everything you need to work within the SDE. However, we are aware that sometimes things go wrong or you may need additional help. If this is the case, then we have the following support available.

If you encounter technical problems accessing the SDE platform, such as your password or multifactor authentication not working, then you should contact the National Service Desk.

For any output or input related questions, it is best to contact the output or input checkers directly. Contact details for these services are listed on this webpage.

Our detailed online guidance will help resolve any common problems and questions you may have. But if you still don't have the information you need, then get in touch with the data wranglers by email or to speak directly to one of us.

Attend one of our drop-in sessions, which occur three times per week. The invite to these will be sent to you upon completing the induction process.
That brings us to the end of the video and the end of the induction series.