Part of Using Databricks in the NHS England Secure Data Environment
Creating reusable processes
Widgets
Widgets allow you to control parameters in your notebooks and dashboards.
If you have an analytical pipeline that produces a regular set of outputs, you can create a widget that allows you to set different parameters, such as year, gender or organisation, before re-executing the notebook. This means you can create a reusable process which can be used multiple times in the future without changing the cells in the notebook.
For more information on using widgets, refer to the Databricks website.
The following code can be used to remove all widgets:
dbutils.widgets.removeAll()
Widgets must be initialised before they can be used. If parameters are ‘passed in’ then this is skipped. When they are created, they take in a parameter value in preference to the value specified or the default value. The following code is an example of how to initialise widgets:
You can pull the values from the widgets into Python and use them, such as in a PySpark query as shown in the example below
You can also pass the parameters in SQL to pull in a widget variable using %<param_name>, as shown in the example below:
You can also pass variables in from Python, as shown in the example below:
Alternatively, you can use format-strings (f-strings). In Python source code, an f-string is a literal string, prefixed with f, which contains expressions inside braces. The expressions are replaced with their values, so f-strings provide a way to embed expressions inside string literals, using minimal syntax. It should be noted that an f-string is an expression evaluated at run time, not a constant value.
See the example below:
Last edited: 11 January 2024 1:57 pm