Adult Social Care Survey 2021-22 Methodology Change Notice

The Personal Social Services Adult Social Care Survey (ASCS) is an annual survey for England that took place for the eleventh time in 2020-21. The survey covers all service users aged 18 and over in receipt (at the point that data are extracted) of long-term support services funded or managed by social services following a full assessment of need.

The survey is designed to help the adult social care sector understand more about how services are affecting lives. User experience information is critical for understanding the impact of services, for enabling choice and for informing service development.

The survey asks service users questions about quality of life and what impact care and support services have on their quality of life. It also collects information about self-reported general health and well-being and these themes are covered in the six sections of the questionnaire:

• overall satisfaction with care and support
• quality of life
• knowledge and information
• health
• layout of home and surrounding area
• help from others

The data collected are sent to NHS Digital by Councils with Adult Social Service Responsibilities (CASSRs) and are used as the basis of analyses that are included in national level Official Statistics reports. Data are also used to populate several measures in the Adult Social Care Outcomes Framework (ASCOF). These data are used to inform and support policy and decision making at both the local and national level; to improve care, services and outcomes for local populations.

The resulting Official Statistics report focuses on some of the key findings from the mandatory questions from the survey, a summary on information about the service users and analysis on questions which have notable changes to the previous survey.

Further information about the survey, including the methodology, is available. 

Significance testing was introduced to the ASCS in 2017-18 to evaluate the key findings. Significance tests are carried out in order to test the relationship between variables. The test is for the main effects only, using a Wald test. The Wald test is a statistical test used to calculate the significance of parameters in a statistical model. In addition to the Wald value, the p-value is also used. A p-value is the probability of the observed result occurring due to chance alone. A p-value of less than 0.05 (p<0.05) indicates statistical significance between two variables. i.e. the relationship between these variables rejects the null hypothesis. In previous publications, backward elimination logistic regression was used to calculate the Wald value and the p-value.

As part of the continuing development of the survey, NHS Digital have reviewed the statistical testing and made a change for the 2021-22 publication.

NHS Digital explored numerous statistical techniques to decide which method worked best with the survey data. To ensure as much data as possible is included when carrying out the statistical tests, nulls will be removed for just the 2 variables in the test, and not from the whole dataset. It was concluded that Spearman correlation analysis was the most suitable technique to use. The new methodology used to identify statistically significant findings will be to use Spearman correlation analysis to find the correlation coefficient and chi-squared tests for the p-values. For consistency the statistical tests will be carried out using weights so that it is in line with the methodology used elsewhere in the survey.

The acceptance criteria for identifying variables that are statistically significant have been set as a p-value less than 0.05 and a correlation coefficient of 0.4 or higher. A correlation coefficient greater than 0.4 indicates a moderate to strong correlation.

As part of the review, testing was carried out to see if the statistical findings identified in the new methodology matched those previously identified using the logistic regression method. For the questions included in the review, all variables that were identified in the logistic regression process as statistically significant, also met the new acceptance criteria. Due to this we consider the impact of the change to be very low.

NHS Digital will be using the new methodology to identify key findings in the 2021-22 ASCS publication.

Questions and feedback on the publication are welcomed and should be sent to [email protected] or alternatively call 0300 303 5678.