Overview

This section briefly reviews issues which CASSRs should consider from the outset of the project. It covers the importance of maximising response rates and lists the materials to be used by the council in undertaking the survey. It also covers the issues of engaging staff and public, maintaining confidentiality and relationships with third party providers.

As you read through this guidance it should become clear that the survey design is largely aimed at maximising the response rates. This is because surveys do not draw information from the whole population they represent. Instead, the views of those who participate in the survey are used to estimate the views and experiences of everyone.

It is natural that any estimate is subject to a degree of uncertainty, which is expressed statistically as a ‘margin of error’. The higher that the proportion of survey respondents is in relation to the total user population, the lower the margin of error. The requirement for the ASCS is that the results have a margin of error of less than +/- 5 per cent. This means that if 50% of survey respondents say they are very satisfied with the services they receive, we can be reasonably confident that the figure for all eligible users from that council will be between 45% and 55%. More information about margin of error calculations are available in the Calculating the sample size section. 

It is also important to maximise response to mitigate the problem of non-response bias, which occurs if there are significant differences between people who do and do not participate in a survey. If for example, those who are more likely to respond to a satisfaction survey are also more likely to be dissatisfied with the services they receive, then the results will overestimate the general level of dissatisfaction amongst users. The higher the level of non-response, the greater this overestimation shall be. Simply increasing the sample size, without changing the response rate, cannot mitigate this problem.

The need to maximise response rates underpins the design of the materials. The questionnaire and covering letters are written to help users understand that participation is a worthwhile activity. These documents guarantee confidentiality and highlight the steps to take if users need help to complete the survey. The layout of the materials is aimed at helping respondents easily negotiate the questionnaires, for example by using Calibri font in 14-point to make the documents easy to read (see Appendix D for more details).

Consideration has also been given to enabling as wide a range as possible of potential participants to respond, including those whose who are not fluent in English, people with sensory and other impairments, people with limited literacy skills and anyone who might find it difficult to complete a questionnaire by themselves. In addition to the self- completion questionnaires NHS England has produced a range of materials (see Available materials below). Councils are also expected to give due consideration to the diversity of needs of the users in their area in terms of provision of materials and assistance.

In past years, some CASSRs have chosen to use incentives to encourage responses to the survey. NHS England carried out some analysis on question responses for councils that used incentives compared to those that did not offer incentives. The analysis discovered that the use of incentives did not seem to have an impact on the overall response rate but there were differences in the question responses. To try and ensure the data is as comparable as possible, CASSRs are advised not to use incentives.

In addition to this guidance, NHS England has posted the following materials for CASSRs’ use on the Adult Social Care Survey, 2024-25 guidance for local authorities:

• Questionnaires
• Face-to-face and telephone interview script
• Demographic questionnaires
• Cover and reminder letters
• Letters to care home managers
• Consent form
• Translation request sheet
• Sample size calculator
• Draft and non-protected survey data return

The questionnaires have different versions for service users who are resident in the community and for those in residential and nursing care.

Most of the materials for service users include large print and easy read versions. Translated versions of the questionnaires and letters are available in 17 languages. For a full list see Appendix E.

If the first language of the service user is known, then the translation request sheet does not need to be sent out with the questionnaire(s).

Councils may also wish to use the MAX toolkit, which is designed to maximise the local usefulness of the survey (see Appendix F).

Further information sources are listed in Appendix G.

Given their essential role in conducting the survey, council staff need to be committed to its aims and objectives. An understanding on their part of the aims of the survey is likely to be of key importance in gaining their commitment. Furthermore, staff who are not directly involved with the survey but work with service users, should be made aware of the survey and its aims. This way, if asked by service users, they may encourage response and allay concerns that response may affect services received. Another way of getting all staff to ‘own’ the survey would be to ensure that they have access to the analysis of the results of the surveys and have some involvement in making use of them.

Senior management must also be informed about the survey. It is an ethical requirement that participating CASSRs obtain senior management permission prior to sending out questionnaires to service users. Generally, it is in councils’ interests that senior management are informed of the need to undertake the survey and engaged as early as possible. This way due consideration can be given to allocating resources to the survey administration (see Appendix H) and to maximising the usefulness of the data obtained to the council.

CASSRs may have to carry out a local ethical review if they do not run the survey in the way outlined in this guidance document, or if they choose to include additional questions in the questionnaires (see the Additional questions section for information about additional questions).

Councils should also consider engaging the public, especially service users, as early as possible. Giving the survey advance publicity should aid the response rate. This leads potential respondents to expect to receive a questionnaire and gives them some idea of what the survey is trying to achieve. Councils might issue a general press release through the local press or through relevant local user group networks. It may also be useful to let local service user organisations know about the survey, so they can publicise it with users. A leaflet with a notice to service users may also encourage people to respond. 

It is essential that the confidentiality of participating service users is assured. Council staff, other than those administering the survey, cannot know whether individuals have completed a questionnaire or how questions were answered. The questionnaires, covering and reminder letters all contain assurances of confidentiality.

A unique code will need to be generated by councils so that returned questionnaires can be mapped back to the participant. This will enable survey teams to follow up those who have not initially responded to the survey, and to bring together information from a variety of data sources. CASSRs will therefore need to consider how information is handled within the council. Due attention must be paid to the General Data Protection Regulation (GDPR) which has been in operation since May 2018.

In particular, access to the list of users’ details used to send out the questionnaire will need to be strictly controlled. Computer files need to be password protected and/or stored on a secure server, and only those involved in mailing out the questionnaire and responsible for either pre-coding the questionnaires or setting up codes for the analysis must have access. Unsecured email must never be used to send content which includes the names or other personal details of users. Completed questionnaires must be stored securely in locked cabinets.

Councils must retain paper copies of the questionnaires until the publication of the final 2024-25 ASCS report. After this point you are free to securely dispose of paper copies, although you may wish to hold on to them for longer if you believe for example, that participants are likely to take part in any follow up studies. The treatment of paper and scanned copies must be in line with your local Information Governance procedures and meet the requirements of the GDPR. 

Councils may consider outsourcing elements of the survey process to a third party. For example, a data capture company might be used to scan completed de-identified questionnaires into a dataset format. This approach may help councils to meet submission deadlines by removing the need to enter data manually.

In determining whether to outsource elements of the survey to a third party, councils should refer to their local Information Governance arrangements in the first instance, as they will need to ensure that all relevant information governance issues are considered throughout the process of exploring, making and implementing an outsourcing decision. Again, councils will need to attend to the new GDPR.

In running the survey, councils are likely to be acting as a data controller, outsourcing tasks, under clear instruction and with strict controls, to third parties who then act as data processors in carrying out the agreed work. Councils therefore are responsible for ensuring that any third party operates by an acceptable code of conduct, particularly around the issue of confidentiality.

If any of the user services are outsourced, councils should ensure that the relevant systems and agreements for data sharing and protection are established with the relevant parties. Some councils have experienced problems persuading partner organisations, responsible for the provision of services, to share contact details in order to enable councils to send out questionnaires. It is recommended that data sharing requirements are written into contracts when setting up such arrangements to avoid this situation.

Where required, councils should also satisfy themselves that service users are made aware that data sharing with third parties may be required to ensure both the smooth running of the assessment and delivery process, and for evaluating the performance of providers and councils via mechanisms such as surveys. As with outsourced survey partners, this issue must be considered in the context of the GDPR.