Overview

This section briefly reviews issues which CASSRs should consider from the outset of the project. It covers the importance of maximising response rates and lists the materials to be used by the council in undertaking the survey. It also covers the issues of engaging staff and public, maintaining confidentiality and relationships with third party providers.

As you read through this guidance it should become clear that the survey design is largely aimed at maximising the response rates. This is because surveys do not draw information from the whole population they represent. Instead, the views of those who participate in the survey are used to estimate the views and experiences of everyone.

It is natural that any estimate is subject to a degree of uncertainty, which is expressed statistically as a ‘margin of error’. The higher that the proportion of survey respondents is in relation to the total carer population, the lower the margin of error. The requirement for the SACE survey is that the results have a margin of error of less than +/- 5 per cent. This means that if 50% of survey respondents say they are very satisfied with the services they receive, we can be reasonably confident that the figure for all eligible carers from that council will be between 45% and 55%. More information about margin of error calculations are available in Appendix D. 

It is also important to maximise response to mitigate the problem of non-response bias, which occurs if there are significant differences between people who do and do not participate in a survey. If for example, those who are more likely to respond to a satisfaction survey are also more likely to be dissatisfied with the services they receive, then the results will overestimate the general level of dissatisfaction amongst carers. The higher the level of non-response, the greater this overestimation shall be. Simply increasing the sample size, without changing the response rate, cannot mitigate this problem.

The need to maximise response rates underpins the design of the materials. The questionnaire and covering letters are written to help carers understand that participation is a worthwhile activity. These documents guarantee confidentiality and highlight the steps to take if carers need help to complete the survey. The layout of the materials is aimed at helping respondents easily negotiate the questionnaires, for example by using Calibri font in 14-point to make the documents easy to read (see Appendix E for more details).

Consideration has also been given to enabling as wide a range as possible of potential participants to respond, including those whose who are not fluent in English, people with sensory and other impairments, people with limited literacy skills and anyone who might find it difficult to complete a questionnaire by themselves. In addition to the self-completion questionnaire NHS England has produced a range of materials (see available materials below). Councils are also expected to give due consideration to the diversity of needs of the carers in their area in terms of provision of materials and assistance.

In past years, some CASSRs have chosen to use incentives to encourage responses to the survey. NHS England carried out some analysis on question responses for councils that used incentives compared to those that did not offer incentives. The analysis discovered that the use of incentives did not seem to have an impact on the overall response rate but there were differences in the question responses. To try and ensure the data is as comparable as possible, CASSRs are advised not to use incentives.

In addition to this guidance NHS England has posted the following materials for CASSRs’ use on the Survey of Adult Carers in England 2023 - 2024:

• standard questionnaire
• large print questionnaire
• face-to-face interview script
• telephone interview script
• covering letter
• reminder letter
• consent form
• translation request sheet
• sample size calculator
• survey data return document
• translated versions of the questionnaire and letters in 17 languages. A full list is available in Appendix F

Councils may also wish to use the MAX toolkit, which is designed to maximise the local usefulness of the survey. See Appendix G.  

Further information sources are listed in Appendix H.  

Given their essential role in conducting the survey, council staff need to be committed to its aims and objectives. An understanding on their part of the aims of the survey is likely to be of key importance in gaining their commitment. Furthermore, staff who are not directly involved with the survey but work with carers, should be made aware of the survey and its aims. This way, if asked by carers, they may encourage response and allay concerns that response may affect services received. Another way of getting all staff to ‘own’ the survey would be to ensure that they have access to the analysis of the results of the surveys and have some involvement in making use of them.

Senior management must also be informed about the survey. It is an ethical requirement that participating CASSRs obtain senior management permission prior to sending out questionnaires to carers. Generally, it is in councils’ interests that senior management are informed of the need to undertake the survey and engaged as early as possible. This way due consideration can be given to allocating resources to the survey administration (see Appendix I) and to make best use of the data obtained.

CASSRs may have to carry out a local ethical review if they do not run the survey in the way outlined in this guidance document, or if they choose to include additional questions in the questionnaires (see paragraphs below for information about (see the Additional questions section). 

Councils should also consider engaging the public, especially carers, as early as possible. Giving the survey advance publicity should aid the response rate. This leads potential respondents to expect to receive a questionnaire and gives them some idea of what the survey is trying to achieve. Councils might issue a general press release through the local press or through relevant local carer group networks. It may also be useful to let local carers’ organisations know about the survey, so they can publicise it with carers. A leaflet with a notice to carers may also encourage people to respond.

It is essential that the confidentiality of participating carers is assured. Council staff, other than those administering the survey, cannot know whether individuals have completed a questionnaire or how questions were answered. The questionnaires, covering and reminder letters all contain assurances of confidentiality.

A unique code will need to be generated by councils so that returned questionnaires can be mapped back to the participant. This will enable survey teams to follow up those who have not initially responded to the survey, and to bring together information from a variety of data sources. CASSRs will therefore need to consider how information is handled within the council. Due attention must be paid to the new General Data Protection Regulation (GDPR) which has been applied from 25 May 2018.

Importantly, access to the list of carers’ details used to send out the questionnaire will need to be strictly controlled. Computer files need to be password protected and/or stored on a secure server, and access to these must be restricted to those involved in mailing out the questionnaire and responsible for either pre-coding the questionnaires or setting up codes for the analysis. Unsecured email must never be used to send content which includes the names or other personal details of carers. Completed questionnaires must be stored securely in locked cabinets.

Councils must retain paper copies of the questionnaires until the publication of the final 2023-24 SACE report. After this point you are free to securely dispose of paper copies, although you may wish to hold on to them for longer if you believe for example, that participants are likely to take part in any follow up studies. The treatment of paper and scanned copies must be in line with your local Information Governance procedures and meet the requirements of the GDPR. 

Councils may consider outsourcing elements of the survey process to a third party. For example, a data capture company might be used to scan completed de-identified questionnaires into a dataset format. This approach may help councils to meet submission deadlines by removing the need to enter data manually.

In determining whether to outsource elements of the survey to a third party, councils should refer to their local Information Governance arrangements in the first instance, as they will need to ensure that all relevant information governance issues are considered throughout the process of exploring, making and implementing an outsourcing decision. Again, councils will need to attend to the new GDPR.

In running the survey, councils are likely to be acting as a data controller, outsourcing tasks, under clear instruction and with strict controls, to third parties who then act as data processors in carrying out the agreed work. Councils therefore are responsible for ensuring that any third party operates by an acceptable code of conduct, particularly around the issue of confidentiality.

If any of the carer services are outsourced, councils should ensure that the relevant systems and agreements for data sharing and protection are established with the relevant parties. Some councils have experienced problems persuading partner organisations, responsible for the provision of services, to share contact details in order to enable councils to send out questionnaires. It is recommended that data sharing requirements are written into contracts when setting up such arrangements to avoid this situation.

Where required, councils should also satisfy themselves that carers are made aware that data sharing with third parties may be required to ensure both the smooth running of the assessment and delivery process, and for evaluating the performance of providers and councils via mechanisms such as surveys. As with outsourced survey partners, this issue must be considered in the context of the GDPR.