Preparing the survey

This section covers the actions CASSRs need to take to prepare the survey before questionnaires are sent out to participating carers. Ahead of this fieldwork period councils must complete a variety of tasks, including making necessary amendments to written materials, determining the eligible population and generating administrative information. Councils also have the option to formulate and submit to NHS England their own questions to include in the survey for local research purposes.

CASSRs need to customise the materials provided by NHS England. Sections which require amending are identified with red text and yellow highlighting (all highlighting must be removed once amendments are completed).

Councils may also adapt the materials to match the formats they use to produce and process their other publications. This may include for example, making changes to enable the use of specialist survey software in the production of, and scanning of, questionnaires. Any such amendments must be minor and must not be made unless prior consent has been given by NHS England

Ideally as soon as amendments to letters and questionnaires are finalised then you should email blank copies of your questionnaires, cover and reminder letters to the team directly at [email protected]. This will allow for further checks on the extent to which the survey has been carried out in line with this guidance. They must be sent by 30 November (the recommended fieldwork period end date). The materials submitted must be the final ones used for the survey. The review by NHS England forms part of the data quality assessment processes and is not to approve the materials before councils send them to users. If NHS England believes that councils have changed materials in a manner which has compromised the robustness or comparability of the survey results, then it may be necessary to note these incidences in the Data Quality section of the survey report.

CASSRs must insert the name of their social services department or relevant carer’s team into the text where indicated along with relevant dates. CASSRs must add contact details including a postal and email address and a telephone number. Also, a telephone number must be included for carers to access independent support and advice to help them complete the questionnaire. The council’s corporate logo and the Director of Adult Social Services’ signature must be added. No other format changes are permitted. No other text may be withdrawn, added or amended.

In the covering and reminder letters, CASSRs are required to add contact details for the carer to make independent complaints. The contact details need to be independent of the Adult Social Care or Research team that is carrying out the survey. It is permittable to include contact details of the CASSRs complaint service provided this is independent of the survey.

Following the introduction of GDPR in addition to the cover letter, CASSRs must also provide additional transparency information to the carer to inform them of their rights. This information should be in the form of a separate printed information sheet and or a link to the CASSRs online privacy or transparency notice. Further information on GDPR can be found on the Information Commissioner’s Office website. 

An appropriate location for a unique serial number for tracking respondents must be identified. This is so that the non-questionnaire information can be added to the questionnaire data on the data return, although not so that council staff can identify the respondent. This should be placed to minimise the opportunity for inadvertent removal by respondents.

Participating councils must also decide how many of the optional questions they wish to use. You may choose as many or as few as you wish and must remove those you do not use from the questionnaire. These include some optional free text questions as well as a question concerning participants willingness to participate in further research. If you choose to include an optional question in the standard survey, then it should also be included in the translated versions.

The questions asking the age, gender and ethnicity of participants should be removed if councils can provide NHS England this information from their records. They must remain in the questionnaires if this demographic data is not obtainable by other means.

Certain changes to the questionnaires must not be made and are not permitted under any circumstances. These include:

• deleting any mandatory question
• changing the wording of existing questions or response options
• changing the size or style of font used
• removing white space so that the questionnaire fits on fewer pages
• adding council details, corporate logos or other council branding.

Given that these unpermitted changes might undermine the survey results, then it is possible that if they occur they will be itemised in the Data Quality section of the survey report to allow those who use the data to assess its strengths and comparability.

The materials sent to carers must include only those needed to complete the survey. The survey must not be used to send out any other information such as promotional leaflets.

As already noted CASSRs may also add questions to the survey for local research purposes, subject to approval by NHS England. Councils could consider including questions in conjunction with other councils in the same region to provide benchmarking data to support locally focussed improvements. The inclusion of locally relevant questions may encourage participation and improve response rates

Additional questions may take the form of discrete questions or free text boxes seeking further information on existing questions. You may also add response options to questions 5 and 6. Additional response options for questions 5 and 6 should be submitted to NHS England using the additional question form for information purposes. Please note that these do not count in the additional question cap.

Questions must be subject to cognitive testing to ensure that they are understood by carers in the way that is intended, unless they are known to have been used in previous surveys. They must also be approved in accordance with local research governance arrangements, since they will not have been accounted for in the research ethics clearance given by SCREC.

A selection of questions used in previous surveys is available in the Question Bank on the user survey pages. 

Any proposals for additional questions and free text boxes must be submitted to NHS England using the Additional Question Request Form. The form is obtainable at SACE, 2023-24 guidance for local authorities and once Section 1 is completed should be emailed to [email protected]. NHS England will reply to any request using Section 2 or by contacting the CASSR directly. The deadline for submission is Friday 8 September 2023.

Note that ALL proposals for additional questions need to be submitted for review by NHS England irrespective of whether these questions have been used before by the same or other organisations.

When considering the inclusion of additional questions, the following should be kept in mind:

Questions must address a genuine research need and relate to the general purpose of the survey. 

The number of questions added must not be excessive. The inclusion of too many questions may be perceived as a burden by respondents and have a negative impact on response rates. NHS England advise no more than six additional questions and / or free text boxes are added to the questionnaires.

When planning the use of further questions, thought should be given to why you are collecting the information and how it will be used. Unnecessary information should not be sought, as this would be an intrusion and a waste of the respondents’ time.

Consideration should be given to adding free-text boxes following existing questions as an alternative to adding whole new questions. These boxes allow respondents to provide contextual information which may be very relevant for local purposes.

Additional questions should be placed towards the end of the questionnaire, before the final optional question on taking part in further research. This rule may be relaxed in circumstances where an additional question clearly follows on from an existing question, and where it is clear that its placement will not affect the understanding of subsequent questions.

It is advised that additional questions are included in the translated versions of the questionnaire.

It is recommended that additional questions are relevant to all carers receiving a questionnaire; this will help to mitigate any potential negative impact on response rates that including additional questions may have. One way to ensure questions remain relevant to all carers is to include a ‘not applicable’ type option.

Additional questions must not be used to gather information about the religion or sexual orientation of carers

Questions designed to assess reactions to proposed new services will not be permitted. This type of question may raise a respondent’s expectations about receiving such services and potentially influence their responses to other questions.

CASSRs will need to determine the list of people who are eligible to participate in the survey (the eligible population), the group from which the sample is drawn (the sample frame) and the list of those to whom questionnaires are distributed (the sample).

Path from eligible population to final sample

Eligible population

All carers over 18 who would be contained within the most recent SALT measure LTS003, table 1, if the table were populated on the council’s chosen extract date rather than 31 March.

Sample frame

The eligible population minus those who have been excluded because their circumstances have changed since the extract date and those excluded due to survey fatigue

Initial sample

Those carers who have been randomly selected to receive a questionnaire.

Final sample

Those who are sent a questionnaire following any final exclusions and replacements. 

On any date between 1 July and 30 September 2023 inclusive councils must extract the population of carers who are eligible to participate in the survey.

The eligible population is all carers who would be contained within the last four rows of SALT measure LTS003, table 1 for 2022-23, if this table was populated in relation to the chosen extract date rather than 31 March (those who would be included in the yellow highlighted section of figure 3). These are all those aged 18 and over, and who either received “support direct to carer” or “no direct support to carer” in the 12 months prior to the extract date, irrespective of whether their cared-for person received respite care. The first row of this table is excluded because it contains carers who are aged under 18.

Once the eligible population has been extracted councils should check that their records are correct, for example that all contact data are up to date.

As the data is checked it may be discovered that some records were out of date at the time of extraction, and that some subjects in the extract should not have been included in the eligible population figure, for example if a carer had died, moved away or stopped their caring over 12 months before the eligible population was extracted. In such cases the subjects must be removed from the eligible population figure.

At this stage, depending on the completeness and quality of the data extracted, councils might also wish to flag and exclude members of the eligible population who cannot be selected to receive a questionnaire, thereby creating a discrete sample frame. Those to be excluded are people whose circumstances have changed since the data extract such as

• carers who have died
• carers who will be in hospital at the time of the survey
• carers who have moved into residential or nursing accommodation
• carers who are in active dispute with the council at the time the survey is being sent out. A dispute is defined as a complaint that cannot be resolved by the council and moves on to be handled outside of the council by the ombudsman
• carers who have contacted the council in the past expressing a preference to opt out of surveys or complain about receiving a survey (please note that the national data opt-out does not apply to the 2023-24 survey)
• carers whose cared-for person has died
• people who were carers for only a short spell that ends before the survey is carried out
• carers should not be removed if they have moved out of the area, if they are still supported by the counci

Note that it is not compulsory to undertake these checks and exclusions at this point. However, if you wait until after the sample has been drawn those who are then excluded will have to be replaced. It is likely that late in the survey process you will have to send out a larger number of questionnaires to carers drawn from a reserve or top up sample, to meet the required response rate.

You may also consider excluding eligible carers if they have recently participated in a survey and you believe that resurveying them could lead to “survey fatigue” and a low response rate. If you require advise on whether or not to exclude carers due to “survey fatigue”, please contact NHS England. However, such carers may be removed only if councils can still achieve a sufficiently large sample size to meet the minimum requirement, and if removing the previous survey’s participants would not cause this survey’s sample to be biased. Bias could occur for example, if your previous survey focused on carers in a certain geographical area or carers of people with dementia. If you are unsure about introducing bias into your sample frame then you must contact NHS England for advice.

It is essential that councils do not exclude carers on the basis that they have a reputation for complaining about services. This would seriously bias the results of the surveys and thus reduce their usefulness. 

Potential participants should not be excluded purely on the basis that their cared-for person has moved into residential or nursing care since their last review.

It is important to remember that any carers removed for the above reasons, remain part of the eligible population, which is used to calculate the necessary response rate and is reported to NHS England.

This guidance has been checked following the introduction of GDPR and the guidance remains that if a carer chooses to be removed from the sampling frame they are still to be included in the eligible population.

Once the size of the eligible population has been established CASSRs must work out the size of sample necessary to ensure that the survey results have the required margin of error of less than +/-5 per cent. This is done by entering the number of the eligible population into the sample size calculator (available at SACE, 2023-24 guidance for local authorities), along with the predicted response rate. The predicted response rate could be estimated from local surveys of carers councils may have conducted, or the response rate to the last SACE.

CASSRs may choose to send out more questionnaires than needed to meet the minimum requirement, thereby reducing the margin of error and increasing the accuracy of the survey. If you decide to take this approach, then the sample calculator also allows you to estimate the precision of the survey.

It is also necessary to ensure that the sample is randomly selected. Councils must randomly list the carers in the sample frame. A random starting point and required sampling interval are calculated in the Sample Size Calculator spreadsheet on the worksheets entitled Selecting the Sample-minimum and Selecting the Sample-more. These tabs will be pre-populated, using the information entered to calculate the sample size.

At this point you may also choose to select a reserve sample, to act as a data source in cases where carers must be removed from the primary sample, or if you find that the response rate is insufficient to meet the margin of error requirement. By removing members of the initial sample from the sample frame and estimating the number of people you might need, you can use the sample size calculate to generate a reserve list.

Once the initial sample has been selected it should be checked for those who should not have been in the eligible population and those who should have been removed from the sample frame for the reasons described above. Even if checks had been made it is likely that between the eligible population extract date and the date the sample is generated some carers will no longer be suitable participants, for example those who have moved away or died. Replacement carers will also need to be randomly selected, either from the pre-existing reserve sample (see above) or from a new reserve sample.

After these checks and changes have been made you will have the list of those who are to be sent questionnaires. Once the sample has been finalised and questionnaires issued, nobody can be removed from either the sample or the Eligible Population; this includes carers that could have been removed after the sample was drawn due to the reasons described. The only event that can result in a change to the sample once questionnaires have been issued is the addition of people to the sample, where a top-up sample is required, usually in cases where response rates haven’t been as anticipated and the overall margin of error requirement looks unlikely to be met.

CASSRs should extract and check the administrative data from their stored records. This is the data which is to be included in the data return document in addition to the question responses. These are listed in Appendix J and include items such as the age and religion of the carer, the type of support provided to the carer and the funding status of the cared-for person.

It is important to remember that the data submitted to NHS England on the data return sheet must be correct as of the date the eligible population is extracted. If any information relating to a carer has changed since the extract date it must not be updated.

It may also be useful for councils to add local variables to the administrative data. Whilst these will not be returned to NHS England they might assist in local analyses. Councils are also advised to record the carer’s address and contact details and whether the carer has requested an interview. This information should help facilitate the smooth running of the survey.

If a carer is caring for more than one person aged 18 or over, you may need to use local information to determine who the main person cared-for is and to populate relevant data items (for example primary support reason). If you are unable to identify this information, then you will have to select 'Unknown' for categories relating to the cared-for person on the data return.

This data should be recorded in a dedicated survey database or spreadsheet along with a unique serial number allocated by the council. You may wish to build other pieces of information that you might find useful into the code, for example local area or team codes.

In most instances the appropriate format for participants will be the standard English questionnaire. However, to maximise response rates and represent the full range of carers, councils must identify any people in their sample who will need access to alternative versions of the survey. CASSRs must check their records for information about a carer’s likely communication needs, and whether it is likely that a carer would be able to complete a postal questionnaire unaided. If not:

• is there someone who is already known to the participant who could help them complete the questionnaire?
• is it necessary to arrange assistance for this carer?
• does the participant require a large-print questionnaire?
• does the carer require the questionnaire translated into a language other than English or the provision of an interpreter?

Much of the above information should be available from your carer database.

CASSRs should have on record the information and communication needs of carers with sensory impairments now that organisations that provide publicly funded adult social care must conform to the Accessible Information Standard. In the case of people with sensory impairments, consideration should be given to the best means to make initial contact and provide details of options for taking part. For example, it might be worth inviting people who are known to have severe visual impairments by telephone. Carers who might require assistance from a sign language interpreter could, if there were no alternative, be first approached by someone known to them, such as a social worker.

In addition to the large print version supplied by NHS England, councils will have to make the questionnaire and covering letters available in any additional formats or languages that are requested by carers, including (but not limited to):

• Braille
• easy read
• audio for visually impaired people
• an electronic version of the questionnaire to be sent and returned via email. Note that this option is currently only open to carers who could not otherwise participate

Please note that it is the council’s responsibility to make the materials available in other formats to carers.

An alternative would be to offer a telephone or face-to-face interview for carers who request this. Note however, that this should not be the standard way of obtaining a response from this group.

It is recommended that all councils store details of the first language of their carers as part of their social care database along with their communication preferences. You must arrange for translated documents locally if you need languages other than those provided by NHS England (see Appendix E).

A summary on what documents to send out is below:

• where the carers users first language is known to be English – send the English version of the materials 
• where the carers first language is known and is not English and the preferred language is either not known or not English – send the English version of the materials and the relevant translated materials 
• where the preferred language is known to be English (even if the first language is not English) then councils can send out an English version of the survey only, alongside the translation request sheet
• where the carers first language is not known by the council – send the English version of the materials and the translation request sheet

In some instances, councils could consider having the questionnaires administered as face-to-face interviews by interpreters.  Given that response rates to postal questionnaires from ethnic minorities are traditionally low, using interpreters may enable more carers from this group to be included. This could be done by family and friends where available, although the use of professional interpreters is recommended. The use of professional interpreters should help to ensure that response options are translated accurately.

Interpreters will need to be able to speak to respondents in colloquial versions of their languages. In addition, the work of interpreters should be monitored to ensure they are not cutting corners or editing respondents’ answers before translating and recording them. One way of doing this would be to have all translated interviews tape-recorded and to arrange for, say, 1 in 10 to be re-coded by a second interpreter as a check..

In addition to making additional formats available councils will have to determine if other assistance is required, usually in response to calls to the additional help contact number on the cover and reminder letters. The council must ensure however, that if the participant needs help to complete the questionnaire that the helper is not involved in delivering services. This would breach confidentiality and decrease the likelihood of carers feeling able to voice criticisms of the service they receive, thereby biasing the survey results. It is acknowledged that this might be difficult to implement, especially in cases where carers are used to turning to staff for help. It is therefore recommended that councils are proactive in offering alternative forms of help to carers to complete a questionnaire, by encouraging friends and relatives to assist them.

It is possible to administer the survey as a face-to-face or telephone interview. This may be the appropriate course for example, for respondents who have sensory impairments or limited literacy skills. Interviews should only be offered to carers who request them in response to the initial mailing or reminder letter and where it is not possible to find appropriate support to enable them to complete a postal questionnaire. Overall the number of interviews completed should be small in relation to the size of the sample. Councils could also consider using advocacy services, although this is only recommended when the carer already has an ongoing relationship with an advocate.