Preparing the survey

This section covers the actions CASSRs need to take to prepare the survey before questionnaires are sent out to participating users. Ahead of this fieldwork period councils must complete a variety of tasks, including making necessary amendments to written materials, determining the eligible population and generating administrative information. Councils also have the option to include additional questions from the question bank for local research purposes. Please see the 'Additional questions' section of the guidance for further details.

CASSRs need to customise the materials provided by NHS England. Sections which require amending are identified with red text and yellow highlighting (all highlighting must be removed once amendments are completed).

Councils may also adapt the materials to match the formats they use to produce and process their other publications. This may include for example, making changes to enable the use of specialist survey software in the production of, and scanning of, questionnaires. Any such amendments must be minor and must not be made unless prior consent has been given by NHS England.

Councils are no longer required to email blank versions of their letters and questionnaires to NHS England for review.

CASSRs will need to finalise two letters (ASCS letter to care home and supported living managers to inform or check capacity and ASCS letter to care home and supported living managers to encourage response) which are to be sent to the managers of residential and nursing care homes and supported living establishments. 

Where the CASSR hold information to assess if a resident has capacity to consent to the survey, this can be used and the survey sent directly to the resident. In this situation, the CASSR just needs to inform the care home that the survey is taking place. If the CASSR does not hold information to allow them to assess if a resident has capacity to consent, the letter to care homes is used to identify service users who do not have the mental capacity to consent to participate in the survey.

The letters also promote the survey to managers and encourage their active involvement in ensuring that questionnaires reach service users and that support is provided to allow service users to participate. Care home managers can inform councils that the service user has permanent loss of capacity. Where this has been indicated, the service user should be excluded from the sampling frame for all future surveys.

If there is any doubt as to whom the letters should be addressed to (for example if the name of the care home or supported living manager is not known), they must be addressed to The Registered Manager to ensure that they are opened. These letters may be sent before or after the initial sample has been selected see the contacting care homes section. 

CASSRs may choose to email the letters to the care home managers rather than sending them via post. If using emails, CASSRs must ensure that only secure email addresses are used for both sender and recipient. As outlined in the confidentiality section unsecured email must never be used to send content which includes the names or other personal details of users.

CASSRs must also finalise the letters which are to be sent to the participants along with the questionnaires. The name of their social services department or relevant service user’s team must be inserted into the text where indicated along with relevant dates. CASSRs must add contact details including a postal and email address and a telephone number. Also, a telephone number must be included for service users to access independent support and advice to help them complete the questionnaire. The council’s corporate logo and the Director of Adult Social Services’ signature must be added. No other format changes are permitted. No text may be withdrawn, added or amended.

In the covering and reminder letters, CASSRs are required to add contact details for the service user to make independent complaints. The contact details need to be independent of the Adult Social Care or Research team that is carrying out the survey. It is permittable to include contact details of the CASSRs complaint service provided this is independent of the survey.

Following the introduction of GDPR in addition to the cover letter, CASSRs must also provide additional transparency information to the service user to inform them of their rights. This information should be in the form of a separate printed information sheet and / or a link to the CASSRs online privacy / transparency notice. Further information on GDPR can be found on the Information Commissioner’s Office website. 

An appropriate location for a unique serial number for tracking respondents must be identified. This is so that the non-questionnaire information can be added to the questionnaire data on the data return, although not so that council staff can identify the respondent (see the confidentiality section). This should be placed to minimise the opportunity for inadvertent removal by respondents.

Participating councils must also decide how many of the optional questions they wish to use. You may choose as many or as few as you wish and must remove those you do not use from the questionnaire.  These include some optional free text questions as well as a question concerning participants willingness to participate in further research. If you choose to include an optional question in the standard survey, then it should also be included in the easy read and translated versions.

If an optional question is removed, the next question must be brought up so new white space is not added to the survey. If CASSRs choose not to use the optional questions, the question numbers should be changed to reflect this and so not cause confusion - for example, if question 4b is not being used, question 4a should be renamed to question 4.

Certain changes to the questionnaires must not be made and are not permitted under any circumstances. These include:

• deleting any mandatory question.
• changing the wording of existing questions or response options.
• changing the size or style of font used.
• removing white space so that the questionnaire fits on fewer pages.
• removing the dotted lines between response categories.
• adding council details, corporate logos or other council branding.
• removing or changing any of the pictures on the easy-read version of the questionnaire.
• changes to the structure of the easy-read version of the questionnaire that result in each question not being on a separate page (with the exception of parts a and b of the same question)

Given that these unpermitted changes might undermine the survey results, then it is possible that if they occur they will be itemised in the data quality section of the survey report; to allow those who use the data to assess its strengths and comparability.

CASSRs must include contact boxes at the end of the survey to ask if the respondent wants to receive a copy of the report. CASSRs can either send the full NHS England report or a local version. The text next to the boxes can be amended to reflect what report the CASSR choose to send. A postal version of a report must be included as an option.

The materials sent to service users must include only those needed to complete the survey. The survey must not be used to send out any other information such as promotional leaflets.

As already noted, CASSRs may also add questions to the survey for local research purposes. Councils could consider including questions in conjunction with other councils in the same region to provide benchmarking data to support locally focussed improvements. The inclusion of locally relevant questions may encourage participation and improve response rates.

Additional questions may take the form of discrete questions or free text boxes seeking further information on existing questions.

Councils can add in up to a maximum of six additional questions into their survey, this includes questions from the question bank, previously approved questions, and new questions.

• previously approved additional questions, and questions from the question bank, do not need to be approved for use in the 2024-25 survey
• all new additional questions need to be requested for approval via the Additional Question Request form once approved they can be added to the survey
• previously approved questions are available for consideration in the question bank on the NHS England website
• the deadline for making new question requests is Friday 22nd November 2024

Questions must be subject to cognitive testing to ensure that they are understood by service users in the way that is intended, unless they are known to have been used in previous surveys. They must also be approved in accordance with local research governance arrangements, since they will not have been accounted for in the research ethics clearance given by SCREC.

When considering the inclusion of additional questions, the following should be kept in mind.

When planning the use of additional questions, you should think about why you are collecting the information and how it will be used. Unnecessary information should not be sought, as this would be an intrusion and a waste of the respondents’ time.

Consideration should be given to adding free-text boxes rather than additional questions with response options in cases where there is a relevant approved free-text box in the question bank. These boxes allow respondents to provide contextual information which may be very relevant for local purposes.

Councils should consider where best to place the additional questions. Ideally, additional questions should be placed towards the end of the questionnaire, before the final optional question on taking part in further research. This rule may be relaxed in circumstances where an additional question clearly follows on from an existing question, and where it is clear that its placement will not affect the understanding of subsequent questions.

It is advised that additional questions are included in all versions of the questionnaire.

CASSRs should not include data from any additional questions in the data return submitted to NHS England.

CASSRs will need to determine the list of people who are eligible to participate in the survey (the eligible population), the group from which the sample is drawn (the sample frame) and the list of those to whom questionnaires are distributed (the sample).

Figure 2: Path from Eligible Population to Final Sample

On any date between 30 September and 31 December 2024 inclusive councils must extract the population of service users who are eligible to participate in the survey.

The eligible population is all those who would have been reported in table LTS001b of the SALT return if this were populated in relation to the chosen extract date rather than 31 March. To be included, a service user must, at the point that data are extracted, be in receipt of long-term support services funded or managed by the council following a full assessment of need. This includes part-funded and full cost paying clients.

Once the eligible population has been extracted councils should check that their records are correct, for example that all contact data are up to date.

As the data is checked it may be discovered that some records were out of date at the time of extraction, and that some subjects in the extract should not have been included in the eligible population figure, for example if a service user had died or moved away before the eligible population extract date. In such cases the subjects must be removed from the eligible population figure.

Once the eligible population has been extracted it must be divided into discrete strata, so that the sample size may be calculated for each stratum and a systemic random sample may be drawn (see the sample section). The ASCS divides the population into four strata which are derived from service users’ primary support reasons, ages and support settings:

• Learning Disability Support – all ages – all settings
• Non-Learning Disability Support – 18-64 – all settings
• Non-Learning Disability Support – 65+ – Long-term residential or nursing care
• Non-Learning Disability Support – 65+ – Community based services (including supported living)

Figure 3 is a flow chart demonstrating how to assign each service user to a stratum.

At this stage, depending on the completeness and quality of the data extracted, councils might also wish to flag and exclude members of the eligible population who cannot be selected to receive a questionnaire, thereby creating a discrete sample frame. Those to be excluded include:

• service users who have died since the data extract
• service users who have moved away from the area since the data extract
• service users who will be in hospital at the time of the survey
• service users who are in active dispute with the council at the time the survey is being sent out. A dispute is defined as a complaint that cannot be resolved by the council and moves on to be handled outside of the council by the ombudsman.
• service users who are involved in an open safeguarding alert or investigation
• service users who have contacted, or whose carers/advocates have contacted, the council in the past expressing a preference to opt out of surveys or complain about receiving a survey (please note that the national data opt-out does not apply to the 2024-25 survey
• service users who are in prison or a detention centre.
• residential or nursing care home service users who are known to have permanent loss of capacity, as indicated by care home managers in their response to the letter to care homes.
• Sservice users who are known to lack the mental capacity to consent to participate in  the current year’s survey, this includes service users in the community, in residential or nursing care or supported accommodation. This information could be from a social care assessment.
• service users who are known to have a Court Appointed Deputy, indicating that they lack the mental capacity to consent to take part. If a council cannot identify if the service user has a Court Appointed Deputy, for example the outcome of a court assessment is not known, then they should not be removed from the sampling frame.
• service users under a Deprivation of Liberty Safeguard (DoLs) authorisation. If a service user is under a DoLs authorisation, then they have had a mental capacity assessment satisfied which says they lack capacity to make decisions about their care and so indicates they lack capacity to consent to take part in the survey.
• service users who are registered within the council but are temporarily placed out of the council’s area should remain in the Eligible Population and in the sampling frame if they are being funded by the council.

Note that it is not compulsory to undertake all these checks and exclusions at this point. However, if you wait until after the sample has been drawn those who are then excluded will have to be replaced on a like-for-like basis. It is likely that late in the survey process you will have to send out a larger number of questionnaires to service users drawn from a reserve or top up sample, to meet the required response rate.

You may also consider excluding eligible service users if they have recently participated in a survey and you believe that resurveying them could lead to 'survey fatigue' and a low response rate. However, such service users may be removed only if councils can still achieve a sufficiently large sample size to meet the minimum requirement, and if removing the previous survey’s participants would not cause this survey’s sample to be biased. Bias could occur for example, if your previous survey focused on service users in a certain geographical area or people with a particular impairment. If you are unsure about introducing bias into your sample frame then you must contact NHS England for advice. Note that a service user appearing in the sample for the ASCS in consecutive years would not be considered to cause survey fatigue and would not be a legitimate reason to remove them from the sample.

It is essential that councils do not exclude service users on the basis that they have a reputation for complaining about services. This would seriously bias the results of the surveys and thus reduce their usefulness. 

It is important to remember that any service users removed for the above reasons remain part of the eligible population figure, which is used to calculate the necessary response rate and is reported to NHS England. Please note, this guidance has been checked following the introduction of GDPR and the guidance remains that if a service user chooses to be removed from the sampling frame they are still to be included in the eligible population.

If information is known that a service user currently lacks capacity to consent to take part in the survey, then the service user should be excluded from the sampling frame. This information could be from a social care assessment or other information provided by a social care professional. As explained in the sampling frame section, if information on the capacity of residents in nursing and residential care home is known, then this can be used, and councils do not need to include these residents in the letter to care homes.

The ASCS employs a stratified random sampling method because there is a wide range of service users accessing different services and support in different settings. By ensuring that participants are selected from each of the designated strata it is possible to ensure that the sample represents the range of the whole population, thereby minimising the sampling error of any statistics which are estimated from the survey data. This helps ensure the robustness of the estimates in the final NHS England publication and also enables NHS England to verify that the proportional breakdown of the sample used, and the responses received, are reasonably aligned with the stratified structure of the eligible population.

Stratification also provides the opportunity for councils to sample a different proportion of users in each stratum if they wish (for example, oversample in some of the smaller strata which may be of interest to them) and thereby increase the precision of any survey estimates for a particular sub-population of their users.

Once all service users in the Eligible Population have been assigned to a stratum, CASSRs must estimate how many service users should be included in the sample from each stratum for the number of respondents to meet the required overall margin of error of less than +/-5 per cent. This can be done using the Sample Size Calculator for the Adult Social Care Survey.

You should enter the number of eligible service users in each stratum along with the predicted response rates for each stratum into the first table, Fulfilling Minimum Requirements by Strata. Response rates from previous ASCS surveys should be used to inform these estimates wherever possible. It is advisable however to under-estimate these anticipated response rates, as if these are not achieved, it is possible that the overall margin of error requirement will not be met.

An example calculation performed using the sample size calculator is shown in Table 1 below.

This shows that for a council with an Eligible Population of 12,000 service users, 373 completed questionnaires need to be returned to meet the minimum margin of error requirement (+/-5 per cent). Along with the anticipated response rates, this shows that the required sample size is 856 (= 63 + 170 + 156 + 467).  

The margin of error that will be obtained is given in the last column. Note that whilst the overall margin of error is +/-5 per cent, the level of accuracy for each stratum is significantly lower, varying from +/-7.1 per cent for strata 4 to +/-17.3 per cent for strata 1. Hence if councils plan to use the stratified data for local analysis, these wider margins of error would make findings less conclusive.

Councils may therefore choose to ‘oversample’ in any stratum to reduce its margin of error and increase the precision of any survey estimates for that sub-population. The second table on the Sample Size Calculator worksheet, Oversampling to Improve Precision in Stratum Results, can be used to calculate the required sample size within a stratum based on a nominated level of precision.

An example calculation can be seen in Table 2 below. Following on from the calculation shown in Table 2, this shows that in order to decrease the margin of error in Stratum 1 to +/-5 per cent, the sample size for this stratum needs to increase from 63 to 557.

Note however that councils must not send out fewer than the number of questionnaires needed in each stratum to meet the minimum requirement. It is not permitted to ‘under sample’ in any stratum, even if this is intended to offset oversampling in another. CASSRs must aim to meet the minimum requirement in terms of the number of returned questionnaires required in each stratum as only this will ensure that the overall requirement on the number of returned questionnaires is met. A check is built into the spreadsheet to identify situations where under sampling occurs.

It is also necessary to ensure that the sample is randomly selected. The ASCS uses a process known as systematic random sampling. A random starting point in a list of service users is identified and then that and every subsequent nth individual is selected to be in the sample. Councils must randomly list the service users in each stratum (this may consist of members of the whole eligible population or a discrete sample frame depending on whether omissions have been done). A random starting point and required sampling interval for each stratum are calculated in the Sample Size Calculator spreadsheet. You will not need to input new data here as these tabs will be pre-populated, using the information entered to calculate the sample size.

At this point you may also choose to select a reserve sample, to act as a data source in cases where service users must be removed from the primary sample, or if you find that the response rate is insufficient to meet the margin of error requirement. By removing members of the initial sample from the sample frame and estimating the number of people you might need, you can use the sample size calculator to generate a reserve list.

To use the Sample Size Calculator to identify the sampling interval needed to select a reserve sample.

Remove those in the initial sample from the list of eligible service users to create an eligible population subset.

Calculate the new population size for each stratum (the original population size minus the original sample size – these totals should match the record level data contained within the eligible population subset created in the step above) and enter these into the first column of the Fulfilling Minimum Requirements by Strata section of the Sample Size Calculator.

Calculate the expected number of service users to be replaced (for example, by looking at the numbers replaced in previous years) plus x per cent, where x is an inflation factor designed to ensure adequate numbers of service users are available on the reserve sample to cover removals from the original sample. Enter the required reserve sample sizes for each stratum into the ‘Sample Size’ column of the Fulfilling Minimum Requirements by Strata section of the Sample Size Calculator by overwriting the existing formulae.

See the Selecting the Sample worksheet for updated random start and sampling interval results.

Once the initial sample has been selected it should be checked for those who should be removed for the reasons described in the sampling frame section. Even if checks and exclusions have already been made it is likely that between the eligible population extract date and the date the sample is generated some users will no longer be suitable participants, for example those who have moved away from the area or died.

Any service user removed from the sample for the reasons listed in the sampling frame section must be replaced with another service user from the same age group, primary support reason and support setting.  For example, if a service user aged 18-64, in receipt of community-based services for mental health support is removed from the sample then they must be replaced with another service user who shares these characteristics. Note that these conditions are more specific than stratum.

Replacement users will also need to be randomly selected, either from the pre-existing reserve sample or from a new reserve sample generated using the methods described above (see the selecting the sample section). As with the initial sample, any top-up sample must be drawn using systematic random sampling, drawn from the matched age group, primary support reason and service setting. If the reserve sample was drawn at the same time as the initial sample, this reserve list may be used to identify like-for-like replacements as an initial alternative to drawing top-up samples.

Service users in the top-up or reserve samples will need to be checked against the criteria listed in the sampling frame section before they can be added to the sample as a replacement. The process of removing and replacing service users may therefore be an iterative one. It is also possible that the supply of service users within a category may become exhausted, making it impossible to replace them on a like-for-like basis. In this situation service users should be matched as closely as possible in respect of age group, primary support reason and support setting, using the preferences ranked in Table 3 below.

Table 3: Ranked preferences for replacing service users in the sample when a complete match is not possible

*First choose the largest subclass within the same primary support reason. Where this is also exhausted choose the largest primary support reason.

Once these final checks and changes have been made you will have the list of those who are to be sent questionnaires.

Once the sample has been finalised and questionnaires issued, nobody can be removed from either the sample or the Eligible Population; this includes service users that could have been removed after the sample was drawn due to the reasons described in the sampling frame section.

The only event that can result in a change to the sample once questionnaires have been issued is the addition of people to the sample, where a top-up sample is required, usually in cases where response rates haven't been as anticipated and the overall margin of error requirement looks unlikely to be met. 

CASSRs should extract and check the administrative data from their stored records. This is the data which is to be included in the data return document in addition to the question responses. These are listed in Appendix I and include items such as the age, religion and the type of support provided to the service user.

It is important to remember that the data submitted to NHS England on the data return sheet must be correct as of the date the eligible population is extracted. If any information relating to a service user has changed since the extract date it must not be updated.

It may also be useful for councils to add local variables to the administrative data. Whilst these will not be returned to NHS England they might assist in local analyses. Councils are also advised to record the service users address and contact details and whether the service user has requested an interview. This information should help facilitate the smooth running of the survey.

This data should be recorded in a dedicated survey database or spreadsheet along with a unique serial number allocated by the council. You may wish to build other pieces of information that you might find useful into the code, for example local area or team codes.

At some point after the eligible population has been determined CASSRs must write to the managers of residential and nursing care homes and supported living establishments using the finalised version (see the letters to care homes section) of the template letter ASCS letter to care home and supported living managers to inform or check capacity. This is to ask them to provide information about whether any service users lack the mental capacity to consent to taking part in the survey, or if this is known, to inform the care home about the survey. If the CASSR do not hold information locally to assess if the service user has capacity to consent to participate in the survey, then mental capacity checks must take place for service users in care homes and supported living arrangements (including shared living schemes and extra-care housing for people with dementia), to ensure compliance with the Mental Capacity Act 2005 and the SCREC ethical clearance for the survey. As outlined in the section letters to care homes if information on capacity is known for a residential or nursing home service user, for example through social care assessments, then councils do not need to request the information for this specific service user from their care home.

Some CASSSRs have fed back that they have had a better response from care homes when they have phoned to contact them. 

This letter can be sent at one of 2 stages:

• before the initial sample is selected, in which case details must be requested for all council-funded or managed service users in the care home or supported living setting
• after the initial sample has been selected, in which case details will be requested only for those services users included in the sample

The advantage of sending this letter prior to selecting the sample is that individuals who lack the capacity to consent can be flagged so that they are not selected and do not have to be subsequently substituted.  However, this approach can be relatively resource intensive and may not be practical for CASSRs which have a large number of service users in these settings.

Note, following the change to the capacity check letter, it is no longer necessary to assess the capacity of all service users each time the survey is run; if a service user is known to have permanent loss of capacity, they should be excluded from the sampling frame of future surveys.

If no response to this letter is received, then it should be assumed that the service users have the capacity to consent and should be retained in the sample frame and/or sample if selected. If responses are received and the outcomes are kept on record then it must be made clear that these relate to an assessment of the capacity to consent to take part in a survey, and not a medical assessment of capacity. A lack of capacity to consent may be temporary and so steps should be taken to ensure that council records are kept up-to-date.

The second letter, which is to be sent out a few days in advance of questionnaires, provides the manager with a list of service users who will be sent a questionnaire, as well as further information about the survey and how they and their staff can assist service users to participate. The letter also gives a commitment that the results of the survey will not be used to assess the performance of the individual care home or supported living establishment and this commitment must be honoured by CASSRs.

Firstly, councils will need to ensure that the correct versions of the questionnaires are allocated to participants depending upon whether they live in the community or residential and nursing setting.

In most instances the appropriate forfmat for participants will be the standard English questionnaire. However, to maximise response rates and represent the full range of service users (see the sample section), councils must identify any people in their sample who will need access to alternative versions of the survey. CASSRs must check their records for information about a service user’s likely communication needs, and whether it is likely that a service user would be able to complete a postal questionnaire unaided. If a service user cannot complete a postal questionnaire unaided:

• Is there someone who is already known to the participant who could help them complete the questionnaire?
• Is it necessary to arrange assistance for this service user?
• Does the participant require an easy read version of the questionnaire?
• Does the participant require a large-print questionnaire?
• Does the service user require the questionnaire translated into a language other than English or the provision of an interpreter?

Much of the above information should be available from your service user database. If a service users record indicates that the letter and questionnaire should be sent to a carer, then the letter should be addressed to “care of” the service user.

The easy read versions have been produced for people with a learning disability and should be used for all service users in stratum 1, unless otherwise requested. They should also be used for any other service user who is known to have a learning disability or is known to have requested the easy-read version in the past.

It is possible that the support setting of some users will change between the date of the eligible population figure extraction and the dates the questionnaires are sent out. In such instances the questionnaire which applies to their current situation should be used. If for example, a respondent moves from a community setting into residential care then the residential questionnaire must be issued even though the recorded support setting remains community.

As noted above it is possible to administer the survey as a face-to-face or telephone interview. This may be the appropriate course for example, for respondents who have sensory impairments or limited literacy skills. Interviews should only be offered to service users who request them in response to the initial mailing or reminder letter and where it is not possible to find appropriate support to enable them to complete a postal questionnaire. Overall the number of interviews completed should be small in relation to the size of the sample. Councils could also consider using advocacy services, although this is only recommended when a service user already has an ongoing relationship with an advocate.

CASSRs should have on record the information and communication needs of service users with sensory impairments now that organisations that provide publicly funded adult social care must conform to the Accessible Information Standard.  

In the case of people with sensory impairments consideration should be given to the best means to make initial contact and provide details of options for taking part. For example, it might be worth inviting people who are known to have severe visual impairments by telephone. Service users who might require assistance from a sign language interpreter could, if there were no alternative, be first approached by someone known to them, such as a social worker.

In addition to the easy read and large print versions supplied by NHS England, councils will have to make the questionnaire and covering letters available in any additional formats or languages that are requested by service users, including (but not limited to):

• Braille
• audio for visually impaired people
• an electronic version of the questionnaire to be sent and returned via email - note that this option is currently only open to service users who could not otherwise participate

Please note that it is the council’s responsibility to make the materials available in other formats to service users.

An alternative would be to offer a telephone or face-to-face interview for service users who request this. Note however, that this should not be the standard way of obtaining a response from this group.

It is recommended that all councils store details of the first language of their service users as part of their social care database along with their communication preferences. You must arrange for translated documents locally if you need languages other than those provided by NHS England (see Appendix E).

A summary on what documents to send out is below:

• Where the service users first language is known to be English – send the English version of the materials
• Where the service users first language is known and is not English – send the English version of the materials and the relevant translated materials
• Where the service users first language is not known by the council – send the English version of the materials and the translation request sheet.

• Where the service users preferred language is English (even if the first language is not English) – send the English version of the materials and the translation request sheet

In some instances, councils could consider having the questionnaires administered as face-to-face interviews by interpreters (see the interviewing respondents section). Given that response rates to postal questionnaires from ethnic minorities are traditionally low, using interpreters may enable more users from this group to be included. This could be done by family and friends where available, although the use of professional interpreters is recommended. The use of professional interpreters should help to ensure that response options are translated accurately.

Interpreters will need to be able to speak to respondents in colloquial versions of their languages. In addition, the work of interpreters should be monitored to ensure they are not cutting corners or editing respondents’ answers before translating and recording them. One way of doing this would be to have all translated interviews tape-recorded and to arrange for, say, 1 in 10 to be re-coded by a second interpreter as a check.