Limitations of the data and analysis

Home


  1. The FYFVMH dashboard and MHS dataset only presents data for children’s mental health services funded by the NHS. As such, this report does not examine figures on mental health provision financed by organisations outside the NHS such as school-based counselling or services provided by local authorities (services which may be supported by the NHS but not considered NHS funded). CCGs that spend more on external or prevention based services at the expense of NHS provided CYPMHS may underperform on indicator scores based solely on CYPMHS datasets.

  2. As with data used for national monitoring, a child is counted as accessing treatment if they have two contacts with CYPMH services. This is the best proxy measure currently available until wider measures are routinely collected via the MHSDS. In some cases, a child may have more than one contact before treatment begins, while others may be referred or not need further support from CYPMHS after one contact. Therefore, we cannot confidently state in all cases that a child with less than two contacts did not have their needs met or that every child with two contacts has entered treatment. However, this remains the best proxy measure available due to a lack of other reliable data sources estimating the number of young people receiving treatment at a single contact. It is also in line with the measures used to monitor progress in the Five Year Forward View for Mental Health.

  3. Children whose referrals were closed may not have required specialist treatment or may have been referred to services funded by other routes and organisations (e.g. local authorities and non-NHS funded charities). Some children may also have chosen not to enter treatment even when offered or advised. However, the data provided does not specify why a referral was closed. Until such data is provided, this will be a key gap in establishing the outcomes and circumstances of those referred.

  4. Since last year’s report was published, multiple smaller CCGs merged to form new combined CCGs (see appendix for full list). Where this report compares rates over time, the average rate of the smaller CCGs in previous years is taken to represent the past rate of the combined CCG. Some of the best performing CCGs last year have been merged with lower scoring CCGs which has thus pulled down their score. This could give the impression that the CCG’s performance has worsened over the past year when this may not be the case (and vice versa when worse performing CCGs are merged with better performing CCGs).

  5. The NHS’ 2021 update to the Mental of Children and Young People report, estimates that the prevalence of children with a probable mental disorder has increased substantially during the pandemic (from 1 in 9 to 1 in 6 children). This combined with the negative pressure of covid on the NHS’ capacity to deliver services has reduced the number of children in need of mental health support successfully accessing treatment (when new prevalence rates are taken into account). However, as the provided FYFV access rate estimate is based on much lower 2004 prevalence data, comparison over time shows that the proportion of children with a probable mental health disorder accessing treatment has increased though this may not be the case.

  6. In some cases, children referred near the end of a financial year may enter treatment early in the following year. These children would be shown in 2019/20 data as “still awaiting treatment” despite a relatively short wait. As a result, we cannot assume that all children still waiting for treatment have waited long periods for their second contact. The NHS has also noted that their waiting times data is marked as experimental and requires development until it is deemed reliable and robust enough to become an official dataset.

  7. In its MHSDS data quality report, the NHS noted that each year there are some mental health service providers that do not submit data to the MHSDS. Though the number of providers submitting data has improved over the past few years, the data presented here is still incomplete due to underreporting. Until a system is put in place to capture data from other providers, schools, local authorities and the health and justice sector, the MHSDS remains the best representation of mental health services information that we have available.