Project Snapshot – The Kids are Not OK – Delphi

The Kids are Not Ok: A Delphi Study to identify consensus research priorities and core outcome sets in paediatric emergency mental health presentations


To determine a prioritised list of research questions and identify a set of core clinical and mental health outcomes for children and adolescents presenting to the ED with mental health concerns.


Children/young people under 18 who attended ED with mental health concerns and their parents/carers. Doctors, nurses, mental health clinicians, police, paramedics, schoolteachers, school counsellors, hospital security.

Current status:

We have completed the three planned survey rounds; however, a fourth round will be introduced to complete the project. Nearly there!

Survey Round 1
  • 184 participants (provided complete and partial responses)
    –  36 patient/carers from 12 participating hospital sites (VIC, QLD and SA)
    –  148 clinician and community-based participants (VIC, QLD, SA, WA and TAS)
  • 989 free-text comments
    –  Service (a comment on service / patient experience / general comments)
    –  Data (a list of things that can be collected about patients who arrive to ED)
    –  Outcome (something that can be measured after ED attendance)
    –  Question (something that could be turned into a research question)
Survey Round 2
  • 267 items from Survey Round 1 were grouped into topics (e.g., ED re-presentation, length of stay, QOL etc)
  • Participants rated items on a 9-point ordinal scale (“not important to extremely important”) and space was provided for optional free text responses for each choice
  • If items were rated 7 to 9 (on the 9-point Likert scale) by 50% or more participants and 1 to 3 by no more than 15% of participants in at least one stakeholder group they were included in Survey Round 3
  • 62 responses in total (25 complete and 37 incomplete responses) from 57 clinician/community-based services and 5 patient/carer respondents
Survey Round 3
  • 127 items were presented for survey round 3.  Grouped and rated as for round 2.
  • A consensus was defined when the following criteria were met (1) 70% of participants scoring outcomes as 7 to 9 and 15% or less scoring 1 to 3 by both stakeholder groups; or (2) 90% or more scoring 7 to 9 from either stakeholder group.
  • 29 responses in total (20 complete and 9 incomplete responses) from 26 clinician/community-based services and 3 patient/carer respondents. The rating for 9 items were inconclusive.
Survey Round 4
  • Instead of convening a consensus meeting for the remaining nine items, we will (after consultation with the steering group) distribute a fourth survey.