Study limitations and important data interpretation considerations Print this page

Study limitations 

The nature of synthetic small area estimates

The synthetic small area estimates modelled may be different to the number of actual cases of youth self-harm and suicidality with communities. As with all statistical models, the model used for the Youth Self-Harm Atlas study have underlying assumptions, which if violated, may adversely impact the accuracy of estimates. 

The model used for the Youth Self-Harm Atlas study assumes rates of youth self-harm and suicidality of small areas can be determined on the basis the socio-demographic characteristics of the area. Further, that the relationship between youth suicidality and self-harm, and socio-demographic characteristics does not vary substantially between areas. 

Areas with limited data

Young Minds Matter survey data was sparse for remotes areas of Western Australia and the Northern Territory, which may adversely impact the accuracy of self-harm and suicidality estimates for these areas. 

The timeliness of data used

Data sources used for the study were the Young Minds Matter survey collected during 2013-2014, the 2016 Census, and 2019 Estimate Resident Populations. It is unknown whether the prevalence of youth self-harm and suicidality reported in 2013-14 and the socio-demographic characteristics of young people in 2016, accurately reflect the contemporary experience of young people. 

Nature of the associations visualised within the ‘risk and protective factors' maps

Area level relationships between self-harm and risk and protective factors are likely interrelated and complex in nature. The risk and protective factors for self-harm prevalence maps cannot account for this complexity and instead display only the relationship between a single measured risk or protective factor and a single measure of self-harm. Therefore, the true association between the risk or protective factor and self-harm may be smaller or larger than what is presented in the maps.

Data interpretation considerations

Use of separate statistical models for self-harm and suicidality outcomes

The Youth Self-Harm Atlas used separate statistical models to generate estimates for each self-harm and suicidality outcome included within the study. Each model generated best estimates for each of the study outcomes separately. In addition, some outcome variables had more data available to generate estimates compared to others. This is because Young Minds Matter survey participants can choose to respond that they “prefer not to answer” a question. 

Due to this modelling design, there may be small inconsistencies where outcome estimates do not exactly add to the total estimate. For example, in South Western Sydney PHN, the estimated 12-month non-suicidal youth self-harm was 10%, and the total estimated self-harm (regardless of intent) was 9.4%. 

Area level and individual person level data

When interpreting the numeric data of the Youth Self-Harm Atlas, inferences can only be drawn about experiences of young people at the level of the geographic area under consideration. Just as there is variation between area level outcomes, there is also variation, and sometimes substantial variation, within areas. This means that area level findings do not apply to every individual living within the area. It is not appropriate to use data or information generated by the Youth Self-Harm Atlas to make inferences about the experience of individual young people. This is regardless of whether the inference is made about a specific young person or about individual young people in general terms. Erroneously drawing conclusions about individual people based on aggregated data for a group of people, is known as the ecological fallacy (Firebaugh, 2015). 

Aggregating data to different types of geography

Analysis of the same dataset about individual people may provide different results depending on the size and shape of the geographic areas used to aggregate the data. This problem, which effects all spatial analysis of aggregated data, is referred to as the modifiable areal unit problem (Wong, 2009; Lloyd, 2014; Tuson et al., 2019). The Youth Self-Harm Atlas data are presented using different types of standardised geographic geographies commonly used by governments and healthcare providers. 

Co-occurrence is not causation

The Australian Youth Self-Harm Atlas study investigates regional variability in suicidality and self-harm, as well as risk and protective factors, for young people aged 12-17 years of age. The study also explores area level co-occurrence of self-harm and risk and protective factors. The study does not investigate or provide evidence that the risk or protective factors may be causing self-harm (or vice versa).