Home - Australian Government - Australian Institute of Health and Welfare - logo
Caution: If at any point you feel worried about harming yourself while viewing the information on this website – or if you think someone else may be in danger – please stop reading and seek help.
Help and support
Suicide & self-harm monitoring

Data Quality Framework

Last updated:

On this page In this section

Overview About the Data Quality Framework Data quality assessment application to quantitative data Data quality assessment application to qualitative data Data Quality Framework criteria

Overview

The Data Quality Framework, developed in collaboration with stakeholders, sits alongside the Outcomes Map to guide decisions on including data measures in the Outcomes Framework. The Data Quality Framework forms part of the Data Quality & Improvement Plan and has been released as a first draft by the National Suicide Prevention Office (NSPO) to help users understand the decision-making criteria, identify quality limitations, and highlight where enhancements to data quality may be required. These quality limitations and any possible actions to improve the quality will be outlined in the Data Quality and Improvement Plan.

About the Data Quality Framework

The Data Quality Framework is essential to the Outcomes Framework because it ensures that the data used is reliable, accurate, and fit for purpose. It supports consistent and trustworthy analysis, protects the integrity of sensitive information through a trauma-informed approach, and promotes transparency and accountability in decision-making. By guiding the assessment and ongoing improvement of data quality, the Data Quality Framework helps ensure that the Outcomes Framework can effectively inform policy, prioritise action, and continuously strengthen suicide prevention efforts.

The Data Quality Framework assesses whether data sources are suitable for inclusion in the Outcomes Framework by applying consistent criteria to assess their quality and relevance. This supports transparent, objective decisions about which sources are included. Criteria 1–8 focus on technical data quality, drawing on established Australian Bureau of Statistics (ABS, 2018) and AIHW frameworks (AIHW, 2014), while Criterion 9 considers how well measures align with the aims of the Outcomes Framework and improving our understanding of suicide prevention. 

The Data Quality Framework helps ensure that decisions about including data measures in the Outcomes Framework reflect both sound data governance and suicide prevention priorities. The NSPO, AIHW and qualitative researchers together use the framework as a joint policy to support collaborative decision-making and clarify partner responsibilities based on each of our respective roles and expertise. To download a copy of the Data Quality Framework, please see NSPO’s Data Quality & Improvement Plan webpage. Please see below more information on the assessment process and criteria applied.

Data quality assessment application to quantitative data

For criteria 1 to 8, measures are rated by each partner using their specialised knowledge to assess the quality of data measures against the criteria using a traffic light system. Green indicates the measure meets the criterion and is suitable, amber indicates uncertainty and the need for discussion between partners, and red indicates the measure does not meet the criterion and is not suitable for inclusion. For criteria 9, people with a lived and living experience, government and sector representatives provided a rating of 1 to 5 of least to most important to the Outcomes Framework. Here, the most frequent rating across individual assessments is used for the overall importance rating for each data measures. Stakeholders are encouraged to use the Data Quality Framework in their own data projects and to assess the strength of the Outcomes Framework.

Data quality assessment application to qualitative data

To achieve a quality assessment in the absence of previously available qualitative data for the Outcomes Framework, a prospective assessment was developed to test the co-designed qualitative research questions. ‘Qualitative research questions’ refers to the broad research question to be used for each Objective and Goal in the Outcomes Framework. A construct analysis has been undertaken for each question at the Objective level. Each construct is either explicit (directly named in the question) or implicit (understood as necessary to the context or meaning of the question). Each construct has then been operationalised to provide examples of what would be expected if the Strategy is working as intended. Examples show the continuum of what we would expect to see less of, neutral/midway, or more of within the qualitative data collected at each administration. For more information on the method, please see the Qualitative Method Technical Notes paper. To explore the quality assessment outline and examples, please see the Outcomes Map Technical Workbook. 

Data Quality Framework Criteria

Table 1: Criteria in the Data Quality Framework used for the National Suicide Prevention Outcomes Framework

Data Quality Framework Criteria 

The quality being assessed

Description of what qualities the criterion considers

Why this is important for the Outcomes Framework

Who makes the assessment

1. Institutional environment

The data measure comes from a reputable source 

The assessment considers the origin of the data collection and the arrangements under which the collection is governed and administered.  

This includes whether these arrangements consider ethical and safety factors, such as the proper inclusion of cultural safety and appropriateness, and the way privacy and confidentiality are managed.  

This also includes assessing the context of the data collection, which may influence the validity, reliability or appropriateness of the data.  

This criterion makes sure the Outcomes Framework only uses data from sources that have robust methods and where the data are collected and managed in safe and ethical ways.  

This also ensures the Outcomes Framework uses data from sources that can safely and appropriately represent different cultures and groups disproportionately impacted by suicide.     

This will provide assurance that the data measures in the framework are not contributing to individual, community or institutional harm and help to build and maintain social licence and trust.      

AIHW 

Senior research experts 

2. Relevance

The data measure is relevant to the construct being measured in the outcomes  

The assessment considers how well the data measure relates to the constructs in the Strategy, its objectives, outcomes and indicators.  

It does this from a technical perspective as well as from a lay perspective to ensure the data measure being used is the most appropriate to measure the outcomes and meet the needs of users.  

It also considers the similarities and differences between data measures to understand how they uniquely add to our understanding and how they can be used to compare findings across data measures.  

This criterion makes sure the Outcomes Framework uses data measures that are directly relevant to the outcomes being sought in suicide prevention.  

This means the Outcomes Framework will only report on what is important to measure for suicide prevention. Thereby having greater meaning and usefulness to people impacted by suicide and/or working in suicide prevention.  

AIHW  

Senior research experts 

NSPO  

3. Timeliness

The data measure is going to provide the most current information to users

The assessment considers the timeliness of the data measure, i.e. how long does it take for the data to be reported after being collected, and the frequency at which the data measure is collected, i.e. is the data collected annually or every 5 years.  

It also considers whether subsequent collections are scheduled and when they will happen, to understand when the data will be available in the future. 

This criterion makes sure that data collected for the Outcomes Framework in the future is up-to-date and identifies when the data will be ready for use.  

This means the Outcomes Framework reporting will be able to report on changes over time and be as current as possible to better inform timely decisions about what needs to be improved.

AIHW 

Senior research experts 

4. Accuracy

The data measure exactly describes the construct and population of interest  


The assessment considers how exactly the data will measure the constructs in the Outcomes Framework, as well as how exactly the data provides information on any population of interest.  

It does this to consider how useful, meaningful and reliable the data measure is for interpretation of the Strategy and its progress.  

This also looks at whether a data measure that describes real life experiences can be scaled to provide a national picture without losing its accuracy, as well as its practicality for administration.  

This also considers how well a data measure can be used to cross-validate with other data measures and triangulate results.  

Doing this is important when analysing and interpreting the findings as it increases the strength of the findings in the Outcomes Framework. 

This criterion makes sure the Outcomes Framework uses data measures that are focused on measuring or assisting to measure the specific question being asked in the outcomes and indicators.  

This includes whether the data measures ask the right question, in the right way and of the right people, for example groups that are disproportionately impacted by suicide  

This means the Outcomes Framework reporting will accurately represent what is important to be measured thereby having greater meaning and usefulness to people impacted by suicide or working in suicide prevention.  

It also makes sure that the data measures in the Outcomes Framework can be used in a mixed- / multi-methods approach.

AIHW  

Senior research experts 

NSPO  

5. Coherence

The data measure is consistent and comparable over time 

This assessment considers whether the data measure uses standard constructs, i.e. the same definitions for things that others are using, and whether the data measure is consistent, i.e. always uses these definitions and samples the same types of populations each time.  

This makes sure the data measures are comparable across administrations, i.e. this year’s findings can be compared with next year’s findings and is crucial to being able to integrate qualitative and quantitative data together in a sequential or parallel mixed- / mulit-methods approach.  

This criterion makes sure the Outcomes Framework uses data that measures the same question and the same populations and concepts each time it is conducted, for example groups that are disproportionately impacted by suicide.  

This means the Outcomes Framework reporting is consistent and therefore easier to understand and compare changes over time and supports a collaborative approach to suicide prevention.  

It also makes sure that the data measures in the Outcomes Framework can be used in a mixed- / multi-methods approach.      

AIHW 

Senior research experts 

6. Interpretability

The data measure ensures that reporting on the outcomes makes sense and has meaning to people 

This assessment considers whether the data measure will provide findings that when reported are readily understandable, they have meaning to people and they support the interpretation of the Strategy’s objectives, aims and indicators.  

This considers aspects like, does it make sense (face-validity) to use this data, are the findings easy to understand, and does it provide understanding of what is happening in suicide prevention in an easy and meaningful way. 

This criterion makes sure the Outcomes Framework uses data that makes sense to most people, is reflective of people’s experiences, is easy to understand and increases people’s understanding of what is happening in suicide prevention.  

This means the Outcomes Framework is more accessible and people and stakeholders can more readily use it to improve what they are doing. 

AIHW  

Senior research experts 

NSPO  

7. Accessibility

The data measure is readily available to the Outcomes Framework and its users  

This assessment considers whether the data measure is readily available to the user of the Outcomes Framework, as well as for the Outcomes Framework itself.  

It looks at the pros and cons of using the data measure as a data source, including, is the data available through a standard or bespoke agreement, does the data need to be requested each time, and is the data ready to be used or does it need transformation to ensure analysis and/or cleaning to reduce concerns about the sensitivity of the data.  

These can be used to determine whether the resourcing and cost to access the data outweighs its usefulness to the Outcomes Framework. 

This criterion makes sure the Outcomes Framework uses data that can be effectively acquired from sources, with limited cost, resource or other limitations with getting the data regularly.  

This also considers whether the source data is readily and directly available to users of the framework.  

This means the Outcomes Framework sources its data carefully and makes sure its data sources are publicly available where possible, helping people to understand what is being reported and why.    

AIHW 

Senior research experts 

8. Sensitivity

The data measure can show change in the constructs and populations of interest  

This assessment considers whether the data measure can identify whether a change has occurred or not.  

There are a range of factors that need to be considered to make sure the measure is sensitive enough but not overly sensitive, where changes in the data are considered unstable or inaccurately representing progress. 

These include making sure the data measure can show any change regardless of its size and direction e.g. a small decrease or a large increase, and that any identified change is the result of real differences in what is being measured and is not just noise from other factors or created by breaking things down into small groups.
There is also a need to balance the high sensitivity that can come from a once-off study and the usefulness of a repeated data measure with relatively lower sensitivity.    

This criterion makes sure the Outcomes Framework uses data that can measure the expected change described by the outcomes and indicators.  

This means the Outcomes Framework can identify meaningful changes and support people to judge how important and reliable those changes are while considering relevant contextual factors. This will help to produce trusted and meaningful reports on what is happening in suicide prevention.     

AIHW 

Senior research experts 

9. Importance to the Outcomes Framework​

The data measure is a better ‘fit’ than others and can give a better sense of what is happening in suicide prevention    

This assessment uses a broader perspective to assess how well the data measure will support the Outcomes Framework to achieve its purpose and intent. 

It considers how well the data measure ‘fits’ with what the Outcomes Framework is aiming to achieve, and how well the data measure aids in people’s understanding of what’s happening with the outcomes and indicators and suicide prevention.  

In cases where there may be multiple measures available, this considers which data measure is the best possible ‘fit’.  

Rating 1 through to 5, 1 being least important, 5 being most important.  

This criterion makes sure the Outcomes Framework is informed by the insights of people involved in suicide prevention and reflects what matters most to them and to the goals of the Outcomes Framework.  

It can also help with weighing up which data measure to use if several of them could be used for an indicator and ratings against every other criterion are similar.  

This ensures the number and relevance of data measures included in the Outcomes Framework keeps it accessible and usable and means the Outcomes Framework stays true to its aim and reflects what is important for people to know/understand about suicide prevention in Australia.   		NSPO in consultation with people with a lived and living experience of suicide and sector stakeholders 

Search

Viewing the monitoring data

Caution: Some people may find parts of this content confronting or distressing.

Please carefully consider your needs when reading the following information about suicide and self-harm. If this material raises concerns for you contact Lifeline on 13 11 14, or see other ways you can seek help.

The information included here places an emphasis on data, and as such, can appear to depersonalise the pain and loss behind the statistics. The AIHW acknowledges the individuals, families and communities affected by suicide each year in Australia.

Aboriginal and Torres Strait Islander (First Nations) readers are advised that the National Suicide and Self-harm Monitoring System includes information about the suicide and self-harm of First Nations people.

The AIHW supports the use of the Mindframe guidelines on responsible, accurate and safe suicide and self-harm reporting. Please consider these guidelines when reporting on statistics on the monitoring of suicide and self-harm.

Exit now