Thermodynamics might not seem to have much to do with healthcare – but one of the best reasons to monitor healthcare standards is the second law of thermodynamics. That is, left unchecked any system will deteriorate and become more disordered.
Ask any parent, gardener, or mechanic. If the child, the garden, or the motor are left unattended, injury, decline, and breakdown are inevitable. Regular effort is required to protect and sustain them. The same is true in healthcare. If progress was inevitable there would be no need for surveillance.
The ideal performance measure is evidence-based, promotes best-practice, is informed by reliable data, based on sound (clinical and statistical) methodology, casemix adjusted, clinically validated, universally applicable, and reported in a simple and intuitive format.
Current performance measures only fulfil some of these criteria.
There are at least four types of performance measure - activity, resource, process, and outcome. The distinction is pragmatic. Activity-based indicators quantify healthcare activity, such as the size of the surgical waiting list. Resource indicators measure events that primarily reflect the resources available such as Emergency Department wait-time. They measure quantity rather than quality of care.
Process-based indicators measure healthcare compliance rates, for example hand hygiene. They focus on the process rather than the patient. Activity, resource, and process indicators constitute a large proportion of routinely reported performance measures, simply because these data are more readily available.
Outcome-based indicators focus on patient-related outcomes. Examples include adverse events, standardised mortality, patient satisfaction, and similar measures. They focus on the patient rather than the process; quality rather than quantity of care; the results rather than the route taken to get there. These indicators are highly desirable but more difficult to design and construct reliably and, therefore, less common in healthcare reports.
How do I interpret healthcare performance reports? First, classify each performance measure (indicator) into one of the above four categories. Focus chiefly on outcome-based indicators; they provide the most insight. If the results are unexpected or disappointing, ask: How was the data defined? How was the metric constructed? Over 50% of ‘unexpected’ results are due to data or methodological errors.
Next, review process indicators for ‘signals’ that may highlight areas of concern. And finally, review activity and resource indicators - resource deficiency is a common cause of outcome and process variations.
Be wary of indicators based on small sample size (n<30), infrequent events, no (statistical) confidence intervals or benchmark for comparison. They may produce misleading results.
Performance measures are screening tools, not diagnostic tests. They do not provide answers. They do, however, help direct our attention and (limited) resources to those areas that may warrant further investigation.
Even if all looks ‘good’, don’t be complacent - there is always room for improvement. According to the first law of thermodynamics, a (health care) system will not improve unless energy and effort is added.
Dr Graeme Duke has joined VAHI as Data Analytic Fellow, in addition to his roles as Deputy Director, Eastern Health Intensive Care Services in Melbourne, and clinical lead for intensive care research. To get in touch with Graeme about how we can collaborate to use data more effectively, contact [email protected]