Science & Insights

Focusing internal audits to improve performance targets and compliance

The UK National Health Service have concluded that having targets for target sake doesn’t improve patient care i.e. performance outcomes. This happens in organisations where the focus of audit, risk and performance management is on the improvement of a KPI. This also has an adverse effect on staff behaviour with evidence of people playing the system in order to meet targets. 

Using mechanistic targets

If this is the case then before we consider replacing existing KPIs with new ones we need to understand why these have not worked. The biggest factor is mechanistic nature of measurements upon which the KPIs are based. These mechanistic measurements work effectively when the inputs to the process can be controlled from within the process itself. In addition they work best when the process is stable.  For example, a car manufacturer can control the inputs. It can predetermine a series of manufacturing activities and model these in precise detail. It can also agree with suppliers when inputs will be supplied and in what form. 

KPIs are valuable as the conditions upon which they are based does not change from one period to the next.  There is an audit and improvement focused on stripping out variation and non-conformance to achieve better results. In doing so everyone becomes a cog in the machine. Each having no choice but to act in a prescribed way to meet the KPI and target.

The problem with mechanistic KPIs and targets

The problem comes when the inputs cannot be controlled from within the process.  For example, an Accident & Emergency department cannot control its inputs, the patients and their illness.  Hospitals can’t predict precisely who will be injured or when they will attend for treatment. Neither will they know when a disaster such as an explosion will occur.  These cannot be predetermined so neither can each cog in the machine.

If these natural input variations exist then the conditions on which the mechanistic measures are based will be continually changing. This makes mechanistic KPIs and targets logically weak as a vehicle of comparison.  In effect we would be auditing and measuring performance using the wrong approaches. This results in incomplete audit and risk information being provided to management. It also distorts people’s behaviour as they strive to get round a situation and approach that is unreal to them.

Thinking about KPIs and audits differently

In conclusion a hospital isn’t a mechanistic system, its an organic living system. Our A&E department, just like any business process or organisation, is a complex interaction of clinicians and non-clinicians, patients, their family and friends all adapting as one team to a situation they could not predict. 

They, like any organisation or team, are focused on delivering organisational and compliance outcomes, not outputs (that is mechanistic thinking). Sure team member involvement can be pre-determined or assumed – a consultant is a consultant, a nurse a nurse, a patient is a patient. How they behave at the granular level is influenced by the reality of the situation they find themselves in. Their behaviour is governed by the context or situation that presents itself. It is this issue which renders weakness in traditional audit, risk and performance management.

If this is the case, then what can we measure and audit?  What we cannot easily do is measure people’s individual behaviour. What we can more easily audit is the outcome or impact that behaviour has on other people, compliance and performance. It is this outcome based objective evidence that is critical.  Each individual behavioural outcome informs the overall performance criteria or KPI to a greater or lesser degree. Instead of KPI we prefer to call these measures drivers of performance and compliance.

Audit – using lead indicators of risk

This changes what is audited and what is reported against. The implication is quite profound as behaviours are lead indicators of risk, mechanistic audits and measurements are lag indicators.  As this qualitative behavioural data that can now be audited and consistently analysed. This analysis quantifies qualitative data to produce a risk profile against desired drivers of compliance and performance performance. Drivers as results provide forward looking and predictive view. The shift and understanding the reasons for this is fundamental we believe.

Balancing audit and measurement techniques

The audit and performance management art is to recognize the nature of what is to be audited. Which process activities are concerned with managing variety and which variability. The audit technique used is dependent on this criteria. The mechanistic methods are used to question auditees against the planned requirements, do they carry out tasks as planned. The more organic unplanned activities that take place as the reality unfolds given the the context are assessed by auditing behavioural outcomes or impacts, simply because there is no predetermined plan.

The same applies to performance management. Use traditional KPIs and targets where the process activities are mechanistic and based on outputs. Use performance drivers where activities are more organic based on outcomes and where risk is an emergent characteristic.

There is a risk in simply changing all KPIs from mechanistic to organic ones, its a balance. For example, an Accident & Emergency department can’t control its inputs so a more reliance on drivers of performance based on outcomes is needed. However a specialist cancer unit can control its inputs as only those people with particular type of cancer are referred. In this case more mechanistic measures could be deployed complemented by the drivers of performance.

The future of auditing

About the author: Ian Rosam from HPO Risk Solutions is the creator of unique intellectual property to measure risk and business culture to business and compliance management outcomes. A system thinker, author and facilitator supporting the implementation of ERM and Management systems. Ian has worked in many different industry sectors.