Whenever I present a course on police performance, I begin by posing the question ‘what IS performance?’
To get the discussion going, I offer the following definition:
Performance is about maintaining good figures, meeting numerical targets, staying in the green and avoiding being in the red.
One or two in the audience might look momentarily troubled: have we paid good money to attend a course where this is the message? But most give a wry smile. Sometimes a nervous smile. They get the joke. They recognise this caricature of a widespread organisational approach to performance management.
More recently, the most common response to this obviously satirical (at least I hope it’s obvious!) definition has been ‘yes, we used to do that, but we’ve moved on’. Which is encouraging.
But it’s still a concern. Only last week I was talking to a senior manager, discussing the launch of her force’s new performance framework. ‘I’m concerned that it might degenerate into a red-green exercise,’ she said.
This illustrates something which is often neglected: the importance of careful definition, of knowing exactly what we mean by the words and phrases we use. Especially the common words, which can be used unthinkingly.
It matters. The government’s National Crime and Policing Measures have been introduced with the explicit intention to ‘allow performance to be measured.’ This sounds uncontentious. After all, don’t we routinely talk about ‘performance measurement’ and ‘performance measures’?
But the whole idea of performance measurement is rather troubling. It embodies an idea of performance as something which can be measured, like height, weight, temperature, time. All you need is the proper measurement scale and hey presto, problem solved. But police performance isn’t like that.
The problem with performance indicators (in line with the current jargon, perhaps we should call them metrics; but they’re the same thing) is the same as the problem with any statistic. They don’t speak for themselves. They need to be carefully interpreted. Anybody who has followed Professor Sir David Spiegelhalter’s commentaries on the statistics relating to Covid-19 will appreciate how complex and nuanced this task can be.
To qualify as a police performance indicator, a statistic should be potentially influenced by what the police do, legitimately, in pursuit of some recognised objective.
But that statistic will also be influenced by a whole range of other environmental factors, many of which have nothing to do with what the police do. The weather, for example. Or Covid-19. It may also be influenced by what other agencies do, independently or in collaboration with the police. To say nothing of creative manipulation or ‘housekeeping’ aimed at making the figures look better (‘of course, we don’t do that, but we’ve heard dark rumours that other people do’; there is usually some leeway in the definitions of an indicator). And then there are the multitude of tiny influences which may render a rise or fall in a statistic effectively random, and so meaningless.
At the risk of being boring, I’ll repeat the message: careful definition is not a luxury, it’s essential. So much of importance flows from it.
We might ask, ‘has performance improved?’ To answer the question, we look at the statistics. But statistics don’t improve (except in the technical sense of being refined, made more reliable). They rise or fall. Simple. And – here is the crucial point – an increase (or decrease) in a statistic is not the same as an improvement, even if the statistic moves in the desired direction. An increase MAY reflect an improvement. But it may also reflect some combination of the other factors I have just outlined.
Call me a pedant if you will, but we need to stop confusing the words ‘improve’ and ‘increase’.
I propose another terminological distinction, one which I think we need to embrace, to make it part of our everyday language, part of our performance culture (whatever that means).
We should forthwith stop using the word ‘performance’ on its own and get into the habit of specifying either apparent performance or actual performance.
Apparent performance is the statistics (or performance indicators, or metrics, whatever you want to call them). It is the evidence, by nature quantitative. It can rise or fall, increase or decrease. Or stay the same. But it is only ever the evidence, and ultimately not what we want to know.
Actual performance is how well we do the job. It is concerned with what we do, how we do it, and whether it achieves what we want to achieve. It should take account of the resources we use, and thereby allow us to make judgements of efficiency as well as effectiveness. And the key word here is judgement: actual performance is essentially a qualitative judgement, made on the basis of the evidence. And it is very much the thing we want to know.
There are distinct practical consequences to embracing this definition. Here are three simple examples.
The first arises as a simple consequence of calling things what they really are. A few years ago, I was asked by a police force to review its arrangements for reporting performance. At my request, they sent me a copy of a recent quarterly force performance report. It was a series of PowerPoint slides, all but one of which consisted of statistical tables, charts, and bullet points summarising statistics, with liberal use of red and green highlighting, but no interpretative text. The only slide which didn’t contain any statistics was the title slide: it said, ‘Monthly Performance Report’.
If I had my way, that title slide would include one extra word. It would say, ‘Monthly Apparent Performance Report’. The effect of this would be simple and profound. It would demand an answer to the question: ‘If this is apparent performance, then what’s really going on?’ And that’s when the questions, discussion and interpretation would have to begin. Without the prompt of that extra word, it’s so much easier just to take things at face value: it’s what it says it is, performance. But it isn’t. It’s only statistics.
The second benefit that follows from this distinction is the resolution of a problem which has been nagging away in the subject of performance management for about thirty years. Targets.
There has long been a widespread and understandable mistrust of targets in policing (and public services in general). At times it seemed that every performance indicator had to have a target attached to it; they often seem arbitrary and unscientific; and perhaps most seriously, they distort behaviour, encouraging ‘gaming’, focusing solely on the achieving of the number.
But there is nothing intrinsically wrong with targets. They can be useful. It can be argued that they have saved lives. Look at the recent vaccination campaign against Covid-19.
Go back further in recent history and look at the spectacular reduction in people killed and seriously injured (KSIs) on UK roads around the turn of the century. Following two demanding, long-term, multi-agency targets set by the Department of Transport in 1987 and 1999 respectively, deaths alone fell from over 5,000 in 1987 to under 2,000 by 2010. Such a dramatic fall was the result of a range of different factors. But it is no coincidence that the steepest decline in KSIs since the war corresponds to the setting of targets. And KSIs is one of the less manipulable, less gameable statistics.
But targets can only be beneficial if you recognise one simple truth: they are set on performance indicators and are therefore apparent performance only. And so, meeting a target is only ever apparent success and failing to meet it is only apparent failure. Reds and greens should be avoided; the focus should be on interpretation.
The third and final benefit (although there are many more) to embracing the distinction between apparent and actual performance is that it clarifies the role of analysis in performance management.
If apparent performance is what we know, and actual performance is what we want to know, how do we get from one to the other? The answer is, via analysis. In principle it is very simple: consider the competing interpretations of the evidence – what that rise or fall (for example) in an indicator could mean – and then use additional evidence and reasoning to narrow down the interpretations and make a judgement of whether actual performance has improved.
Of course, it is more complicated in practice, and often the answer will have to be a frank ‘we don’t know’. But it puts analysis at the heart of performance management. Analysts should not be mere ‘number crunchers’, producing apparent performance reports. They should be active, creative and rigorous in pursuit of a better understanding of actual performance.
These issues and their implications are explored in Malcolm Hibberd’s Performance Masterclasses.