Society is constantly measuring how we can make best use of public money. However, the constant evaluations of daycare institutions, hospitals or upper secondary schools rarely take the employees into consideration. How do childcare workers, nurses or teachers actually perceive the evaluations of their efforts? And how does this perception affect the way in which we can use these performance indicators?
A group of researchers from Aarhus BSS asked themselves these questions, and in a new study, they found an answer to the first question. The answer may seem trivial, but has a significant effect on the second question, namely the potential of measuring performance in the public sector.
The results of the study have been published in the article ’Acceptance or Disapproval: Performance Information in the Eyes of Public Frontline Employees’. In the study, more than 1,700 Danish upper secondary school teachers answered a questionnaire that, for half of the respondents, included information on how their workplace had scored on the “value-added indicator”. This indicator aims to measure the ability of an upper secondary school to improve the academic performance of students compared with other upper secondary schools. Afterwards, the researchers compared the teachers who had received information about their school’s position on the indicator with the teachers who had not. The decision on who was to receive information was made by drawing lots and was thus completely random.
It turned out that if the workplace performed well, the information made the teachers more positive towards the performance indicator in terms of its validity (is the value-added indicator a valid indicator?), its legitimacy (is it fair to be evaluated on the value-added indicator?) and its applicability (is the value-added indicator a useful indicator?). However, if their workplace performed poorly, the information made the teachers more negative towards the performance indicator.
"Our study shows that those with the lowest score find it hardest to accept the indicator. This might sound quite trivial, but if you dig a big deeper, the implications are far-reaching,” says Associate Professor Morten Jakobsen from the Department of Political Science, Aarhus BSS. He is responsible for the study together with his colleagues PhD student Niels Bjørn G. Petersen from the same department and Trine V. Laumann, a former student at Aarhus BSS who is currently an administrative officer in the Danish Ministry of Culture.
One of the purposes of generating large amounts of performance information from public sector institutions such as upper secondary schools is to learn from this information. What are we doing well and in which areas do we need to improve? However, when those with the lowest scores, and thus with the greatest need for improvement, refuse to acknowledge the performance indicators, the opportunity for learning is likely to be rather limited. That is, in cases where the need for learning is actually the greatest.
“Managers may launch as many initiatives as they like on the basis of the performance indicators, but the teachers are the ones who need to change their way of doing things; be it their behaviour in the classroom, the curriculum or something completely different. And if they think that the indicators are rubbish and thus refuse to accept their premise, well, that makes it more difficult to use the indicators to foster learning,” says Niels Bjørn G. Petersen.
Until now it has been assumed that due to their professional background, employees, in this case upper secondary school teachers, would not be influenced by how well their workplace scored on the performance indicators in their assessment of these indicators. The belief has been that the professional norms and values of employees determine whether or not employees accept the indicators. However, the study suggests that another mechanism is in play.
“The reason might be that people with a professional background are more likely to equate their identity with their work. Thus it hits them harder when their organisation doesn’t perform as well as expected,” says Petersen.
The fact is that unconscious psychological mechanisms come into play when employees are faced with indicators of how their organisation - and thus how they – are performing. Everyone has a need for appearing competent (also) to themselves, and people with specific professional backgrounds, such as an upper secondary school teachers, will have a particular need for appearing competent within their field. When faced with information that does not fulfil their basic need for appearing competent, they will unconsciously try to neutralise that information. They might try to convince themselves that the information on which they have been measured is inappropriate, illegitimate and inapplicable - or they will try to blame the result on others.
“We have identified these psychological barriers in upper secondary school teachers, but this so-called ‘defensive bias’ is present in everyone. That’s why everyone who has to use the performance measurements results will always be faced with this challenge - and will thus have to be able to handle it in order to make best use of the information gathered,” says Morten Jakobsen.
It is important that politicians and managers who are required to apply the measurements in relation to their employees understand the limitations of the measurements and be very aware of their purpose.
The aim of showing that a certain organisation performs better than others - e.g that a particular upper secondary school scores better on the value-added indicator - is to let citizens and politicians know how things are going. However, if the performance indicator is meant to contribute to learning and development in a specific institution, people’s ‘defensive bias’ means that you need another approach. The manager might have to refrain completely from using performance indicators as a basis for changing the organisation as the employees might reject the new initiatives simply because they do not accept the premise. Instead, the manager could use the performance indicators purely as background knowledge without sharing them with the employees.
“We don’t have a clear answer for how you should present and work with the performance indicators in order for your employees to accept them. This will vary from organisation to organisation. However, it’s something that you need to take seriously and accept,” the researchers stress. They offer the following suggestions for how future studies could contribute to finding a solution to how public sector managers could transform the performance information into employee learning:
Research shows that if people are presented with a result showing that they performed the worst, their perception will be different if they have been presented with good results immediately before. In other words, people are more likely to accept a poor result if their self-confidence has been boosted shortly before.
Another possibility is to let the employees select the performance indicators on which they will be measured. In this way, they might be more likely to have a positive perception of the indicators. However, along with employee involvement new challenges will arise: The employees might choose performance indicators that they know they will live up to. In addition, each organisation will have their own set of indicators, which makes it impossible to compare organisations such as hospitals and upper secondary schools. Thus, organisations will be unable to look to each other for best practices.
Finally, future studies might explore whether the psychological barriers are as common in managers as in employees without managerial responsibilities? If so, it means that the challenge is found not only on employee level, but also on managerial level. This makes it even more difficult to overcome the barriers.
"We love to hate these performance measurements, and although not everything can be measured, the alternative of not measuring anything would have grave consequences."
Niels Bjørn G. Petersen, Ph.d.-student, Department of Political Science
Many questions prevail, but neither the unanswered questions nor the recently discovered limitations of working with performance indicators mean that we should simply discard the performance information and stop measuring organisations in the public sector.
First of all, it might be that some of the negative information does actually come to good use. In other words, we do not know whether the performance indicator is rejected by all upper secondary school teachers who work in schools that score low on the value-added indicator - we have simply idenfied a trend.
Second of all, the performance indicators may be useful to others apart from the employees. For example, information on the value-added indicator may be useful for people deciding which school to apply to.
Finally, it is difficult to justify spending so much money on a large public sector such as the Danish if no one knows how things are actually going.
“We love to hate these performance measurements, and although not everything can be measured, the alternative of not measuring anything would have grave consequences. Just imagine if you would never find out that a primary school or upper secondary school simply wasn’t up to par,” says Petersen.
Morten Jakobsen adds:
“Instead you need to consider the purpose and be aware of the limitations. If you have a sense of the psychological barriers, you also have a better sense of how the performance indicators are received and applied in the individual organisations.”
Performance measurements in the public sector
In the public sector, the concept of performance measurements or performance indicators can be used for various purposes. They can be used to measure the citizens’ level of satisfaction with public institutions such as daycare institutions or schools. Or they can be used to measure public sector performance on other parameters: Hospitals might be measured on how many patients with a particular illness they succeed in curing, how much money they spend on specific treatment, or how many of a specific type of treatment they carry out. For upper secondary schools, performance indicators may be grades or the so-called “value-added indicator”. The latter is calculated by the Danish Ministry of Education and reflects the ability of the upper secondary schools to improve the academic performance of the students taking into account their academic and sociaeconomic background. This means that you account for the differences between the upper secondary schools that are beyond the school’s control, and that you only measure the effect of the differences that the schools are able to create on their own. This means that an upper secondary school may score high on the value-added indicator - and thus be a success - despite having a low average grade that simply reflects what kind of students are admitted to the school. In contrast, the value-added indicator reflects the extent to which the school elevates the students academically throughout their time at upper secondary school. |
|---|
Source:
Journal of Public Administration Research And Theory, 2018, 1–17
Read the article here (Payment wall may occur)