«Comparing to Improve, or Simply to Assert? A Case Study of the Application of the Benchmarking Theory within the Public Sector Lii Lindgren Wictor ...»
4.1.9 Key Performance Indicators and Data Collection The workgroups determines and selects their KPIs, by meeting and viewing at the current and prior year’s situation, and further what is of importance the upcoming years. The KPIs chosen are often similar to earlier selections, and overall the workgroups try to preserve the reports composition. Sometimes the CFOs have suggestions and recommendations regarding which KPIs to include, but essentially they only decide the orientations, i.e. the areas to include. In rare cases, politicians suggest indicators they consider relevant for the workgroups to produce.
The usage of public databases to collect KPIs differs between the workgroups. The majority is using public databases where figures are easily retrieved, but some want to produce KPIs that measure specific aspects interesting to them. One interviewee stated "I am in favor of using public KPIs because you get so much back, it is not easy to compare ourselves with each other because we are so different, but it is precisely then that we can assure our quality" (Official, municipality F). This further enables the analyzing part of the process.
The majority of the respondents consider the number of indicators to be redundant, and that the work often loses its purpose. Further resulting in assignments where the participants’ experience that they only measure for the measuring sake, combined with that the reliability sometimes feels low. One group member argues, "it is a measure to measure sort of task, it is important to measure […] so okay, I guess we will measure then" (Official, municipality E). Many experience that the number of indicators must be limited, to further improve clarity and structure. One CFO argued that the KPIs need to have objectives attached, in order to actually measure aspects that provides valuable reflections, which is something that need further development. He further stated that, ”if you don’t measure, you don’t know things, it is like playing a soccer game, it always becomes more interesting if we count the goals” (CFO, municipality E).
Currently, KPIs do not provide much of a base for decision-making. However, some believe that policymakers historically have glanced at these results to find support for priorities and to further allocate resources. The majority, however, argues that most decisions are made without considering the KPIs. Many believe that KPIs could be more centralized if they received more attention, but this is difficult since many other things also impacts.
4.2 Intermediation of Information The communication that occurs is carried out on numerous levels (see Table 2). However, it can be generally concluded for the networks operations that it is lacking on all levels (particularly in area 1). The communication which seems to be most critical for the cooperation is the communication of the project directive, and between and within municipalities.
1. Communication within the municipalities:
Of the project directive and the workgroups tasks, between managers and towards the different operational areas and their respectively workgroups, in order to construct the networks operation
2. Communication between the municipalities:
In order to utilize the attained and compiled data as well as presented best practice examples networks operation
3. Communication towards the politicians:
In terms of, constructing a reporting function of which each municipal presents their performance level networks operation Table 2. Different Levels of Communication Concerning the project directive, some municipalities communicate it more frequently, while some hardly are aware of its existence. Some CFOs distribute the directive to the convener for each workgroup, but this is not made consistently from all parts. It is stated that, ”It is intended, but we find that it falters sometimes” (CFO, municipality D). It is further requested that the project directive would be communicated more, since it otherwise cannot be followed, and in terms loses its purpose. One CFO stressed that it is not enough to just write it and expect everyone to understand its meaning, rather it need to be further deliberated. One official argued that it is so many different levels that need to function and that there is a need to receive extensive information in order to reach valuable results.
The respondents experience that other municipalities are open with sharing information, and a willingness to contact others in order to exchange information is prevalent. One respondent labels it ”stolen with pride” (CFO, municipality B); meaning that municipalities should use what has already been invented. The project directive states that the municipalities should take advantage of good examples presented, which further is expressed as being “the point of it” (Official, municipality F). An official argues that “they often feel very alone”, and therefore many of them appreciate the occasions to interact with other municipalities (Official, municipality C). These opportunities are created through the network, and participants are given a chance to expand their field of contacts.
4.3 Evaluation and Implementation 4.3.1 Utility and Value All respondents were polled if they perceive that the benefits of the cooperation correspond to the time dedicated to it, where the answers received were scattered. Some believed that politicians and higher authorities consider the benefits to correspond to the work amount invested. Thus, other argued that due to the amount of work enjoined to attain the data, together with the time required for meetings with the workgroups to discuss the material and prepare for the conference, it can be difficult to realize the true advantages. However, a few interviewees pointed out that as a result from the ‘ghost-writer’ (which many municipalities use today) the benefits start to lay more on the same level as the work rendered.
A regular apprehension is that the greatest value arises during the year within the interactions in the workgroups, where officials receive opportunities to meet with other officials working in the same area. This implies that a substantial part of the usefulness prevails during the cultivation of material up to the conference day. One CFO reasoned that the utility further is impacted by the engagement from the participants. This interviewee also stated that the longer the workgroups have been in operation, the more the engagement increased. This relates to that in the beginning, significant time is directed toward selection of KPIs, and to make sure that you do not compare apples and pears. Although, as further argued, ”It is easy to get stuck in a rut […] it feels like you need to shake it up and make changes” (CFO, municipality D).
However, some interviewees stated that the usefulness varies, due to the municipalities’ divergent sizes and preconditions. To make comparisons are valuable because lessons to be learned can be brought to surface, and as emphasized by many respondents, they don’t have to reinvent the wheel themselves. The opinions regarding how much attainments the cooperation brings about, are diverse. One interviewee exemplified this as; if a municipality has large deviations, there is often an explanation for this, which is a useful insight. Further one interviewee argued that; "It is first when you do something concrete with the matter that gains can be made. It is vital to do something active and to alter" (Official, municipality B).
It was reasoned if the network has contributed to more effective organizations. Two CFOs stated that this is difficult to pinpoint (but they wish the answer to be yes), since matters always occur and it is rather elusive to distinguish exactly what leaves impacts. In general, the organizations are under pressure from stakeholders to ameliorate, meaning that pressure to streamline partly lies in the economic climate. One official argued that the effectiveness varies depending on operational area, and often "You get fancy reports but they do not contain that much of tributary meaning" (Official, municipality F). Although, it was further argued "it becomes somewhat of a competition, which is good as it stimulates improvements, since nobody wants to end up in the bottom" (CFO, municipality A).
4.3.2 Secondary Analysis The last part of the benchmarking cycle is something that each municipality is expected to do on own initiatives. The time and effort that actually is being devoted to this varies both between municipalities and respectively operations. It was argued that, "the analysis and practical application of the conclusions that are drawn from these kinds of comparisons are the things that are the most interesting" (CFO, municipality A). Another CFO stated that they sometimes compile the key findings, which the administrative managers thereafter should communicate to their operations. Others stated that certain findings are used in their internal functioning, e.g. in budgetary processes. A CFO vindicated that, although the workgroups and administrative management uses this in their daily running, more could be made of the analysis, considering the time devoted to attaining the material.
A commonplace conception among the CFOs was that analysis taking place after the conference day surely does exists. However, it is argued that ”It is little, it is not recognized as official agenda items, it happens, but it is more common that it do not happen […] in that regard it always feels like it is short of time (CFO, municipality D). Some interviewees argue that this is a question difficult to pinpoint, since the workgroups simply ceases to exist after the conference.
Further, a third respondent stated that "we have this as a checklist which we follow; the report is done, we have participated, good, next" (Official, municipality C).
The majority of the interviewees agree that the reasons behind the lacking analysis and evaluation is too a large extent related to time and resources. The time is limited (partly due to that the network is situated outside everyday responsibilities), meaning that the possibilities of going deeper into a manner is absent, resulting in that explanatory factors come secondarily.
Some also indicated the benefits of dedicating less time to data acquisition, as well as cutting down on the amount of KPIs, and in that regard enable more time for analyzing and evaluating.
As part of the deficiency of analysis is also the question of involvement and competence regarding engaged individuals, which some interviewees attach importance to. It is argued that, ”One can fairly quickly attain the KPIs […] then it can be difficult to understand, intellectually, what explains the differences (CFO, municipality D). Another interviewee strongly argued that;
The same interviewee further stresses the significance of quality assurance of KPIs, something that is first noticed when they sit down and analyze them. Two officials emphasized the significance of always taking a critical approach, and questioning what really is seen, which is essential for the analyzing part.
4.3.3 Change Implementations Divergent opinions prevailed considering change implementations, where some interviewees claimed that no changes had occurred resulting from the network, whilst others had concrete examples of this. This is further a question of time and resources, as neatly put by one CFO;
"The economy always sets the limits" (CFO, municipality C). Change implementations are conducted when examples of best practice are current. Thus, each municipality is solely responsible to make use of these, which includes contacting other municipalities, if needed. One interviewee also emphasized the importance of having a plan for change implementation, since changes as a goal takes on a long-term perspective.
4.3.4 Deficiencies The respondents expressed multiple difficulties regarding the undertaken work. The overall identified deficiencies that appear to exist in the network are presented in in Table 3.
Table 3. Deficiencies
4.4 Document Analysis of the Project Directive of Network Södertörn The established project directive describes the structure of the intended work procedure. It is stated that workgroups consisting of officials from different operational areas should produce and interpret KPIs. The undertaken work is to be compiled in a report, and further presented during a conference day. The directive stresses the importance of producing KPIs with high quality and reliability, and that these are adjusted to suit each operational area.
The aim of producing and reporting KPIs is to,
Some other purposes of importance are stated in addition to the main aim, which are presented in Table 4.
Politicians and officials with responsibilities for the different operational levels should analyze and explain (intentional or unintentional) similarities and differences in serviceand cost relations within the different operational areas.
Participating officials should provide opportunities for collegiate exchange of experiences regarding implementation within the different operational areas.
Provide attention for enhancements and efficiency improvements that has been achieved by someone of the other participating municipalities, in order to enable an earlier deployment of these solutions, to achieve a more adjusted operational development within the comprising municipalities.
Highlight the usefulness of invested resources.
Table 4. Complementary Aims The document stresses that the content in large should be based on qualitative KPIs, even though some economic KPIs also must exist.
The workgroups should also focus on public statistics from national databases when producing the figures, in order to aim emphasis on analyzing and making conclusions. The workgroups also need to work with quality assurance of the figures.
The project directive describes the networks organization, and further which role the different levels possess. These assignments are presented in Appendix VII.
The project directive is finalized with a time schedule, which presents when different tasks should be carried out and completed. This can be viewed below in Table 5 (note that this is the specific time schedule for the work undertaken during the year of 2013).
Table 5. Time Schedule for the Undertaken Work