By REID A. WODICKA
For one reason or another, some fire departments are better at doing the job than others. Some departments, or perhaps companies within a department, have a special reputation for having the best firefighters. In many cases, members of those organizations are quick to remind us of that fact.
A recent Facebook status of a friend of mine-a member of an East Coast volunteer company-said something similar to, “We’re better than you.” I laughed a little, but this made me wonder: What exactly does this mean? How do we define the performance of fire departments? This article presents the arguement that traditional methods of performance measurement of fire departments insufficiently assess the quality of service provided to the public, and it hopes to encourage an alternate measurement method. I also hope that it can spark some discussion on reevaluating our measurement tools.
As a part of my PhD course work, I studied extensively performance measurement systems in public sector organizations. Public performance measurement can be defined as a tool with which public sector agencies can determine if the operations of the organization are leading toward accomplishing goals. Regarding the fire service, a performance measurement system defined in this manner would attempt to determine if a department is successfully performing the various functions associated with firefighting. This article focuses on fire suppression only.
You might ask, “What about measurement tools like accreditation rating that some departments achieve? Doesn’t that indicate a superior department?” These questions are consistent with an interesting 10-year debate within the academic literature focused on public performance measurement. Accrediting agencies tend to focus heavily on the individual outputs that fire departments’ operations achieve-that is, they determine how well a department conforms to an established set of professional standards it determines is important to fire suppression. For instance, they consider how many engines are in service in relation to the total amount of developed land to which they respond or the distance of structures from a fire station. The magnitude of the outputs of the public policy designed to improve public safety is, thus, the measure by which we determine the quality of public safety service. This is likely inadequate.
This output-based style of performance measurement is consistent with the traditional academic literature on performance measurement. However, just because several engine companies are nearby and they can get to the fire quickly does not necessarily mean that the fire will be contained more quickly or that lives will be saved. Although there may be correlation between those two events (arriving on scene quickly and the fire going out quickly), in many cases, the dynamics of other variables not considered might limit the validity of the existing output measures regarding the assessment of fire departments. Although sometimes fires go out quickly when departments arrive quickly, there may be other factors that lead to good or poor service.
An example is a department that has an ISO Class 1 rating but which tragically endures a heavy loss of fire personnel or tends to burn down valuable structures because of the quality of operational decisions or other factors that might lead to inability to perform. By all output measures, the department should be able to perform, but it does not. Alternately, there may be departments that have a poor ISO rating but achieve a high quality result for its citizens despite failing to meet centrally established output standards. The solution to this is to add additional variables into the assessment of the quality that might account for other issues we theorize will have an impact.
However, adding variables because a theory has been proven fallible is not particularly good science. We need to reassess the theoretical framework with which we measure the performance of fire departments. In the past 10 years, there has been a movement in the academic literature on public performance measurement to suggest that a public agency should be assessed on its historical record of societal impacts or outcomes instead of the outputs of public policy (i.e., the number of fire engines in service). The assessment of outcomes of fire department operations would attempt to show how well the fire department actually performs, regardless of an arbitrary established standard. Thus, when I call the fire department, does it put out the fire and limit the property damage? How does it compare to other departments? These are simple questions, but attempting to define a method becomes a little more complicated.
One way I have accomplished this is by considering generic methods by which I can mathematically calculate the quality of fire suppression service. Although the math is inconsequential, start with the generally accepted mission of the fire service: “to save life and property.” Conceptualize performance by considering the value of the property damaged while controlling for total value of the structure. For life safety, develop a similar measure considering the rescue of victims, with all victims being of equal value, which is the way the fire service approaches rescue victims. To control for a particularly good or a particularly bad year, aggregate this data collection over a 10-year period. Other functions may be measured in a similar light. Once that occurs, an index of public safety could be created to provide a performance score that can be compared with those of other departments.
There are a number of major issues with this generic method. One perceived issue is that it does not consider the value of the property lost prior to the notification of the fire department. This is a reasonable criticism. However, because the intention of the measurement is inherently comparative, assume that every department faces the challenge of delayed notification to some degree. Although I respect that some jurisdictions have more of a hardship with this issue (such as rural areas; there are just fewer people to see the fire), the idea behind outcomes-based measurement systems is to encourage invention and innovation. If a rural locality is having a notification problem, it may be time to develop new fire detection practices and technologies. This type of measurement system provides an opportunity for fire department leaders to assess the outcomes of operations and policies and benchmark their abilities to provide a high level of service compared with other departments. If a department receives a low score, it may be time to change operations. This rationale can be applied to any number of intervening factors not directly addressed within the measurement, not just notification delay. Altering outputs, thus, should lead to a change in outcomes.
It really does not matter how a fire department accomplishes good outcomes for the citizens it is protecting; the only concern is whether or not the fire department is providing a high level of service. Fire departments have been “professionalized” in the past 50 years-i.e., leaders are becoming more highly educated-and we are developing best practices. (This does not refer to career vs. volunteer systems because a paycheck does not necessarily determine professionalism.) Our leaders have gained an increasing ability to create new and innovative methodologies to service delivery that meet the individual characteristics and demands of the unique communities they serve. Although establishing best practices that relate to the policy outputs of the fire department are still useful and their consideration is an imperative step in the overall process, these outputs remain insufficient to provide an educated answer on the question of quality.
From a competitive standpoint, if the fire department in one locality is able to accomplish the same outcome as another department, but with an alternate implementation strategy, who cares? Who cares how the firefighters get the job done as long as the job is getting done? This method recognizes the competence of the leaders in the fire service to understand the needs of their departments and the communities they serve rather than having them rely on an outside agency to determine what amounts to operational decisions.
Further, this inclusion of outcomes-based performance measurements requires a new and more challenging approach to the accountability of public safety systems. As the availability of information has increased and the leadership of the fire service has professionalized, it is no longer adequate to point to various circumstances in the societal environment such as a high fire load or a lack of water supply when making excuses for poor public performance. As firefighters, it is our responsibility to function within the environment we find ourselves and adapt our implementation strategies to meet those challenges. To accomplish this, our leaders should not be tied to centrally established measures that fail to consider the environment in which departments are operating.
I challenge the fire service to take hold of its future and demand that it be assessed based on the merits of the service delivery it accomplishes. This requires thinking. I recognize that there are challenges with the measurement and other components of outcomes-based performance. However, the concept of measuring the performance of fire departments based on what has actually occurred in operations is valid. Though there may be other considerations to take into account when developing an outcomes-based performance measurement system, it is an important conversation to begin as we attempt to continually improve the quality of the provision of fire services. This certainly provides a much more direct method by which we can assess how well a department is performing. Perhaps this will provide an opportunity for my friend who has such confidence in his department’s abilities to make an even more enlightened statement on how his department is “better than yours.”
REID A. WODICKA is a 10-year fire service veteran, mainly as a volunteer with Hose Company #4 in Harrisonburg, Virginia, and as a part-time employee with the Rockingham County (VA) Department of Fire and Rescue. Wodicka has a B.S. degree in public policy and administration and a master of public administration degree with a concentration in local government management from James Madison University. He is a PhD student in public policy at the University of North Carolina at Charlotte and is a former town manager.