Improving by Evaluating

By Thomas N. Warren

One of the many things any fire department can do to improve their fireground operations is to evaluate the effectiveness of their response to a fire. One of the quickest ways to pick up on the strengths and weaknesses of a department is to take a candid, grass-roots level look at a fire operation after it occurs.

When I was a young lieutenant eager to take command of my first fire company, ready to take on any fire in my district, I soon found out that I still had much to learn. I realized quickly that we all had much to learn about our responsibilities to the department’s mission and to the firefighters under our command. It soon became apparent that as I was mastering the craft of being a fire lieutenant there were also new captains and battalion chiefs learning to master the art of their respective crafts. The fire service is an organization in a constant state of flux where we are all learning all the time.

New firefighters are always joining the fire service and others are being promoted to higher ranks as firefighters retire or move on to other careers. These are normal, healthy transitions for our organizations. Nothing brings more enthusiasm and renewed passion for our job than a new group of probies in the firehouse or a promotion to the next rank.

As we move through our careers, learning takes on a different significance than it did before we were promoted. Our concerns become more global in nature and less centered around mechanical aptitude and personal stamina. We begin to focus on the concept of organizational goals and operational priorities. Routine evolutions that we performed begin to come into focus as part of a larger operational plan that fits into an overall fireground operation. The roof ventilation we did as laddermen was more that cutting a hole that would make our captain proud; it was a tactical operation that allowed the engine company to move a line into position to extinguish a fire and search for victims. We now see and evaluate that ventilation by how effective it was in allowing heat, smoke, and fire gasses to be released from the building.

With this informal education process comes the formal education process of earning college degrees. Every firefighter should pursue this avenue regardless of rank. A veteran fire officer will learn from many sources and develop a more comprehensive picture of fire service operations, operationally and administratively. Through this growing process, a veteran fire officer will learn the value of evaluating one’s self and the organization in which he operates.

The “after-action” critique is probably the best known tool used to evaluate the any fireground operation. I was first introduced to this concept through a new battalion chief who was also one of the youngest battalion chiefs in our department. I was surprised to receive notice to report to a neighboring fire station at the beginning of our shift. The chief wanted to discuss a fire we had the night before. I was a little nervous about the prospect of sitting down with the battalion chief to review our efforts at this fire, but I gathered my crew and we drove to the neighboring fire station.

On our way to the station, I heard other fire companies go “on the air” to the same fire station. It soon became obvious what we were in for. I was familiar with the concept of critiquing a fire, but it was never done in our department, and I had a sinking feeling that it would not be well received.

I was reminded of a quote from Field Marshall Sir William Slim, a World War II British officer who once commented that “military officers have to make vital decisions on incomplete information in a matter of seconds, and afterwards the experts can sit down at leisure, with all the facts before them, and argue about what might, could or should have been done.” We all engage in this type of Monday morning quarterbacking, and it often leads to hard feelings and nothing being learned from our shared experience.

Reluctantly, we all went to the sitting room at the fire station, had some coffee, and waited for the battalion chief to arrive. He arrived on time as expected with pastries from our favorite bakery and immediately began to set all the firefighters at ease. The battalion chief knew this was a new experience for all of us and there was a strong possibility that we would become very defensive of our actions. He explained that he understood that this was a new concept and that he intended to do this with some regularity—not necessarily after each fire—but that he wanted to introduce us to this concept with this fire because, in his words, “we made a really good stop.”

He let us know that there would be a systematic approach to these meetings and that he would model it after the U.S Fire Administration’s (USFA’s) Special Report: The After-Action Critique: Training Through Lessons Learned. The first rule was that no one would be allowed to criticize anyone else or any fire company; we would simply explore the course of the fire and compare our actions to our department standard operating procedures (SOPs). With the guidelines established, we proceeded by allowing every company officer to describe his actions and how those actions fit into the overall fireground operations.

The process went surprising well. Every company officer was eager to explain what he did and how it fit into the overall fireground operation and measure the actions against our SOPs. Company officers began to think about their fireground operations beyond being a series of task assignments and more like a component of an effective fireground operation.

The interconnectivity of our individual actions soon became apparent. Our first after-action critique lasted about 30 minutes, and we all left with a sense accomplishment; first, for our operations the previous night and second, and more importantly, for participating in a process designed to help us improve. We essentially learned a new way to learn. We learned that we needed to identify and measure our operations before we could improve them. We learned that as comprehensive as our SOPS were, they required revisions and updating. We learned that other departments might do things better than we did and we could learn from them. We learned of what ours and our fellow firefighters’ strengths and weaknesses. We also learned how to improve our organization.

We had many more after-action critiques in the following years. Slowly, the other battalion chiefs began to follow suit. The program gained support because it was a learning technique employed by the Federal Emergency Management Agency, the U.S. military, and the USFA. Our program was designed to identify operational trends that need to be modified, uncover potential safety issues, evaluate our SOPs against new challenges, develop new training programs based on critical lessons learned, and to recognize good decision making and operational successes.

Our department won several grants and began to host multijurisdictional hazmat, mass casualty, and terrorism drills funded by several federal agencies. These drills brought in many local fire departments as well as state and federal agencies. One requirement to hosting these drills was to conduct a “hot wash” following the drill. The concept was to evaluate what went right, what went wrong, and what could be improved. The sponsoring agency served as the facilitator for these hot washes, trying to bring out conversation relating to our performance at the drill. The members of our department had learned through the previous years’ after-action critiques the value of this exercise and how to make it work to everyone’s benefit.

Our members quickly began the discussion and coaxed some of the more reluctant exercise participants to engage in the discussion. We were teaching others what we had learned years earlier—evaluation conducted in a respectful manner is a valuable learning tool. After the hot wash, we would often go out for lunch with some of the other drill participants and talk about the day’s activities. For most of the departments, this was their first formal review of any kind in which they participated professionally. They simply did not do these kinds of things in their own departments, and they were more than a little bit anxious about engaging in the conversation. As the discussion went on regarding the drill, they indicated they began to feel a little more comfortable with the process and found themselves engaged in the dialogue; they even felt that they benefitted from it.

I saw in them the young lieutenant I was when I first participated in one of these after-action critiques many years ago: a little bit apprehensive about this new learning tool, but willing to try it. When one takes an open and honest look at his performance and measures it against established standards, it opens the door to improvement, personally and for the organization as a whole. We all are the beneficiaries when that happens.


Thomas N. Warren has more than 40 years of experience in the fire service in both career and volunteer departments. He retired as assistant chief of department of the Providence (RI) Fire Department after 33 years of service. Presently he is a faculty member at Bristol Community College in the Fire Science Technology Program teaching a variety of subjects in the fire science discipline. He holds a Bachelor’s Degree in fire science from Providence College, an Associate’s Degree in business administration from the Community College of Rhode Island and a Certificate in Occupational Safety and Health from Roger Williams University.


More Thomas N. Warren



No posts to display