How Good Are You?

How Good Are You?

MANAGEMENT

To say for sure, you need good evaluation procedures.

You know you’re one of the best. The fact that you’re a firefighter is proof in itself of just how good you are. You have the scars and memories of the fires that you killed before they killed someone. You also have the memories of the tears when the fire won and someone died.

In addition to your own memories, when you look around the kitchen table as an officer, you think of the other officers and firefighters of your unit and you know positively that they’re the best. Many times, all of you have said that if only you were first-due rather than third-due, the fire would have been killed before it killed or went to a greater alarm. Those other guys couldn’t carry your boots.

But if I walked into your quarters and told you I didn’t believe your war stories proved anything to an outsider, would you have any means to dispel my doubts? I hope to give you a tool that will allow you and your firefighters to prove you have the basic individual skills necessary to be a safe, proficient crew. This tool is called the evaluation procedure.

Good evaluation procedures help you to determine if the people you’ve been teaching have truly learned. Just as important, though, a good evaluation procedure determines how well you have taught. In addition, evaluation procedures can be put together into a package that measures individual skills, unit skills, multiunit skills, and finally, command skills. One just naturally leads to another.

This tool will make you be more painstaking as an instructor because, if you don’t do well in teaching the material, the trainees can’t do well in the evaluations. Effective evaluations will make you set realistic objectives for the trainees to meet. They’ll help you become more innovative to involve more trainees in learning.

The trainee is going to know what you expect because you will have stated this when you explained the objective. Your student will know the evolution can be done, because you will have demonstrated it and the trainee will have practiced the procedure, as will others in the group. This instills confidence in you as the leader and in the other members of the crew as partners.

Before we look at how you go about designing a good evaluation system, a few words of caution are necessary. You can design socalled evaluations that are all fluff and bluff—”Well, did this firefighter get water on the line?” “Yes, but he took five minutes to do it,” or “Yes, that firefighter did get the SCBA on … in three minutes and with no facepiece seal.” We know that drill only too well.

Just as bad is the nitpicker approach. If you’re a nitpicker, I ask that you kindly stop reading this article. This system is not for you. If you continue reading and try to use it, you’ll end up destroying your crew members’ confidence in themselves and their respect for you as a trainer. You’ll mark yourself as someone who’s interested in exact form, not practical fireground results.

I classify as a nitpicker the person who requires the trainee to tie a six-foot loop in a line, and who, when the trainee easily completes the tie, whips out a tape measure and says, “Aha! 6’2″! Don’t you have any sense of judgment?” Another classic is when the trainee is required to carry out certain functions in the cab of a pumper and does so flawlessly. Instead of praise, the firefighter gets “How come you used your right hand instead of your left?” for the next 20 minutes.

Figure 1. Detailed Evaluation Procedure

1. Check the cylinder valve assembly.
2. Check that gauge pressure indicates a full cylinder.
3. Put safety lock in position.
4. Open valve fully and listen for the audible alarm. If there is none, don’t use the unit.

1. Don the unit.
2. Place your hands on the back plate, valve assembly pointed toward your head. Hands are to be just above the waist straps.
3. Lift the unit over your head while keeping your elbows close to the body and bending forward.
4. Allow the harness to slide over the points of the elbows.
5. Lower the unit onto your back and remain bent forward.
6. Grasp the shoulder straps and slide your hands down the straps until you pick up the regulator and chest strap.
7. Clip the chest strap into position. Remain bending forward.
8. Take hold of the adjusting strap tabs. Pull downward and toward the back until tight.
9. h) Buckle the waist belt and snug it up.

1. Don the facepiece.
2. Grasp the facepiece at the bottom and place your chin into the chin pocket.
3. While holding the facepiece in place, slide your other hand under the webbing and bring it back over your head.
4. Stroke the harness down.
5. Bend your head slightly forward.
6. Starting with the lowest strap left and right, draw them snug simultaneously. Draw straight back, not away from your head; that could break a strap.
7. Repeat with the temple straps.
8. g) If necessary, repeat with the head strap.

1. Check the seal.
2. Start inhalation.
3. Place your hand over the breathing tube connection to seal it. The facepiece should draw in against your face, indicating a good seal. This is a prime safety check.
4. While continuing to block the seal, exhale sharply to clear and check the exhalation valve.

1. Put the regulator into operation.
2. Break the seal on the breathing tube, but continue to hold the tube.
3. Inhale and couple the breathing tube to the regulator.
4. Snug up the fitting.
5. Open the yellow mainline regulator valve fully.
6. After the unit operates, crack open the red emergency bypass valve. This is a prime safety check. Then unlock and close the yellow mainline regulator valve.
7. If the emergency bypass valve operates, reopen the yellow mainline valve and lock ring fully.
8. Close the red emergency bypass valve.
9. Your unit is now fully operational.

With those warnings, on to the job of properly designing your evaluation procedure. The first step is to state the job we want to accomplish. It can be very simple: “Don a self-contained breathing apparatus according to established procedure.”

That was a piece of cake, so we’ll have to work a bit harder from here on out. Let’s break this job into major tasks. In the Buffalo (N.Y.) Fire Department’s system, it’s worked out like this:

1. Check the cylinder valve assembly.
2. Don the unit.
3. Don the facepiece.
4. Check the seal.
5. Put the regulator into operation.

Now it’s time to define each step in carrying out these major tasks. This step is vital, because it’s where the safety checks are carried out. So let’s do all the detail work. The result will be what you see in Figure 1. After three or four hours of training, the donning procedure shouldn’t take more than 45 seconds.

Exactly how your department wants a procedure carried out is up to you. Just make sure that you include all of the tasks and that the trainees have a handle on the prime safety checks.

Refinements

Now that you’ve defined the job and its tasks, you’re almost there. Two questions remain. How do I score the trainee being evaluated, and what do I do with someone who doesn’t carry out the necessary steps?

The answer to the second question is easy, in my eyes: Keep training until the person can carry out all the steps. That’s your job.

The first question is somewhat more difficult, because it gets into scoring strategies. Let’s look at a quick and dirty answer first—a simple yes-or-no checklist will do the job. As the trainee does or doesn’t carry out each step, just check it off in the proper column.

Figure 2. Evaluation Guidelines

Point value 20

Passing 14

Time: 60 Seconds

Correlation with NFPA 1001, Sections 3-3.4, 3-3.5, 3-3.6 Student to don SCBA according to set procedures within stated time.

Task Deduction

Failure to check cylinder gauge – 8

Failure to open cylinder valve – 8

Failure to stoop forward to place unit on back – 4

Failure to connect chest strap – 4

Failure to connect waist strap – 4

Incorrect procedure in donning facepiece – 5

Failure to tighten adjusting straps – 6

Failure to check face mask seal – 8

Failure to check regulator gauge – 7

Breathing tube not secure and fully tightened -10

Failure to check bypass valve – 7

Procedure out of sequence 4

Not completed in allotted time – 4

This is a good way for you to get started on using an evaluation procedure. It’s simple, easy to set up, and easy to score.

As you become familiar with this easy system, though, you’ll become aware that it has some serious drawbacks. First of all, it doesn’t allow you to observe improvement or deterioration in a skill. Second, it doesn’t point out those who’ve missed a major safety check and therefore require major retraining. Third, it doesn’t point out where you, as the instructor, may not be covering specific points properly.

Let’s take a look at a refinement that will meet some of these problems. It’s called evaluation guidelines. It allows a trainee to pass at 100 or 70 percent and to fail at 60 or 0 percent. It also gives an instructor or commanding officer the opportunity to take a look at why trainees are passing or failing. The Buffalo Fire Department uses these guidelines regularly in all training, but more important, we use them in the recruit school battalions at our academy.

We use a narrative-type sheet that requires the evaluator to state very specifically what the trainee does or does not do. Based on the guidelines, that information is converted into a grade point mark. Remember, this can’t be subjective; it must go back to and be based on the objective. It’s more work for the evaluator, but it gives more precise data on how well or poorly we or the trainees are doing.

Figure 2 is one example of guidelines for SCBA. As you can see, they go right back to National Fire Protection Association Standard 1001 and other standards. The time changes as training increases, but the overall guideline remains pretty much the same. Any use of equipment in a manner that could injure a person or cause damage to the equipment during the evaluation or in any field class will be immediately stopped, and a zero will be given that trainee for that session.

This set of guidelines probably won’t be suitable as a complete package for your department, but it will give you some ideas on how to evaluate your individual trainer and trainee skill levels.

Now a quick look at a small-unit evaluation. For example, the objective for individual skill levels for a trainee operating on an engine company would include engaging the pumps, stretching differentsize hose lines, operating hose lines, and meeting and overcoming problems that an engine company normally runs into.

Being objective is more work, but it gives more precise data on how we’re doing.

The next step is to tie this engine company operation into the procedures required to support another engine company—such as laying supply lines—or to coordinate operations with a ladder company, such as advancing hose lines in conjunction with horizontal ventilation. That lets us look at the application of combined skills and operations yet keep the evaluation based on individual skills.

Command evaluations can be set up the same way, using a realistic set of circumstances based on the resources normally available to the command officer. This primarily requires the officer to give orders and resolve problems that can be expected during various types of incidents.

When you’ve mastered these evaluation methods, you’re going to find that your overall programs start to come together a lot more easily, because you’re taking one step at a time rather than just looking at that awfully long, 10-mile hike.