Fire Fighter Training and the Teaching Machine
A Final Report—
Photos courtesy Los Angeles County F. D.
This report describes the second phase of a two-part study involving the Los Angeles County Fire Department, the Hughes Aircraft Company, and the Hughes Videosonic Teaching Machine. Phase I (FIRE ENGINEERING, June 1963) established the basic feasibility of using machined instructions in the training of professional fire fighters (Silvern and Fuchs, 1963).
This article is a condensation of a paper prepared for the 90th Annual Conference, International Association of Fire Chiefs, Memphis, Tenn., October 1963.
SINGLE-LEARNER teaching machine concepts are clearly effective, but there has been growing reason to wonder if the group approach might not be applicable also. Due to the serious implications for the purchaser, as well as the designer, programmer and tester, of complex instructional equipment, it was important the answer be found.
The aim of Phase II was to determine whether an aural-visual teaching machine produces more effective results when applied to one learner at a time or three learners at a time. Effectiveness is measured by individual performance on a course-content comprehension test administered immediately following and one month after instruction.
From the results of Phase I, four sets of test scores were randomly deleted, leaving an even 30 to represent the individual-learner condition. Thirty recruit firemen of equivalent age, experience, education and intelligence were then drawn from stations throughout Los Angeles County to represent the group-learner condition. All of the other controls specified in the Phase I report pertaining to equipment, training room, lesson material and test procedures were applied in an identical manner in Phase II. The only detail changed was the number of learners per machine. In Phase I, each man had a machine to himself (Figure 1). In Phase II, each machine was shared by three men (Figure 2).
Six men were processed per day in Phase II. Following the 15-minute, individually applied pre-test, they were formed into groups of three and apprised of the ground rules. It was explained they would have to rotate positions in front of the screen every half hour, so as to give each man an equal chance to sit in the middle, where he could see best. They were further told to talk over the alternatives before giving any of the pushbutton answers called for by the program, then let the man in the middle push the button. The same type of cooperation was urged on written questions appearing in the program. In short, the men were given to understand they were in no way competing with each other and that there would be no penalties for wrong answers. Rest and lunch periods were to be taken whenever the group wished.
As each group finished, its members were post-tested individually. Thirty days later, the combined test forms and opinion questionnaire were mailed to each man’s station, administered by his captain and returned to headquarters in sealed envelopes. As the data emerged, it was categorized as shown in Table 2, Test Scores and Comprehension Gains for the Group-Learner Condition. Table 1 contains comparable data which had already been accumulated in Phase I for the singlelearner condition.
In experimental work where measurements tend to vary from one individual to another, it is necessary to apply statistical tests to determine whether, all variations considered, the values observed are meaningful. The test employed in this case was the Mann-Whitney U test, a mathematical model able to compare two sets of ranked scores and produce, for samples of this size, what is referred to as a “z” value. Along with the test is a tabulation of all possible z values and the probability associated with each, assuming set equivalence. The reasoning behind the test is as follows: If a particular comparison yields a z value which the table shows to be very improbable, that comparison is said to be “significant.” Any other result is attributed purely to chance and is regarded as inconsequential. In the present study, it was decided to set the level of significance at 1 per cent, which is characterized in the z table by all values greater than +2.58 or —2.58. Those values, in other words, are so extreme they occur by chance only one time (or less) out of 100. They are therefore considered indicative of forces other than chance in the data. Any z value between +2.58 and —2.58 would signify the relative absence of “real” differences between the two sets of scores. In this particular situation, minus z values would favor the group condition, while plus z values would favor the single-learner condition.
As illustrated by Figure 3, the comparison of immediate gains (post-test scores minus pre-test scores) revealed very little observable difference between single learners and group learners. When the statistical test was applied, a z value of —1.06 resulted (see Table 3). This showed the group condition to be slightly superior but not nearly enough to satisfy the requirements of the test, the; value needed for significance being — 2.58. In terms of immediate gain, therefore, neither experimental condition was found to be dominant.
Table 4 reflects similar results for the comparison of eventual gains (retention test scores minus pre-test scores). The group condition appeared a shade better but the z value obtained was substantially below the — 2.58 needed for 1 per cent significance. Figure 4 further illustrates the closeness of the relationship.
TABLE 1—Test Scores and Comprehension Gains
TABLE 2—Test Scores and Comprehension Gains
The two conditions also showed marked equivalence in the amount of time taken to complete the lesson. Each averaged 3 ¼ hours in the morning and 3 ¼ hours in the afternoon, exclusive of tests, lunch and rest periods.
Private opinions regarding group participation are shown in Table 5. Item 1 reflects the general preference for machine training in smaller concentrations than this particular program allowed. Items 2, 3 and 4 suggest that relatively little difficulty was caused by shared viewing, shared sound control and potentially annoying member behavior. In Item 5, members acknowledge they received help from each other. Items 6 and 7 indicate that, although half the members felt the pace was not exactly right for them, only a few would have favored complete separation from the group.
The statistical expressions at the right of Table 5 may be interpreted in much the same manner as explained earlier for the Mann-Whitney U test. The test here (named Chi Square) measures how evenly the votes on each question fell into the various available categories. By chance alone, they would have distributed themselves smoothly, as in Item 7. The test value obtained there was only .80. Since a value of 9.21 was needed, the distribution was not significant. The same may be said of Item 1. Items 2 through 6, however, had very uneven distributions, so that the critical value was exceeded in each instance. By chance alone, such values would have occurred less than one time out of a hundred. Therefore, something other than chance may be considered to have caused the warpage. Hopefully, that something was the innate attractiveness of group condition.
The object of Phase II was to determine whether the Videosonic Teaching Machine produces more effective results when applied to one learner at a time or to three learners simultaneously. On the basis of test scores alone, experimental evidence gave support to neither approach. In terms of economy and equipment utilization, however, the very closeness of the results may be interpreted as favorable to the group condition. Without any sacrifice in subject matter acquisition the machine was able to deliver three times as much learner coverage as had been thought possible earlier. Of corollary importance were the responses on the opinion survey which systematically reflected satisfaction with group participation.
Prior to collection of the data, a number of factors were shown to he linked with the learning process. These were active participation, knowledge of results, and self-pacing, later named as a special form of group harmony. Each was watched for carefully during the experiment and the following observations made:
Active Participation: As expected, the single learner received and used many opportunities for active participation. Each lesson point was aimed directly at him and he was obliged to share with no one in composing his responses. However, the single learner tended to make only those responses specifically called for by the program; he wrote words and he pushed buttons. As a general rule, group members did that and more. They talked about the questions, suggested answers, agreed or disagreed with the suggestions of others, cited evidence to support their views, and commented appropriately when the correct answer finally appeared. The act of sharing seemed to cause group members to become more, rather than less, involved with the training material. This was undoubtedly facilitated by the requirement that they reach a consensus on each question before proceeding.
These notions are supported by Items 2 and 5 of the opinion survey. Item 2 indicates that little difficulty was encountered in viewing the screen. This is a vital element in participation. On Item 5, only one member reports that very little help was given in his group, while 20 members recall very much help. Only a high level of interaction could have brought this about.
TABLE 3—Statistical Comparison of Immediate Gains
TABLE 4—Statistical Comparison of Eventual Gains
TABLE 5—Private Opinions of Group Members
Knowledge of Results: Knowledge of results was given to all participants in precisely the same manner. For multiple-choice questions a green light signified a correct (push-button) response, and a yellow light meant “incorrect—try again.” Immediately following a correct response, a restatement of the correct answer appeared in the audio mode. The same was true for written-completion questions, except that these were handled in blocks for review purposes.
A slight difference was noted in the way the members of each condition made use of this information. Individual learners characteristically reacted in silence, while groups exhibited a noticeable degree of enthusiasm. Mock congratulations were accorded any group member who answered a particularly difficult question; a fumbled response generally invited good-natured derision. On written-completion questions, single learners seldom modified their answers after being told they were wrong. Group members nearly always made the change. Nothing in the program required this and all participants knew in advance that such answers would have no bearing on their final evaluation. These are clues which might be interpreted as a tendency of the group to make better use of the feedback received from the instructional system.
Group Harmony: As already suggested, most participants seemed to enjoy the group activity. Cooperation was readily apparent. In some cases, groups voluntarily realigned control of the response panel, giving one button to each member. In other cases, when blocks of questions had to be answered in writing, the man in the middle, with the best view, would slip into the role of oral reader, letting the others suggest the answers.
There is also the possibility that the element of vigilance was shared. Taking this lesson all in one day was a fatiguing experience. It was difficult to remain alert throughout every section. From time to time, postures would slump and eyelids droop. To the single learner, such relaxation was damaging, for there was no way to retrace lost steps. To the group learner, it was less serious, since fellowmembers could always be appealed to for assistance. Group members may thus have lightened the burden by rotating in some tacit manner the opportunity for brief rest periods.
While several minor expressions of disharmony occurred, not one instance of genuine conflict was observed. Typical irritants were pencil-drumming, irrelevant conversation, button-pushing out of turn, exclusion of one member in formulating responses, and crowding of the screen by the middle man. The suspicion that none of these was taken too seriously is reinforced by the opinions given on survey Items 2 through 5, in which satisfaction is reported with screen viewing, joint sound control, member behavior, and exchanges of assistance.
With respect to the specific problem of pacing, there was a substantial difference among member preferences, as shown by Item 7. Fully half the respondents say they would not have proceeded at the rate of speed chosen by their groups had they been working alone. It is interesting to note, however, that only five of the 30 regarded working alone as the best situation possible. In other words, even though fully aware of its consequences (loss of self-pacing) group members voted consistently in favor of the group approach.
It is believed that the data presented here are evidence of the fundamental soundness of the group approach with the complex teaching machine. As such, it should be of value in at least four areas of the education and training community. To the machine designer, it might mean consideration of larger screens and more conveniently located response mechanisms. To the program developer, it might mean more emphasis on discussion items, less wordiness on the screen, more use of workbooks, and possibly some attempts at team training. To the program tester, it might mean faster development of a finished product through acceleration of the tryout-revision cycle, heretofore sorely hampered by the slow pace of single-learner data collection. To the machine user, it might mean more people trained per unit of equipment and a way of treating subject matter not readily amenable to other forms of instruction. The question is no longer “will the machine work on groups?” but rather “how large a group could one unit handle?”
Continued on page 65
Continued from page 45
This marks the end of the teaching machine project carried on by the Los Angeles County Fire Department and the Hughes Aircraft Company since September 1960. During that three-year period, the special program in Fire Behavior was developed, the feasibility of the new method firmly established (Phase I), and (Phase II) an extension of that method successfully executed. The entire effort was without precedent in the field of fire fighter training.
Two recommendations emerge: (1) That the fire service give serious consideration to the wider use of programmed instruction, in paper form if necessary, but as soon as possible; and (2) that as the complex teaching machine is brought into play, training activities take full advantage of the group approach in their operations. Teaching machine technology is here to stay. The sooner fire organizations begin harnessing its power, the more fully its benefits will be realized.
Many men participated in these studies—too many to list by name. It would be appropriate, however, to identify four, without whose early vision and sustained encouragement the project might never have borne fruit: Keith Klinger, chief engineer, Los Angeles County Fire Department; William A. Harker, manager, Videosonic Division, Hughes Aircraft Company; B. Richter Townsend, general manager, International Association of Fire Chiefs; and Dr. Leonard C. Silvern, formerly of the Hughes Aircraft Company, now a senior scientist with the Northrop Corporation.