How Centers Can Aid Training Program
In today’s world of class action employment discrimination lawsuits and large-scale budget cuts, assessment centers are an especially useful personnel management tool.
Regardless of the exact job in question, or the desired personnel decision, the goal of all assessment centers is to accurately simulate essential job skills under standardized conditions. As the chart indicates, deciding exactly what to simulate is the first crucial step in constructing an assessment center. Not only will this influence the accuracy of the final personnel decision, but it also provides an opportunity to demonstrate job-relatedness, as required by Equal Employment Opportunity Commission (EEOC) regulations.
The key to appropriate simulation lies in a thorough job analysis and regardless of the exact method used, an adequate job analysis should be based upon expert consensus. Too often, the decisions about which job features to simulate are arbitrary executive decisions.
Getting valid job definition
Instead, care should be taken to have people who are thoroughly familiar with the job define its requirements and their relative importance. The people considered to have the necessary expertise for this task are usually drawn from a pool of incumbents, superiors or outside consultants.
A popular way to init iate the job definition and rating process is to break the official written job description into job elements and have the experts rate the importance of each. For example, “disciplining subordinates,” “making a budget,” and “leading company drills” are job elements that can be rated for their relative importance to a battalion chief’s job.
The primary reasons for preferring an expert consensus to an individual decision revolve around statistical considerations. First, any expert is likely to make some errors or to have certain biases, and at least some of the errors and biases of one expert are unrelated to those of another expert. Given this pattern of errors and biases, a composite rating, based on expert consensus, will cancel some of the individual errors and result in a more accurate job definition.
A second reason for basing the job definitions upon more than one expert is because statistical analyses can then be used to measure the amount of agreement among the experts. Strong agreement among the experts is the basis for a sound job definition.
Selection of tasks
Once the job components have been defined and rated, intelligent decisions can be made about which tasks will be included in the assessment center (see stage II in the chart). Again, the most essential characteristic of a good assessment center is accurate simulation of the target job and if care is taken at this stage, the resulting assessment center should meet the EEOC mandate for validity.
In exactly the same manner as the essential job skills were defined, the weights assigned to the component ability measures should be derived from multiple expert judgments. Based on these weights, the component scores can be combined into an overall ability index. Again, a measure of agreement among the judges should be obtained and a method adopted for summarizing the central tendency of the ratings. With good agreement among the experts, confidence can be placed in the resulting rule for aggregating the component scores.
How Assessment Centers Work
Once the assessment center tasks and their weights have been defined, care should be taken to ensure that the scoring procedure is standardized and doesn’t give any candidate an unfair advantage. The scoring system should also incorporate the features of multiple assessment and accurate summarizing. It is essential that the scoring procedure include the added precaution of blind rating. That is, the raters should not know the people they are rating, as this could introduce bias. High agreement among the judges at this stage establishes the inter-rater reliability required for accurate, defensible results.
Useful for training
Assessment centers are as useful for training as they are for selecting employees and that should be good news to budget-conscious city managers and fire chiefs. (See stages IIIa and IIIb in chart.) In the broadest sense, assessment centers are useful training devices because (if designed properly) they represent a sample of central job skills. Because assessment centers test candidates under standardized conditions, the candidates can be compared to each other as well as to an external standard.
Real payoffs to your department’s training program begin when the assessment center candidates are evaluated more thoroughly than just in terms of their final finishing order. If candidates are assigned absolute scores, rather than mere rank orderings, valuable information is obtained about the differences between the candidates. Are candidates 2 and 4 essentially equal on leadership? Is candidate 1 far behind candidates 3 and 5 on everything but public speaking ability? Absolute scores can always be converted to a rank ordering, but the reverse is impossible.
Comparing the candidates to an external standard, such as minimally acceptable competence, provides even more valuable training information. To have a rank ordering of candidates, all of whom are exemplary, is very different from the same rank ordering but with some candidates performing at a substandard level. Careful analysis of the performance patterns can reveal both individual and departmental training needs.
Assessment centers also provide the microscopic perspective required for certain training decisions. For example, after observing a leaderless group exercise, it might be apparent that candidate 1 is too timid while candidate 3 is too impulsive for effective leadership. This outcome suggests that candidate 1 needs assertiveness training and candidate 3 needs self-control training. A blanket prescription of self-control training would help candidate 3, but would make candidate 1 even worse. Similarly, assertiveness training would help candidate 1, but would make candidate 3 worse. Such detailed information should be part of every assessment center and once gathered, it should be used to help plan future training.
Although often minimized, or overlooked completely, post-assessment center conferences between the individual candidates and the assessment center manager can play a pivotal role in determining the overall effectiveness of the process.
The first order of business should be to explain the basis for the candidate’s final score(s). The candidate should be briefed on how the scoring system was established and every effort should be made to document the fairness of the procedures.
Value of feedback
Presenting the candidates with concrete feedback about their strengths and weaknesses, along with explanatory comments, can play a tremendous role in improving the candidates’ future performance. The debriefing conference provides a ready opportunity for using this process. This is the time to make sure the candidates accept the results and every effort should be made to leave the candidates with a good feeling about the assessment center and themselves. Not everyone can get the top mark, but everyone can leave with increased self-knowledge and a feeling that the process was fair. Whenever possible, candidates should receive suggestions for improving their performance.
Inappropriate job placement might become evident during the course of an assessment center, providing a springboard for vocational counseling. The goal of this effort should be to provide the best match of the person’s interest, abilities and job assignment. The debriefing conference provides the perfect opportunity for beginning a counseling process that can increase personal satisfaction and departmental effectiveness.
Since many of the issues discussed involve technical principles of experimental design and statistical analysis, fire departments shouldn’t hesitate to hire specialists in these areas to help with the assessment process. Close collaboration between such experimental methodology experts and fire service experts should result in a well-run assessment center that will pay for itself many times over.