Opinion
Nos. 74-30C(3), 74-200C(3).
June 4, 1979.
David Lange, St. Louis, Mo., for plaintiffs in No. 74-30C(3).
Gerald F. George, U.S. Dept. of Justice, Washington, D.C., Joseph B. Moore, Asst. U.S. Atty., U.S. Dept. of Justice, St. Louis, Mo., for plaintiffs in No. 74 200C(3).
James J. Gallagher and Thomas J. Ray, Associate City Counselors, St. Louis, Mo., for defendants in both cases.
John H. Goffstein, Bartley, Goffstein, Bollato Lange, St. Louis, Mo., for Firefighters Assn., Local # 73 in No. 74-30C(3).
Jerome A. Diekemper, Diekemper, Hammond Shinners, Clayton, Mo., for intervenors in both cases.
Jack L. Koehr, Associate City Counselor, St. Louis, Mo., for defendants in No. 74-200C(3).
MEMORANDUM
This matter is before the Court upon the application of plaintiffs for a preliminary injunction prohibiting defendants from promoting any individual to the position of fire captain based on the results of the 1979 examination for said position. On May 15, 1979, this Court issued a temporary restraining order prohibiting promotions. On May 22, 1979, the temporary restraining order was extended, for good cause shown, pending completion of the introduction of evidence relating to the validity of the examination in question. Rule 65, Federal Rules of Civil Procedure.
Originally, this action was brought by black firefighters and the United States Department of Justice, alleging racially discriminatory practices in the St. Louis Fire Department. Pursuant to the mandate of the appellate court, Firefighters Institute for Racial Equality v. City of St. Louis, 549 F.2d 506 (8th Cir. 1977), the matter was remanded to this Court in part to exercise continuing jurisdiction over the cause pending development of a valid promotional examination. Following remand, various orders of this Court were issued which are relevant herein. Particularly noteworthy are the following portions of orders of this Court:
IT IS FURTHER ORDERED, ADJUDGED, and DECREED that the City of St. Louis utilize best efforts to develop a properly validated examination for the position of fire captain by January 1, 1979. The Court shall assist in the expedition of resolving any parties' objections to said examination. December 19, 1978 order issued herein. [emphasis added]
If defendants wish to assert the validity of an examination, they shall submit to counsel for all parties at least sixty days prior to any intended use, evidence of the validity of the elements of the process, including a copy of the validation study and all underlying documents or data concerning the development of the selection process and its validity. If the parties are unable to agree upon the validity of the process, the process shall not be utilized unless and until the Court determines, upon motion and such evidentiary hearing as it deems appropriate, that the process has been properly validated. January 26, 1978 order issued herein. [emphasis added]
Notwithstanding these explicit directives, plaintiffs chose to ignore the procedure outlined by the Court. Plaintiffs did not contest the examination prior to its administration. Instead, defendant incurred the expense of its administration, estimated at over $20,000.00. Examination scores were computed and posted. Not until promotions were about to be made, and after expectations of promotion of individual firefighters had been raised, did plaintiffs seek to contest the validity of the examination process.
Sadly, the parties have hardened their attitudes rather than conscientiously working together toward a common goal — the finest possible firefighting units for the City of St. Louis. There have been letters to the newspapers and to this Court which approve as well as criticize the written portion of the Assessment Center portion of the examination. There have been suggestings of cheating, although the evidence offered did not reveal any racial connection with respect to the alleged cheating. Tensions have been greatly exacerbated. Over and above all of this, there is a serious problem. The development, administration and evaluation of an examination requires time, which the St. Louis Fire Department can no longer afford if it is to properly protect the citizens of the city of St. Louis. The testimony of the Chief of the St. Louis Fire Department, Charles Kamprad, as well as that of Captain Daniel Austin, reveal widespread problems with morale. Captain Austin stated that these problems resulted from the examination process and the posting of scores with resulting expectations on the part of certain firefighters who anticipated promotions, and disappointment on the part of those who did poorly. Chief Kamprad testified as to the impact of the unfilled positions not only upon morale but upon the efficiency of the department as a whole. There are approximately 55 vacancies at the present time in the position of fire captain. Chief Kamprad estimated that there would be 120 vacancies during the next two years, the life of the promotional list prepared as a result of the 1979 examination. He said morale was lower than he had seen it in many years and cited instances of the effect of this in fighting fires.
The issue to be decided by this Court at this juncture in the proceedings is whether plaintiffs have established "a clear showing of probable success and possible irreparable injury" based upon plaintiffs' claim that the examination was not properly validated. Checker Motors Corporation v. Chrysler Corporation, 405 F.2d 319, 323 (2d Cir. 1969) (emphasis in original). See also Dino DeLaurentiis Cinematografica, S.p.A. v. D-150, Inc., 336 F.2d 373 (2d Cir. 1966); American Smelting and Refining Company v. Pennzoil United, Inc., 295 F. Supp. 149 (D.Del. 1969) adding the factor of balance of hardships. The Court will assume, without deciding, that promotions based upon an invalid examination having a disparate impact sufficiently establishes possible irreparable injury.
The evidence adduced by the parties revealed the following: the promotional examination consisted of a written portion and an Assessment Center portion. The written portion was weighted 30% of the final grade and the Assessment Center score constituted 70%. The grand mean ratings on the Assessment Center portion of the examination were as follows:
Black applicants: 2.864
White applicants: 2.865.
Of the 404 applicants who underwent the Assessment Center portion of the examination, 56 or 13.86% were black. Of the top 100 scores, 10 applicants or 10% were black. These results fail to reveal a disparate impact upon black applicants. The statistical difference between the percentage of applicants who were black and the percentage of applicants scoring within the top 100 places is minimal. In view of this minimal difference, and in light of the small statistical pool, Harper v. Trans World Airlines, Inc., 525 F.2d 409 (8th Cir. 1975), the Court concludes that there is no disparate impact. See Moore v. Southwestern Bell Telephone Company, 593 F.2d 607 (5th Cir. 1979) in which the court stated that "a 7.1% selection differential between black and white applicants does not evidence the required disproportionate impact needed to make a prima facie case of discrimination". [emphasis in original]
A disparate impact did result, however, from the written portion of the examination. Of the 429 applicants who took the written portion, 65 or 15.2% were black. While 97.8% of the white applicants passed the written portion, only 87.7% of the black applicants did so. Because the scores achieved by white and black applicants on the Assessment Center portion revealed no disparate impact, the effect of the written examination, although only 30% of the total score, on the final ranking of candidates resulted in a disparate impact upon black applicants. Thus, of the top 100 final scores, only 7 or 7% were black.
The Court therefore concludes that plaintiffs have established a prima facie case of the examination's discriminatory impact. Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971). Thus, the burden shifts to defendants to establish the validity, or job relatedness, of the examination process. Albemarle Paper Co. v. Moody, 422 U.S. 405, 95 S.Ct. 2362, 45 L.Ed.2d 280 (1975).
Validation of an examination process can be accomplished in a variety of ways.
The preferred method of test validation is criterion-related or empirical validity, which includes what are referred to as the predictive and concurrent methods of validation. Predictive validation consists of a comparison between the examination scores and the subsequent job performance of those applicants who are hired. If there is a sufficient correlation between test scores and job performance, the examination is considered to be a valid or job-related one. Concurrent validation requires the administration of the examination to a group of current employees and a comparison of their relative scores and relative performance on the job.
An examination has content validity if the content of the examination matches the content of the job. For a test to be content valid, the aptitudes and skills required for successful examination performance must be those aptitudes and skills required for successful job performance. It is essential that the examination test these attributes both in proportion to their relative importance on the job and at the level of difficulty demanded by the job. Vulcan Society of New York City Fire Department, Inc. v. Civil Service Commission of the City of New York, 360 F. Supp. 1265, 1273-74 (S.D.N.Y. 1973), modified, 490 F.2d 387 (2d Cir. 1973).
This Court has previously concluded that content validation was and is the only feasible approach herein. United States v. City of St. Louis, 410 F. Supp. 948 (E.D.Mo. 1976), rev'd in part, 549 F.2d 506 (8th Cir. 1977). See also Vulcan Society of New York City Fire Department, Inc., supra; Commonwealth of Pennsylvania v. Glickman, 370 F. Supp. 724 (W.D.Pa. 1974); Kirkland v. The New York State Department of Correctional Services, 374 F. Supp. 1361 (S.D.N.Y. 1974), modified, 520 F.2d 420 (2d Cir. 1975). The evidence adduced at the hearings held in connection with the present application for injunctive relief reaffirms this conclusion. All of the parties seem to concede this point.
In order to establish that the examination herein has content validity,
. . . defendants must demonstrate not only that the knowledge, skills and abilities tested for by [the examination] . . . coincide with some of the knowledge, skills and abilities required successfully to perform on the job, but also that 1) the attributes selected for examination are critical and not merely peripherally related to successful job performance; 2) the various portions of the examination are accurately weighted to reflect the relative importance to the job of the attributes for which they test; and 3) the level of difficulty of the exam matches the level of difficulty for the job. Kirkland, supra at 1372.
Because a written examination could not accurately assess certain of the skills required of the position, the examination included not only a paper-and-pencil examination but also an Assessment Center portion in which applicants were required to respond to simulated situations. This method was suggested by plaintiffs' expert, Dr. Richard Barrett at the prior trial and approved by the United States Court of Appeals for the Eighth Circuit in its February 2, 1977 decision herein.
The items to be included on the written portion of the examination as well as on the Assessment Center portion were determined after development of a thorough job analysis. An expert panel of fire personnel was constituted and assisted in the preparations of both portions of the examination. Two of the four panel members were black. The Court notes, at this juncture, that black fire personnel, although encouraged and invited to participate, were unwilling to do so. Captain Austin participated as a member of the expert panel only when directly ordered to do so.
Task questionnaires were completed. Each task was rated on a number of factors and then ranked in order of importance. Each task was linked to the knowledge, skills and abilities ["KSA"] to which it related. Each KSA was then rated on the basis of different factors and ranked in terms of importance. There can be no doubt as to the thoroughness of the job analysis.
The list of KSAs was used to construct the examination. The expert panel determined which KSAs could be tested for by a written examination. Based thereon, test items were prepared by Edmond Knowles, an employee of the Department of Personnel of the City of St. Louis and an expert in the development and validation of examinations, members of his staff, and the expert panel of fire personnel.
The expert panel linked the KSAs to the test items. They rated the difficulty of each item in accordance with the difficulty of the position. Similar care was taken in connection with the Assessment Center portion of the examination, concerning the items to be included for testing, the simulations developed therefor and the reliability of the raters.
A vast number of statistical analyses were performed to determine the reliability of both the written examination and the Assessment Center examination. These analyses consistently supported the reliability of the examination.
The criticisms levelled by plaintiffs' expert, Dr. Richard Barrett, and by Captain Austin, are insufficient to warrant the conclusions that the examination was not valid. Dr. Barrett, concededly an expert in these areas, testified that no matter how extensive and thorough the preparation of an examination might be, it could still prove to be invalid based upon post-administration statistical analysis. He stated that even he might not be able to construct an examination which would withstand validation scrutiny. The ease with which Dr. Barrett attacked certain portions of the examination and the general construction of the examination leads this Court to observe that any examination can be picked apart in "Monday morning quarterback" fashion.
The Court would observe that Dr. Barrett is an expert at this "picking apart" process. Yet, he admittedly has very limited experience in constructing examinations. He has testified in litigation concerning 65 examinations and found that only two of those examinations passed muster. Of course, it is possible that 63 of the examinations truly did fail to meet the validation requirements of the law. Thus, his litigation statistics may be meaningless but they do point up Dr. Barrett's ability to criticize examinations. The Court would state that Dr. Barrett is an intelligent person, a qualified man in the field. His honest nature, as indicated above, caused him to concede that even he might not be able to construct an examination for the position of fire captain that would prove valid under the law.
The City of St. Louis, through the efforts of Edmund Knowles, also an expert in the field, made every effort to produce a valid examination, a fact conceded by plaintiffs herein. The Court finds that defendants undertook to follow the mandate of the United States Court of Appeals for the Eighth Circuit and the orders of this Court in developing a fair testing procedure. Acting in good faith, defendants developed a new examination. Help was sought from fire personnel, the obvious true experts in this area. Knowles did all that could be done to eliminate any possible bias in the examination process.
It is not the job of this, or any other Court to tell experts how to write an examination. Even having to determine the question of validity gives this Court pause. Two experts presented this Court with diametrically opposed opinions, based on diametrically opposed statistical analyses, of the identical data. The area in which these experts testified is not an area in which most judges have great expertise.
It is this Court's conclusion that the criticisms levelled at the examination simply do not defeat the overwhelming evidence that this examination was valid. The evidence also failed to support the claim that racially discriminatory appointments to the position of acting captain, or senior man, affected test results. The statistical evidence failed to establish a correlation between examination scores and such temporary appointments.
Based upon the record presented to this Court, the Court must conclude that plaintiffs have not fulfilled their burden of establishing probable success on the merits. Although the examination resulted in a disparate impact upon black applicants, it was sufficiently validated. Thus, defendants have fulfilled their obligations under the law.
The application for preliminary injunction will be denied, and the temporary restraining order dissolved.