Opinion
No. CV00 37 44 31 S
January 10, 2002
MEMORANDUM OF DECISION
Before the court is the plaintiffs' action seeking a declaratory judgment that the civil service examination, administered in May, 1999, by the City of Bridgeport, for the position of fire lieutenant is invalid as to the plaintiffs, and injunctive relief barring the defendants from using the results of the test as to the plaintiffs, and from transferring or reassigning them out of their present positions. The plaintiffs, Luigi Perelli, Steven Earl, and Deborah Ramos, are sworn firefighters and employees of the City of Bridgeport Fire Department. The defendants herein are the city of Bridgeport, the city's mayor, Joseph Ganim, the city of Bridgeport civil service commission, the fire department's personnel director, John Colligan, the members and office holders of the Bridgeport civil service commission, and the chief of the fire department, Michael Maglione. In addition, the Bridgeport Fire Fighters Local 834 (the union) is also a defendant herein.
This action was commenced by six plaintiffs: the three remaining plaintiffs and Joseph Vitale, Lawrence Bussell, and Gary Fanning. The latter three plaintiffs have since retired from the fire department, and only the three remaining plaintiffs still seek relief.
The city of Bridgeport civil service commission is authorized by, and created pursuant to the Charter of the city of Bridgeport. The Charter divides city employees into two categories, classified and unclassified. The plaintiffs are members of the classified category of employees within the civil service commission. Appointment to positions within the fire department is governed by the civil service system of the city. The Charter and civil service rules of the city provide that permanent positions within the civil service system shall be appointed by the civil service commission on the basis of merit from appointment lists. These appointment lists are created from the results of competitive civil service examinations developed and administered by the personnel director and his staff. The Charter also authorizes temporary appointments to vacant positions pending the administration of a civil service examination to permanently fill such positions.
The plaintiffs presently hold positions as fire alarm supervisor, lineman and administrative clerk. The positions of fire alarm supervisor, lineman and administrative clerk were created by the city, the mayor, the civil service commission, the chief of the department, and the personnel director of the department. Fire alarm supervisors dispatch fire assignments, keep track of the department's manpower resources, tend to the communications system, supervise civilian operators, and carry out other related functions. Linemen manage, inspect, operate, change and repair the department's communications systems. Administrative clerks handle the administrative paperwork of the department.
The plaintiffs at one time received training in fire suppression, but, while the plaintiffs were in the position of fire alarm supervisor, lineman, and administrative clerk, they did not engage in fire suppression duties and did not receive the type of training given to line firefighters. Civil service examinations have never been administered for appointments to these positions, and they are denominated either temporary or provisional. There is no formal training from the Bridgeport Fire Department for these positions, and any training can be characterized as on the job training. Between 1997 and 1998, the city sought to incorporate in its contract with the fire union a provision that abolishes the positions of fire alarm supervisor, lineman, and administrative clerk as separate civil service positions and consolidates these positions within the rank of fire lieutenant. This new provision would supersede contrary provisions of the Charter. As a result of contract negotiations with the fire union in 1998, a contract including the new provision was approved in the Spring of 1999. Accordingly, effective January 1999, the positions of fire alarm supervisor, lineman, and administrative clerk were consolidated into the position of fire lieutenant, and the functions of these positions were now included in the functions of the fire lieutenant position. As a result of the consolidation, the plaintiffs would not remain in their present positions unless they were promoted to the rank of lieutenant.
In 1998, the personnel director, Jack Colligan, developed a promotional test for the position of fire lieutenant with the assistance of his staff and Bruce Davey, an expert consultant retained by the City. To aid in the development of the promotional test, a validation study was conducted to determine the knowledge, skills and abilities required by the fire lieutenant's position. At the time the validation study was conducted, the personnel director and Davey were unaware that the positions of fire alarm supervisor, lineman, and administrative clerk were abolished and consolidated into the position of fire lieutenant. Upon notice a written examination was given on May 1, 1999, and an oral examination was given on May 12, 1999. The weighting was 60 percent for the oral portion, and 35 percent for the written portion, with the remaining 5 percent going to seniority. The passing score was set at 70 percent of the highest score. The City anticipated promoting only the top 24 candidates. The candidate ranked 24th on the promotion list had a score of 81.35, which is approximately ten points higher than plaintiff Steven Earl's score.
Of the three remaining plaintiffs, Steven Earl is a firefighter who has been assigned to duty as a fire alarm supervisor whose duties involve, generally, emergency dispatch. Firefighter Earl took both the written and oral parts of the test and received a combined final score of 71.95, which was enough to pass the promotional exam. Earl ranked 66th on the list, and he was the plaintiff with the highest score.
Luigi Perelli is a firefighter who has been assigned to duty as a lineman whose duties involve, generally, computer assisted dispatch and technology related functions. Firefighter Perelli took both the written and oral parts of the test and received a combined final score of 68.89, which was enough to pass the promotional exam. Perelli ranked 74th on the list.
Debora Ramos is a fire alarm supervisor and is certified as a Fire Fighter I which permits her to teach new firefighters. Firefighter Ramos performed above average on the written test and received a score of 75, which is higher than the score of several people who have already been promoted to the rank of lieutenant. Firefighter Ramos did not take the oral examination and therefore did not receive a combined final score.
The parties are in agreement that the plaintiffs, in order to be successful, must establish that the promotional examination was created and administered "unreasonably, arbitrarily, illegally or in abuse of . . . discretion." Murchison v. Civil Service Comm., 234 Conn. 35, 51, 660 A.2d 850 (1995); See Ziomek v. Bartimole, 156 Conn. 604, 611, 244 A.2d 380 (1968) ("Where the board acts capriciously . . . in conducting the . . . examination, its action will be invalidated."); See also Lukachik v. Jankura, 27 Conn. Sup. 1, 8, 228 A.2d 523 (1966) (judgment of civil service agency not to be interfered with by courts in absence of proof of bad faith or arbitrary, capricious or illegal action).
The plaintiffs assert that the examination was deficient under the above quoted standard in the following respects: (1) there was inadequate documentation of the validation study; (2) the establishment of the passing score was improper; (3) the defendants failed to revise the test or conduct a study after the consolidation became known; (4) the changing of test scores after appeals were taken was performed improperly and without the input of subject matter experts; (5) the oral portion of the test was improper in that the test did not permit sufficient prompts by the examiners; and (6) the test was unfair to the plaintiffs.
"To be competitive [a] test must employ a standard or measure sufficiently objective to be capable of being challenged and reviewed by other examiners of equal ability and experience." (Internal quotation marks omitted.) Del Vecchio v. Civil Service Commission, 178 Conn. 490, 492, 423 A.2d 139 (1979). The plaintiffs claim that there are three sources of authority in test design and validation generally accepted in the employee testing field: (1) the Equal Employment Opportunity Commission's Uniform Guidelines for Employee Selection Procedures, 43 FR 38295, § 60-3 et seq. (August 25, 1978) (Uniform Guidelines); (2) the Principles for the Validation and Use of Employment Tests promulgated in 1987 by the Society for Industrial and Organizational Psychology (SIOP Principles); and (3) the Standards for Educational and Psychological Tests of the American Psychological Association (APA Standards).
The various standards do not have the force of law statute or regulation. Thus, the Uniform Guidelines states:
"These guidelines incorporate a single set of principles which are designed to assist employers, labor organizations, employment agencies, and licensing and certification boards to comply with requirements of Federal law prohibiting employment practices which discriminate on grounds of race, color, religion, sex, and national origin. They are designed to provide a framework for determining the proper use of tests and other selection procedures. These guidelines do not require a user to conduct validity studies of selection procedures where no adverse impact results. However, all users are encouraged to use selection procedures which are valid, especially users operating under merit principles."
(Emphasis added.) Uniform Guidelines, supra, at 60-3.1. In the present case there is no claim that there is a disparate or adverse impact involving any protected class or claims of discrimination on the grounds of race, color, religion, sex and national origin.
Similarly the SIOP Principles provide:
"A final caveat is necessary in view of the prominence of testing issues in litigation. This document is prepared as a technical guide for those within the sponsoring professions; it is not written as law. What is intended is a set of standards to be used, in part, for self-evaluation by test developers and test users. An evaluation of their competence does not rest on the literal satisfaction of every relevant provision of this document. The individual standards are statements of ideals or goals, some having priority over others. Instead, an evaluation of competence depends on the degree to which the intent of this document has been satisfied by the test developer or user."
(Emphasis in original.) SIOP Principles, p. 3. Similarly the APA Standards are intended to offer guidance and do not require "the literal satisfaction of every standard in this document." APA Standards (1999), p. 4.
(1) Whether the validation study was inadequately documented
The plaintiffs claim that the test prepared by the expert retained by the City of Bridgeport, Mr. Davey, was not properly documented. The most serious claim of lack of documentation was the failure of Mr. Davey to prepare a narrative report explaining to an independent observer what Mr. Davey had done. The plaintiffs claim that the documentation of a validity study must be contemporaneous and cannot be prepared only after a test comes under attack. The plaintiffs further claim that it was impossible to understand from the raw documents what information Mr. Davey looked at, relied upon, and considered. They claim that there was no documentation of quantified linkages between the job duties and the knowledge, skills, and abilities necessary to perform that job. The plaintiffs further claim that such linkages should be documented by subject matter experts but that there was no documentation here that such experts were used. Additionally, they claim that there should be documented linkages between the exam components and the job analysis, that there was no documentary evidence that this had been done prior to the administration of the exam, and that the generation of after the fact linkages by Mr. Davey was inappropriate. Another failure of documentation, the plaintiffs claim, is that there should have been records concerning the calibration training provided to the people who rated the oral portion of the examination. The plaintiffs argue that because of these claims the examination was unreasonable, arbitrary, capricious, and illegal because the validation study was not property documented.
The defendant's expert, Mr. Davey, testified that a narrative report was not prepared because the City did not want to pay for one. The defendants also claim that a validation study documented as plaintiff asserts is not necessary where the test maker himself is giving the test and where there is no adverse impact upon a protected class.
The failure to have a final narrative report and to have the documentation in the form claimed by the plaintiffs do not in themselves invalidate the test. It is therefore necessary to review what actually was done in preparing and scoring the test.
Mr. Davey first met with the Chief and Assistant Chiefs to discuss the job of fire lieutenant and the needs the department hoped to meet in testing for the position. He documented the many observations and concerns of those officers. For example, Mr. Davey learned that approximately forty-five lieutenants were assigned as company officers, eight lieutenants were assigned as Chiefs aides, and four were assigned to training. He learned that those assigned to the positions of Chiefs aide and training officer performed specialized functions they were taught upon assignment, and that they could be transferred back to line duty at the Chiefs discretion. These facts, combined with the fact that a low number of people were assigned to these positions, confirmed that the core functions of a fire lieutenant were those of the company officers. Since Mr. Davey wanted to test only for the "core," "critical," or "important" functions of the fire lieutenant position he did not include questions on the examination relating to the specialized positions of Chiefs aide and training officer.
Mr. Davey interviewed ten fire lieutenants to determine exactly what their jobs entailed. This information was recorded in thirty-five pages of hand written notes. From the information acquired at these meetings Mr. Davey created a Job Analysis Questionnaire, which is a list-in question form-of the knowledge, skills, abilities, and duties of varying importance of the job of fire lieutenant. The questionnaire was divided into three sections, and each section included an "importance rating scale." Mr. Davie utilized such information to determine which areas of knowledge, ability, and duties were most important and needed the most attention in the test development process. Additionally, Mr. Davey performed a statistical summary of the job analysis questionnaire results, enabling him to assign importance ratings to each knowledge, skill, and ability required of new lieutenants, and to rank them in order of their importance. He also calculated the "KSAP weight" (Knowledge Skills Abilities Percentage weight) to determine what traits were both important and required at the point one was hired into the job. See Joint Exhibit 4. All of this information culminated in a determination that the most important knowledge, skills, and abilities would be best tested through an oral examination, and that others would be better suited for a written examination.
Mr. Davey next developed the written portion of the test using existing questions from Mr. Davey's file and others prepared for this particular test. A test question review booklet was prepared consisting of 186 questions grouped according to subject matter. The questions were submitted to three firefighting experts, together with a test question rating form, seeking to determine whether the answers given on the test were correct. The experts were also asked to rate the importance of each question and comments were received. A group meeting was then held to discuss the questions, and statistical summaries were prepared of the experts' ratings for each question. These ratings were then used as an index for selecting the most important and acceptable questions. The summary revealed that it would be statistically appropriate to use any question with a certain rating. See Joint Exhibit 6. Mr. Davey then selected 100 questions for the written examination. Questions selected were deemed appropriate by the experts and met the statistical standard.
For the oral portion of the test Mr. Davey enlisted the same three experts that reviewed the draft scenarios and questions for the written portion of the test. The oral questions were then selected and benchmarks — elements of a correct response — were established for each of the questions. The questions and benchmarks were reviewed and refined by the experts and consultants over the course of three meetings, after which they were judged as a representative sampling of the job's important duties and requirements.
Mr. Davey next prepared a reading list of written resources which could be used by the candidates to prepare for the examination. It appeared from the testimony that the questions in the examination could be answered from the materials contained in the reading list
Some forty-five panelists from throughout the tri-state area, each having attained the rank of fire lieutenant or higher, administered the oral examination. They were trained with respect to the procedures and applications of the rating system to be utilized in the oral examination. After this preparation, the examination was administered. The written and oral components were administered separately. The written portion of the exam was administered to 142 candidates, and the oral examination was administered to 129 candidates, in accordance with established procedures.
The court finds, based on the foregoing, that the administration of the test, and the preparation of the components of the test, were neither unreasonable, arbitrary, capricious or illegal.
(2) Whether the establishment of the passing score was improper
The plaintiffs next claim that the passing score used on the examination was neither reasonable nor supported by professional analysis. The Civil Service rules provide for a 75 percent cut-off score, and the defendants claim that they have discretion to change that score where appropriate. Accordingly, the City utilized a cut-off score of 70 percent of the highest score obtained on the examination. It is unclear whether the cut-off score was determined by Mr. Colligan or by Mr. Davey, or by a combination of the two. "It is clear, however, that Mr. Colligan and Mr. Davey did discuss the passing score and decided to utilize the 70 percent of the highest score as opposed to a flat 75 percent cut-off. The plaintiffs claim that the determination to utilize a particular passing grade was not properly documented, and that the passing score was unrelated to proficiency within the work force.
In the communication between Mr. Davey and Mr. Colligan the number of candidates who were likely to be included or excluded based upon a cut-off set at 70 percent of the highest score versus 75 percent of 100 was considered, and it was decided that it was better to be inclusive. The intent was to be able to assure that the appointing authority would be able to fill its vacancies with competent employees.
Two of the present plaintiffs, Mr. Earl and Mr. Perelli, passed the test because the 70 percent of the highest score cut-off was utilized. If the 75 percent out of 100 cut-off had been utilized, neither of those plaintiffs would have passed the test. The third remaining plaintiff Deborah Ramos, would not be affected by the passing score determination because she only took the written examination and did not take the oral examination. Therefore, none of the present plaintiffs could have been adversely affected by the manner of determining the passing score for the test. It also appears that the determination of the passing grade was a result of conversation, based on valid reasons, and did not involve, as many cases do, claims by plaintiffs that they were excluded from consideration because of the passing score. A more inclusive passing score made the scoring less restrictive, not more, thereby increasing the number of capable applicants.
(3) Whether the defendants should have revised the test or conducted a study after the consolidation became known
The plaintiffs further claim that the defendants failed to revise the test after the consolidation became known. Mr. Davey, who prepared and administered the test, did not know that the positions of fire alarm supervisor, lineman and administrative aide had been consolidated with the rank of lieutenant until after the test had been administered. Thus, the test, as administered, did not contemplate that the rank of lieutenant would include the duties of these three specialty positions. Mr. Davey testified that he would not have tested for the specialty functions had he known of the consolidation.
The rank of lieutenant is a company officer whose core duties include supervising crews and performing the functions of fire suppression, research and safety. The plaintiffs do not contest the fact that the examination should be designed to test for "core" or "critical" functions of the position. The three speciality positions involve on the job training and their duties cannot be regarded as either "critical" or "core" functions of a lieutenant. The firefighters handling the duties of the specialty positions could, at the discretion of the chief, be reassigned to line duties at any time.
Based on the foregoing facts, the court finds that there is no compelling reason that a study should have been performed, or that the test should have included questions related to the functions of the specialty positions, after the consolidation.
(4) Whether the changing of test scores after appeals were taken was performed improperly and without the input of subject matter experts
The plaintiffs also claim that after appeals were taken the test scores were changed improperly without the imput of subject matter experts, and that changing the scores of some but not all candidates on a certain question created an inconsistency.
After appeals were taken, Mr. Davey and his associate listened to the tapes of the candidates for the oral examinations. The answers given by the candidates were then compared with the benchmark answers previously prepared with the input of subject matter experts. A determination was then made as to whether the scoring procedures outlined in training were followed. In a small number of cases (18), additional credit was given, but this did not amount to major changes in the scores. The maximum change in any candidate's score was 1.18 points.
Because the change in the scores after the appeals were taken had no prejudicial impact upon the plaintiffs, the court finds that the method of determining the issues raised by the appeals was reasonable. Furthermore, the court agrees with the defendant's argument that changing a candidate's score for a particular question does not require the review of all candidates' answers to that question. Each candidate's answer is unique, and all candidates had the equal right to appeal any answer they felt was incorrectly scored.
(5) Whether the oral portion of the test was improper in that the test did not permit sufficient prompts by the examiners
The plaintiffs claim that the test raters for the oral examination should have been allowed to utilize more "prompts," which are instructions which the raters can use if a candidate is not answering all aspects of the answer. The various guidelines do not specifically address the issue of prompts except to indicate that they should be standardized so that they are the same for all candidates.
The oral test allowed for limited prompts on one question, and, aside from pauses, the only prompt allowed was a closing prompt which involved the statement: "Unless you have more to add, we'll go on to the next question." The limited use of prompts was done with the recognition that there are no prompts in real life. The test was designed for issue spotting. Furthermore, the claims of the plaintiffs do not involve any specific question or any specific prompt but are more generalized in nature. Thus, the court cannot find that the limited use of prompts causes the test to be unfair, or that it is an appropriate reason to invalidate the test.
(6) Whether the test was unfair to the plaintiffs
The plaintiffs also claim that the test was unfair as to them because they had unequal training opportunities as compared with line firefighters who received regular training. The plaintiffs do not provide viable statistical analysis to support that claim, and the available statistical analyses do not support it. The plaintiffs also do not identify any specific question upon which such a claim is based. For example, they have not identified any new firefighting procedures among the benchmarks for the oral questions. On the other hand, the available evidence supports the fact that the benchmarks for the test answers were taken from, and contained in, the materials on the reading list that was provided to all candidates, and that other firefighters have always studied for these tests with the help of these same materials.
While the plaintiffs will lose their current positions in the specialty groups because those functions must now be performed by Lieutenants, that result was caused by the fact of consolidation and not by the test. Issues relating to the consolidation itself have not been presented to the court.
The relief requested by the plaintiffs is, therefore, denied.
RUSH, J.