Opinion
No. 88 CV 3773 (Damages Trial No. 1) No. 88 CV 3773 (Damages Trial No. 2)
May 30, 2002
MEMORANDUM OPINION EXPLAINING THE EXCLUSION OF THE TESTIMONY OF THE CITY'S PROFFERED EXPERT WITNESS DANIEL GARCIA DURING DAMAGES TRIALS NOS. 1 and 2
In the first and second trials on damages in this case, the City proffered the testimony of Daniel Garcia, Ph.D. ("Garcia") "as an expert witness in the field of economies." (2/15/02 Tr. 979). Garcia testified he "was assigned basically to calculate losses for a group of firefighters essentially by estimating how much money they would have made in the absence of the race standardization of the 1986 exam and comparing that with how much money they made." (2/15/02 Tr. 996-97). In testifying, Garcia "assumed that all of them [, the plaintiffs,] absent race standardization of the 1986 exam would have been promoted to lieutenants for sure and then calculated their probabilities of becoming captains through the 1992 exam and their probabilities of becoming battalion chiefs later on." (2/15/02 Tr. 997).
As to the plaintiffs who were denied promotions to the rank of lieutenant in the Chicago Fire Department ("CFD") from the 1986 Lieutenants' Exam ("Exam"), this court excluded Garcia's testimony regarding the probabilities of promotion to the rank of captain and to the rank battalion chief because: (1) Garcia's testimony would not have assisted the trier of fact; (2) Garcia's qualifications were inadequate; and (3) Garcia's testimony was based on a flawed methodology.
I. The Law Regarding Expert Testimony
Federal Rule of Evidence 702 governs the admissibility of expert testimony at trial. In applying Rule 702 law, when a witness is proffered as an expert, the trial court must evaluate initially two issues: (1) whether the witness's proffered testimony "will assist the trier of fact to understand the evidence or to determine a fact in issue," and (2) whether the witness has the pertinent "knowledge, skill, experience, training, or education," to provide that expert testimony. Once those issues have been evaluated, then the court must determine the question of whether the expert's testimony is based on sound methodology. The Seventh Circuit has stated that:
Fed.R.Evid. 702 states:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
"A court has wide discretion in determining the competency of a witness as an expert and the relevancy of his testimony with respect to a particular subject. The district court's decision will be overturned on appeal only if manifestly erroneous."
United States v. Lanzotti, 205 F.3d 951, 956 (7th Cir. 2000);United States v. Stevenson, 6 F.3d 1262, 1266-67 (7th Cir. 1993).
This court, in the exercise of its discretion, considered each of the issues as explained below before excluding Garcia's proposed testimony.
II. Garcia's Testimony Would Not Assist The Trier of Fact
The key jury determination to which Garcia's testimony as to probabilities appeared aimed was the "chance" of promotion each plaintiff lost as a result of the City's discrimination. The application of "lost-chance" theory was approved by the Seventh Circuit in Bishop v. Gainer, 272 F.3d 1009 (7th Cir. 2001), an employment discrimination case. In Bishop, "the judge proceeded to calculate plaintiffs' damages by assessing what the chances were that each would have received the promotion he sought." Id. at 1016. The Seventh Circuit explained the factual details of the case as follows:
For this promotion, Hanford placed third and Robert fourth on the promotion list. The person who was first received a different promotion and the person who placed second had been out of the particular district for several years, and for that reason the judge reasoned that his chances of getting the promotion would be reduced to 25 percent. Then the judge assessed that Hanford had a 45 percent chance and Robert had a 30 percent chance to receive the promotion. The other appellant, Volle, was competing for a promotion with two other white males who placed higher than he did on the list, so his chances were assessed at 15 percent.
The approach obviously involves more art than science. But as we said in Doll, that is true in all comparative negligence calculations as well. It strikes us that in this particular situation, it was the likeliest way to arrive at a just result. We think the judge (Judge Harry Leinenweber here) did a wonderful job of cutting this Gordian knot. We have examined the evidence and find no reason to disturb the thoughtful calculations he has made and the result they have produced.Id. at 1016-17. In approving Judge Leinenweber's approach in Bishop, the Seventh Circuit quoted from its opinion in Doll v. Brown, 75 F.3d 1200, 1207 (7th Cir. 1996) recommending to the bench and the bar that they consider the lost-chance theory in dealing with employment discrimination cases in which the number of available job openings was fewer than the number of candidates seeking those jobs. Bishop, supra, at 1016.
Because the Bishop case involved a bench trial, Judge Leinenweber made the specific, individual determinations as to the lost-chance opportunity each plaintiff had to the job opening at issue in Bishop. In the case at bar, the juries, consistent with Bishop, were asked to make the requisite specific, individual determinations as to the chance of promotion to each CFD higher rank which each plaintiff lost based on the evidence. The juries were also asked to make individual determinations as to the date, if any, on which each plaintiffs subsequent promotion would have occurred absent each plaintiff having been denied the promotion to which he was entitled from the 1986 Exam's unadjusted list. As the Seventh Circuit recognized in Bishop, the task the juries were asked to perform here "involves more art than science." Id. at 1016. This is not to say that the testimony of a qualified expert based on appropriate statistical criteria and using proper scientific methodology could not have assisted the juries in their assigned endeavors to determine the individual percentages of each plaintiff's lost chances. Garcia was not that person. He testified: "I don't think there is a scientific approach to determining individual probabilities." (2/15/02 Tr. 1006).
Since he harbored this expressed belief, Garcia's approach to the juries' tasks of making the required individual determinations of each plaintiff's lost chances was simplistic and misleading: divide the number of jobs by the number of candidates for those jobs. Id. The information as to the number of test-takers and the number of promotions from each CFD promotion test was presented as uncontested evidence. The jurors had all the information and means to perform the simple calculation Garcia used. The jurors did not need expert testimony for that. What Garcia did, however, was to take that simple calculation of job openings divided by applicants and assign to each plaintiff, indeed each test-taker, the same, identical "average" lost-chance probability without consideration of any individual factors. Garcia testified he "assumed they ["the plaintiffs,] would have had the average probability of promotion." (2/15/02 Tr. 1007). He further testified he did not figure the individual probability of each of the plaintiffs because: "I didn't have enough information to do that in a reliable manner." (2/15/02 Tr. 1008). By assigning the identical, "average" probability of promotion to each test-taker, even those who historically scored high and then were promoted, Garcia ignored Judge Leinenweber's approach in Bishop approved by the Seventh Circuit of using the qualities and circumstances of each applicant whose lost-chance was to be determined. Consequently, Garcia's testimony which the court excluded would have mislead the juries into believing, contrary to Bishop that each plaintiff's individual qualities and circumstances could be ignored under the law. That would not have aided the juries in understanding the evidence presented at the damages trials and would not have aided the juries' determinations of the specific facts that the verdict forms called upon them to find, see Dhillon v. Crown Controls Corp., 269 F.3d 865, 871 (7th Cir. 2001).
The City's counsel voiced objections on the basis of Federal Rule of Evidence 403, which the court sustained, when plaintiffs' counsel sought to introduce similar evidence (2/15/02 Tr. 855-62) through Dr. Goldman, whose expert qualifications where neither questioned nor objected to by the City. (2/15/02 Tr. 818).
III. Garcia's Qualifications Were Inadequate
Garcia received his bachelor's and master's degrees in his homeland of Argentina. In 1988, when this case was filed, Garcia was teaching Argentine history at the Buenos Aires Bible Institute and at another school in Argentina. (2/15/02 Tr. 975). Garcia's focus remained on Argentina while he was at the University of Chicago. Eight of the nine "Working Papers," which he wrote during 1997 and 1998, while he was working on his Ph.D. degree, deal with Argentina and Latin America. (Damages Trial No. 2, Final Pretrial Order ("FPTO") p. 64-65). Garcia admitted that his Ph.D. thesis "focused specifically on Argentina" and its labor market relating to issues that come up in international trade on "relative wages." (2/15/02 Tr. 980-81). That thesis document was never published and has not been the subject of peer review beyond the Ph.D. process. According to his curriculum vitae, the only publication Garcia ever completed was a chapter in a book on comparative economic development in North and South America. (2/15/02 Tr. 981).
In addition to Garcia's curriculum vitae, the City submitted a sworn declaration from Garcia regarding his credentials. (Damages Trial No. 2, FPTO p. 67-72). In that declaration, Garcia stated that among other courses, he took courses in mathematics and econometrics at the University of Chicago, as well as "two other courses that focused on the application of statistics and probability models to the analysis of labor market issues and labor market data." (Id. at p. 2, para. 3). Garcia also noted in his declaration that certain of his courses were taught by some of the Nobel Prize recipients on the faculty at the University of Chicago. He also stated that his Ph.D. courses on labor economics "focused on the U.S. labor market." (Damages Trial No. 2, FPTO p. 68). Merely taking courses from experts, even Nobel laureates, does not make a person a qualified expert witness. Garcia, himself, demonstrated no expert experience regarding the calculation of damages like those the juries and this court had to determine in this case.
In looking at his experience, Garcia has never taught any courses in statistics, probability, or mathematics at any educational level. He never has published any writings on those topics. Garcia has never been accepted as a expert witness by any court, state and federal. Although this court realizes there has to be a first time for every expert witness, that first time should occur when the proffered expert has the requisite "knowledge, skill, experience, training, or education" as required by Federal Rule of Evidence 702.
Garcia did testify at a state administrative hearing in November 2001. He, however, could not remember the name of the administrative body. (2/15/02 Tr. 983).
Because Garcia's formal higher education had such a limited focus, and because Garcia had no demonstrated experience as an expert witness, the question became: what knowledge did Garcia have of the general subject matter about which he was asked to opine? The answer to that question is that he does not have very much. Garcia lacked an understanding of basic concepts used by the City in making CFD promotional decisions over the years about which Garcia was asked to opine. Garcia testified, though he may have heard the phrase "standard error of measurement" used in connection with testing, he did not understand what it meant. (2/19/02 Tr. 1173). It is not disputed that the City used "standard error of measurement" in making CFD promotional decisions during the years about which Garcia opined. Moreover, Garcia did not understand the concept of"banding," (2/19/02 Tr. 1173-75), which the City also used in making numerous promotional decisions, including decisions regarding promotions from the 1993 Lieutenant's exam. (Joint Exhibit 3). In short, Garcia demonstrated inadequate knowledge and skill as to the subject matter about which he was opining. Accordingly, the court determined, Garcia's demonstrated knowledge, skill, experience, training, or education was insufficient to qualify him as an expert witness to testify beyond the areas he was allowed to testify in this case.
"The term 'SEM', standard error of measurement, is the degree of error for a test and is used to establish a range of scores in which an examinee's true score likely falls." Chicago FireFighters Union Local No. 2 v. City of Chicago, 1999 WL 1289125 *76-77 (N.D. Ill.), aff'd., 249 F.3d 649 (7th Cir.) cert. denied, 122 S.Ct. 465 (2001).
IV. Garcia's Testimony Was Not Based On Sound Methodoloy
In addition to Garcia's other infirmities, the methodology he employed was not sound. Rule 702, enumerates, among other things, three requirements of expert testimony: (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case. These three requirements, added to Rule 702 in December 2000, followed the trilogy of cases,Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993),General Electric Co. v. Joiner, 522 U.S. 136 (1997), and Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999), in which the United States Supreme Court clarified the law regarding the admissibility of expert testimony. As explained in the Advisory Committee Note which accompanied the amendment addressing the enumerated criteria: "The amendment affirms the trial court's role as gatekeeper and provides some general standards that the trial court must use to assess the reliability and helpfulness of proffered expert testimony." Fed.R.Evid. 702, Advisory Committee Note, 2000 amendments.
In determining whether particular expert testimony is grounded in reliable methodology, the Supreme Court in Daubert offered a non-exclusive list of factors: (1) whether the proffered theory can be and has been tested; (2) whether the theory has been subjected to peer review; (3) whether the theory has been evaluated in light of potential rates of error; and (4) whether the theory has been accepted in the relevant community. Id. at 593- 94. of course, the Daubert opinion's non-exclusive criteria were meant to provide a flexible test for trial courts with no single factor being dispositive, Kumho Tire at 526 U.S. at 151- 52, but Garcia's testimony met none of these criteria.
Garcia's approach in arriving at his purported expert testimony was substantially flawed because: (1) Garcia's testimony was not based upon sufficient facts or data and (2) Garcia's testimony was not the product of reliable principles and methods.
(1) Garcia's Testimony Was Not Based Upon on Sufficient Facts or Data
During voir dire, Garcia acknowledged that the groups of persons on the top of the 1986 Exam list did better on the 1992 captain's exam (2/19/02 Tr. 1187) than those who ranked lower. It was undisputed that the plaintiffs in Damages Trial Nos. 1 and 2 ranked in the top 10% to 12% on the unadjusted promotional list (Joint Exhibit 17) from the 1986 Exam. Garcia. however, refused to consider the fact of the plaintiffs' high ranking on the 1986 Exam and did not attempt to consider other facts regarding the individual plaintiffs in formulating his opinions. (2/15/02 Tr. 1169-70). Despite turning a blind eye to the facts available to him, Garcia testified: "I didn't have enough information to incorporate those individual probabilities within a scientific approach." (2/15/02 Tr. 1031). Consequently, Garcia treated all the test-takers on each of the CFD promotional exams the same by assigning each of them the identical, average probability of promotion from whatever CFD exam about which Garcia was opining. (2/15/02 Tr. 1007).
In assigning each plaintiff the same average probability of promotion from the 1992 captain's exam, Garcia divided the 136 promotions made from the 1992 captain's exam by the 528 people who took that exam, 136/528=.26, (2/15/02 Tr. 1021), and assigned each of the "plaintiffs a 26 percent chance of becoming captains through that exam." (2/15/02 Tr. 1023). Following that same pattern, Garcia assigned the same, average probability as all other test-takers to each plaintiff, for the 1994 and 1998 battalion chiefs' exams (2/15/02 Tr. 1032-33). As to the 1994 battalion chiefs exam, he divided the 54 promotions from that exam by the 137 people who took that exam (54/137 =.41) and assigned a 41% probability of becoming a battalion chief to each of the plaintiffs. (2/15/02 Tr. 1024).
Although Garcia testified that he "took many factors that were specified to each individual into account" (2/15/02 Tr. 997), he did not. Garcia's theory was to lump together all the takers of each CFD promotional exam as having the same chance of promotion. No objective or quantifiable qualities possessed by any individual candidate or any identifiable group of candidates were taken into account. (2/15/02 Tr. 1169-70).
It is uncontested that the City had two promotional categories in the CFD at the time of the promotions from the 1986 Exam: one category for minorities, and another category for everyone else. It is also uncontested that the promotions made from the CFD promotional exams after the 1986 Exam demonstrated that the City repeatedly promoted minorities over higher-scoring non-minorities on the basis of race. Yet, Garcia rejected this relevant data and applied a blanket analysis to all the test-takers, regardless of race or any other factor. Garcia did not take into account that the plaintiffs who had not been promoted from the 1986 Exam in time to take the 1992 captain's exam actually had scored higher and ranked higher on the unadjusted promotional list than the black and Hispanic CFD personnel promoted to lieutenant ahead of the plaintiffs. Garcia also admitted that his conclusions regarding homogeneity of the 1986 Exam based on performance on subsequent exams did not consider the effect of the delay in promotion on certain 1986 Exam test-takers due to the City's racial discrimination challenged by this suit. (2/19/02 Tr. 1193-97).
Because Garcia rejected data regarding the people who were promoted from the 1986 Exam, and because he refused to consider any relevant, proven facts distinguishing the various takers of the 1992 captain's exam and the 1994 and 1998 battalion chiefs' exams, Garcia's testimony was not based on sufficient facts or data. Even he admitted, he believed he did not consider enough information to opine as to individual probabilities. (2/15/02 Tr. 1031).
(2) Garcia's Testimony Was Not The Product of Reliable Principles And Methods
By not employing the proper methodology for determining the population from which to calculate each plaintiff's probability of promotion, and by not calculating a standard deviation to assess the accuracy and reliability of his calculations, Garcia's testimony was not the product of reliable methods. Garcia's methodology did not comport with any of the four non-exclusive factors listed in the Daubert opinion. Daubert, 509 U.S. at 593-94. Specifically, there was no proof that Garcia's approach had been tested or accepted in the relevant community. Additionally, Garcia's methodology supporting his opinion in this case has never been subjected to peer review, except for the review of Dr. Jerry Goldman, plaintiffs' expert, who criticized Garcia's methodology substantially.
Dr. Goldman's qualifications as an expert witness in mathematics and statistics were not disputed by the City. Dr. Goldman credibly explained that Garcia's methodology was fundamentally flawed because, in addition to Garcia's failure to select the appropriate population for his probability calculations, Garcia did not include any measure of accuracy in terms of standard deviation for his computations. (2/19/02 Tr. 1221-24).
Since the supposed objective of Garcia's testimony was to help the juries determine each plaintiff's probability of success, proper methodology required looking to the success achieved on the subsequent exams by the population of CFD personnel who had been promoted from the 1986 Exam. Garcia, however, chose to use as his comparison population, the total population of all persons taking the subsequent exams, a population he conceded, was less successful than the 1986 Exam promotees and certainly less homogeneous. (2/19/02 Tr. 1161-1163). Garcia chose the less homogeneous, less successful population for his calculations, presumably because it favored the City's position not because it was the proper methodology. To allow an opinion based on such an inappropriate and flawed methodology to be presented to the juries would have been to present to the juries an unreliable predictor of each plaintiff's lost-chances for promotion from each of the post-1986 Exam CFD promotional exams.
The City's counsel argued that Garcia's approach has been endorsed by other courts, (City's Rule 59 memorandum p. 3), citing Griffin v. Michigan Department of Corrections, 5 F.3d 186 (6th Cir 1993). The Sixth Circuit in Griffin, however, did not endorse the methodology used in Garcia's probability model of the "average worker." Rather, the Sixth Circuit expressed caution about using such a theory where advancement is the result of "the difficult traversing of the shoals and channels, pools and narrows, of a bureaucracy." The Sixth Circuit said that awarding damages based on the average person s career path "is a highly suspect enterprise, if other methods are available." Id. at 190. Garcia's methodology which dictates that the probability of success for each individual in a group of test-takers is limited to the average success of the whole group of test-takers is not consistent with the methodology endorsed in the Griffin case by the Sixth Circuit, and is not consistent with the approach approved by the Seventh Circuit in the Bishop case as discussed earlier in this opinion.
CONCLUSION
For the above stated reasons and those previously articulated on the record at Damages Trials Nos. 1 and 2, this court excluded the testimony of Daniel Garcia as an expert witness at those trials in this case.