Summary
holding that direct supervision of all work conducted by staff was not necessary, and it was permissible for an expert statistician to rely on analysis from computer code not personally written where expert would be able to tell from examining the analysis whether the code contained any errors
Summary of this case from Bouygues Telecom, S.A. v. TekelecOpinion
Civil Action No. 01-0510 (ESH).
December 20, 2004.
Kerry Alan Scanlon, David Laurent Cousineau, Karen Rae Robinson, Nicole Jelani Becton, Kaye Scholer LLP, Washington, DC, for Plaintiffs.
Barbara Louise Johnson, Paul, Hastings, Janofsky Walker, LLP, David R. Warner, Venable, LLP, Washington, DC, George W. Johnston, Todd J. Horn, Venable, Baetjer Howard, L.L.P., Baltimore, MD, Nancy E. Rafuse, Ashe Rafuse, LLP, William B. Hill, Jr., Paul Hastings Janofsky Walker, LLP, Atlanta, GA, for Defendant.
MEMORANDUM OPINION
Before the Court is defendant's motion to exclude the testimony of plaintiffs' statistical expert pursuant to Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). Sodexho contends that Dr. Bernard Siskin's reports are both structurally flawed and "riddled with errors," he knows too little about defendant's operations to analyze them competently, he failed to control for major factors in performing his analyses, he destroyed documents underlying his analyses that should have been disclosed to defendant, he lacked the competence to perform certain work, and his segregation analysis is not reliable. More generally, defendant's criticism is that Siskin is untrustworthy and so careless that, regardless of his extensive experience and his impressive credentials, his reports are unreliable.
Defendant's contention that Siskin's testimony must be rejected because he did not consider the major, nondiscriminatory variables (education and experience) that purportedly played a role in the promotion decisions at issue (Def.'s Mot. at 26) goes to the heart of defendant's summary judgment motion, and therefore, it will not be addressed here, but is considered in the Court's Memorandum Opinion relating to that motion.
In essence, the parties invite this Court to become enmeshed in a classic "battle of the experts," but courts are well advised to avoid such a role, absent a showing that the challenged evidence will prove either unreliable or unhelpful to the trier of fact. See, e.g., Dukes v. Wal-Mart, Inc., 222 F.R.D. 189, 191 (N.D. Cal. 2004). While the Court has considered each of defendant's claims, it concludes that the parties have each hired competent, well-paid professional experts who have used similar methodologies, but have reached different results regarding the rate of promotions of African Americans at Sodexho. Although other statisticians might well differ with the parties' experts over various details of their analyses, in the end the Court concludes that Dr. Siskin's work does not fail the test set forth in Daubert.
ANALYSIS
I. Legal Standard Daubert made clear that expert testimony should not be considered in a case unless the expert has genuine expertise and the testimony will assist the trier of fact to understand or determine a fact in issue. 509 U.S. at 592. The twin requirements for expert testimony are relevance and reliability. Kumho Tire Co. v. Carmichael, 526 U.S. 137, 149-50 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999). In determining whether the testimony is based on the expert's "scientific knowledge," the Court must "focus on `principles and methodology, not on the conclusions that they generate.'" Meister v. Med. Eng'g Co., 267 F.3d 1123, 1126-27 (D.C. Cir. 2001) (quoting Daubert, 509 U.S. at 595 113 S.Ct. 2786).If the Court finds Siskin's opinions to be clearly unreliable, it may disregard his reports in deciding whether plaintiffs have created a genuine issue of material fact. Muñeoz v. Orr, 200 F.3d 291, 301 (5th Cir. 2000). The Court thus exercises a "gatekeeping function" by examining the expert's qualifications, the methodologies used, and the relevance of the final results to the issues confronting the jury. Adams v. Ameritech Servs., Inc., 231 F.3d 414, 423 (7th Cir. 2000).
However, "the question before [the Court] is not whether the reports proffered by plaintiffs prove the entire case; it is whether they were prepared in a reliable and statistically sound way, such that they contained relevant evidence that a trier of fact would have been entitled to consider." Id. at 425. "No one piece of evidence has to prove every element of the plaintiffs' case; it need only make the existence of ` any fact that is of consequence' more or less probable." Id. (citing Fed.R.Evid. 401). Thus, it may be the case that although the expert's analysis is admissible, it is nonetheless insufficient to establish a prima facie case of discrimination. See, e.g., Scales v. George Wash. Univ., No. 89-0796, 1993 WL 304016, at *6 (D.D.C. July 27, 1993) (while expert's testimony and exhibits were "admitted for what they are worth," they probably did not constitute reliable evidence sufficient to establish a prima facie case of discrimination).
The party offering the expert's testimony must establish by a preponderance of the evidence that the expert testimony is admissible and that the expert is qualified. See Meister, 267 F.3d at 1127 n. 9 (citing Fed.R.Evid. 104(a) and Daubert, 509 U.S. at 592 n. 10); see also Allison v. McGhan Med. Corp., 184 F.3d 1300, 1312 (11th Cir. 1999) ("[T]he proponent of the testimony does not have the burden of proving that it is scientifically correct, but that by a preponderance of the evidence, it is reliable."). On the other hand, in making the preponderanceof-the-evidence reliability determination, the question is "not whether [the Court] find[s] one set of expert reports more persuasive than another. It is whether, taking the facts in the light most favorable to the plaintiffs, a trier of fact should be permitted to make that choice." Adams, 231 F.3d at 425 (citations omitted) (emphasis added). Thus, the question before the Court is not which of the parties' experts is "right," but rather whether defendant's criticisms of Siskin's analysis affect its admissibility, or only its "probativeness" or weight. Id. (quoting Bazemore v. Friday, 478 U.S. 385, 400 (1986)). If the latter, then "our adversary system provides the necessary tools for challenging reliable, albeit debatable, expert testimony." Perkins v. Origin Medsystems, Inc., 299 F. Supp. 2d 45, 54 (D. Conn. 2004) (quoting Amorgianos v. Nat'l R.R. Passenger Corp., 303 F.3d 256, 267 (2d Cir. 2002)) (internal quotation marks omitted).
II. Defendant's Criticisms of Siskin and His Work
A. Siskin's Qualifications to Testify About the Analyses of Data
Defendant claims that Siskin's "unhesitating acceptance" of the data provided to him and the analyses done by his staff "violate the principles of the scientific method," and on that basis, he must be excluded as lacking the requisite level of reliability. (Def.'s Mot. at 9.) Specifically, defendant argues:
1. Siskin is not an expert in computer programming ( see id. at 5 (citing Def.'s Ex. A1 [Siskin dep.] at 134)), and he allegedly "cannot review any computer program language for accuracy." (Def.'s Reply at 5.) Instead, upon receiving the electronic data from Sodexho, "someone other than Siskin created and ran computer programs, which in turn generated printouts for Siskin to review." (Def.'s Mot. at 7.) "Shockingly," Siskin relied on an online discussion in deciding how to structure part of an analysis using the STATA statistical program, and allegedly no one on his team is sufficiently familiar with STATA, the program used to analyze the stratified sample data in the logistic regressions. ( Id. at 9; Def.'s Reply at 7 n. 5.)
2. Siskin did not determine whether the programmers conducted the analyses he requested and cannot attest to the accuracy of the computer output upon which he relied in drafting his reports. (Def's Mot. at 7-8 (citing his testimony that "hopefully his staff did it correctly," and "the other side will . . . make all the corrections and pick up any errors that occur.")) Moreover, he relied on analyses generated by "unidentified programmers." ( Id. at 9.)
Essentially, defendant argues that Siskin's reports lack foundation and are unreliable. But defendant's protestations amount to disputed factual issues, which are insufficient as a matter of law to warrant his exclusion. For instance, Siskin testified that, although he is not familiar with STATA programming, he is familiar with the techniques used in structuring an analysis for the program, and the results were always reviewed for errors after running a program. (Def.'s Ex. A1 [Siskin dep.] at 134-36.) As plaintiffs point out, many senior statisticians design tests, but for various reasons — including costs to the client — do not personally run them, but instead rely on their assistants to do so, reviewing their output to ensure that the test was properly conducted. (Pls.' Opp. at 10.) ( See also Pls.' Ex. F (Haworth dep. at 16 in Bryant v. George Wash. Univ., No. 94-5522 (D.D.C. Mar. 21, 1995)) ("A: [U]sually I will work with one of the three senior economists to help get the work done in a reasonable cost effective way. Q: Do you have people to do programming for you? A: Yes."); Pls.' Ex. H (Haworth dep. in instant case at 56 ("It probably would be more efficient to have someone else [check back-up data] because it's been a long time since I programmed.")); see generally Pls.' Exs. G H (naming various programmers and assistants on whom Haworth relies).)
As a Ph. D. statistician, past chairman of the Temple University Department of Statistics, and an expert with decades of experience performing analyses such as those at issue in the instant case ( see Pls.' Ex. A(2) at ¶ 1), Siskin need not personally be an expert in STATA in order to be a qualified expert under Daubert. Rather, pursuant to Fed.R.Evid. 703, an expert may rely on any facts or data "of a type reasonably relied upon by experts in the particular field," including facts, data, and opinions that are otherwise inadmissible. This includes relying on one's assistants to carry out analyses that the expert designed. See Astra Aktiebolag v. Andrx Pharmaceuticals, Inc., 222 F. Supp. 2d 423, 492 (S.D.N.Y. 2002) (there is no requirement that an expert must run his own tests); see also Dura Automotive Sys. of Ind., Inc. v. CTS Corp., 285 F.3d 609, 612 (7th Cir. 2002) ("An expert witness is permitted to use assistants in formulating his expert opinion."). Here, it is clear that Siskin, like defendant's statistical expert (Dr. Joan Haworth) and other expert statisticians, does not need to personally write computer code in order for a resulting analysis to be admissible. The Court has no reason to doubt Siskin's statement that "[w]hen I review the computer output I am able to determine if the programming was performed as I requested and if any significant mistakes were made in the programming." (Pls.' Ex. A [5/04 Siskin decl.] at ¶ 5). Thus, even if his staff members are likewise not masters of the STATA programming language, the issue of their expertise and of Siskin's ability to learn if the programming underlying an analysis contains a serious error is for the jury and is not an issue to be resolved under Daubert.
Defendant cites Dura for the proposition that the expert must be able to testify that he "supervised his staff carefully," claiming that Siskin cannot and did not do so here. Dura, however, does not counsel exclusion of the expert if the expert cannot (or did not) appropriately supervise the work of his assistants. Instead, the proper approach is to question the expert regarding his reliance on the assistant's work. "The opposing party can depose them in order to make sure they performed their tasks competently; and the expert witness can be asked at his deposition whether he supervised them carefully and whether his relying on their assistance was standard practice in his field." Dura, 285 F.3d at 613.
In Derrickson v. Circuit City Stores, Inc., No. 95-3296, 1999 U.S. Dist. LEXIS 21100, at *20 (D. Md. Mar. 19, 1999), the court rejected a challenge similar to the one at issue here. Defendant moved to exclude plaintiffs' expert on the basis that he had submitted a report in a discrimination case based upon analyses conducted by his assistant.
In his report, Dr. Medoff presented various tables regarding compensation and promotions at Circuit City, and he based his opinion in part on the information in those tables. That information, however, is not the raw data produced by Circuit City. Instead, the tables are based on data that was subjected to various selection, aggregation and weighting processes performed by Dr. Medoff's assistant. Dr. Medoff testified at deposition that he told his assistant in general terms what manipulations and analyses he wanted performed on the data. His assistant then wrote a series of computer instructions using a commercially available statistics program that analyzed the data and produced the tables upon which Dr. Medoff relied.Id. at *17-18. The court nonetheless held that while the defendant is entitled to know what the expert's assistant did in order to "properly prepare to cross-examine" him, the testimony was not excludable. Id. at *18-20.
Furthermore, defendant's reliance upon Washington v. Vogel, 880 F. Supp. 1545 (M.D. Fla. 1995), is misplaced. There, the court excluded the testimony of an expert who intended to testify about a study conducted by someone else, when the expert had not independently verified the study. Id. at 1547-48. Here, the facts are very different — Siskin created and orchestrated the analysis; he and his assistants "worked hand-in-glove, and the fruits of their labor are indivisible." Derrickson, 1999 U.S. Dist. LEXIS 21100, at *18. In sum, Siskin's analyses are sufficiently reliable, notwithstanding his reliance on his assistants, especially given the fact that collaboration, such as occurred here, is typical of experts (including Haworth) in his field. See Fed.R.Evid. 703.
B. Siskin's Knowledge About Sodexho and the Lack of Independent Verification of Sodexho's Data
Defendant argues that Siskin did not conduct independent investigations and does not know how different divisions in the company function, which purported failures "diminish unacceptably" the reliability of his opinions. (Def.'s Mot. at 22, 26.) Defendant also contends that Siskin failed to independently verify the data that defendant supplied him. ( Id. at 6.) Plaintiffs respond that Siskin's analyses did not require intimate familiarity with the inner workings of Sodexho, because as a statistician, his role was not to interview individual Sodexho employees to determine what they considered to be a promotion, but rather his job was to look to the company's own internal data in determining what constituted a promotion. (Pls.' Opp. at 12.) Plaintiffs further note that Haworth and Siskin both relied on basically the same data, such as Sodexho's MARRPAY database, which was used to analyze promotions and which was "the most accurate and reliable data available." ( Id. at 11 n. 8, 12.)
It is also noteworthy that Siskin did in fact interview several present and former Sodexho employees regarding defendant's reliance on RVP codes. ( See Pls.' Opp. to Def.'s S.J. Mot. Ex. 158 [5/04 Siskin decl.] at ¶¶ 7-10).
The Federal Rules of Evidence specifically provide that an expert may rely on facts or data "perceived by or made known to the expert at or before the hearing." Fed.R.Evid. 703. See also Gussack Realty Co. v. Xerox Corp., 224 F.3d 85, 94-95 (2d Cir. 2000) ("[A]n expert may rely on data that she did not personally collect."). Cases that discount one party's expert analysis for failing to examine adequately the underlying evidence generally note that this failure is traceable to the fact that the data analyzed came from that same party. See, e.g., Contractors Ass'n of Eastern Pa., Inc. v. City of Philadelphia, 893 F.Supp. 419, 437 (E.D. Pa. 1995); Muñoz, 200 F.3d at 301-02. Here, by contrast, Siskin reasonably and necessarily had to rely on data produced by defendant. See Lewis v. Booz-Allen Hamilton, Inc., 150 F. Supp. 2d 81, 92 (D.D.C. 2001) ("The court recognizes that the defendant's own affirmative-action data is a reliable source from which [plaintiffs' expert] conducted his examinations."); Adams, 231 F.3d at 427 ("The underlying information about the RIF [reduction-in-force] came from defendants ultimately, and as such we see no problem in [the expert]'s decision to rely on it."). See also Dukes v. Wal-Mart, Inc., 222 F.R.D. at 196-98 (rejecting as unreliable a "survey" conducted by defendant's attorneys, which Haworth (defendant's expert in both that case and this one) used in her regression analysis despite her admission that the data collected did not qualify as a valid survey because it was not collected in an anonymous and neutral manner). Thus, Siskin's failure to independently verify Sodexho's data raises no barrier to the admissibility of his analyses.
Furthermore, the cases defendant cites to support its contention that Siskin's "ignorance" about the company are inapposite. As a statistician working with company-supplied data, Siskin does not need an understanding of "what the laundry operation does or how it conducts its operations." (Def.'s Mot. at 22 n. 7.)
C. Purported Errors in Siskin's Reports
Defendant claims that Siskin frequently relied upon analytic output generated from the wrong data files (Def.'s Mot. at 10), and his various reports are so riddled with errors as to render them unreliable. ( Id. at 11-19.) In his declaration, Siskin rebuts each alleged "error," and according to plaintiffs, the supposed errors are, in many cases, not mistakes at all, but rather misrepresentations of the record; some of the supposed "errors" were corrected in subsequent reports; and even where flaws may arguably exist, they are minor in nature and do not affect Siskin's conclusions. ( See Pls.' Opp. at 19-24 ("the few times that mistakes were made in the over 200 pages of reports and supporting tables that Dr. Siskin submitted in this case, they were of a minor nature, and he often corrected them and resubmitted the corrected versions to defendant").) For example, defendant alleges that Siskin "`made up' promotions that did not actually exist at Sodexho in order to gain a significant outcome." (Def.'s Mot. at 13.) Plaintiffs (and Siskin) rebut this charge by explaining that in fact Siskin was making an assumption to test an hypothesis and was not asserting facts, and he identifies the hypothesis as such in his report. (Pls.' Opp. at 22; Pls.' Ex. A [5/04 Siskin decl.] at ¶ 13.)
The Court notes that — as is probably inevitable in a case as complicated and based on so much data and different analyses as this one — Haworth also apparently made errors. ( See, e.g., Def.'s S.J. Mot. Ex. 45 [12/03 Siskin report] at 11 n. 6.)
The gist of defendant's argument is that Siskin's work product is so bad that his testimony should be excluded. ( See Def.'s Mot. at 3 ("His failure to carefully review the analytic output provided to him combined with his repeated production of incorrect information in this case should cause the Court to conclude that all of his work in this case is inherently unreliable and as such inadmissible.").) However, beyond enumerating alleged errors, defendant fails to explain how these errors had any substantial bearing on the reliability of his reports, particularly when most purported mistakes were not errors to begin with or were contained only in early reports that were subsequently corrected or clarified. Absent such a showing of materiality, and in light of Siskin's reasonable explanations for many of the purported deficiencies, the Court is unprepared to exclude his testimony on this basis. Further, the fact that Siskin was open to and in fact did correct deficiencies in his preliminary reports argues for the reliability of his testimony, not for its exclusion.
Moreover, insofar as many of Siskin's "errors" result from a different understanding of the facts, his methodological decisions are not properly the subject of a Daubert motion. Instead, such issues are addressed in the Court's accompanying summary judgment Memorandum Opinion, and as held there, they may ultimately need to be resolved by the finder of fact in determining what weight to give Siskin's analysis. See Adams, 231 F.3d at 427-28.
It is clear that the Court exercises a "gatekeeper" function, see Gen. Elec. Co. v. Joiner, 522 U.S. 136, 142 (1997), and may keep out scientific evidence that it deems irrelevant because the basis of the expert opinion is clearly unreliable. Muñoz, 200 F.3d at 301 (citing Daubert, 509 U.S. at 596). But the purported errors about which defendant complains simply do not rise to the level of incompetence that the Fifth Circuit cited in Muñoz as the basis for rejecting a statistician's testimony. See id. at 301-02 (rejecting plaintiffs' expert statistician's testimony, based on the cumulative effect of the flaws, that relied on unexplained tables that added to far less than 100% and an analytical structure that evidenced clear bias and that was inconsistent with the norms of the field; the failure to conduct a multivariable regression analysis; the failure to account for major variables; and reliance on plaintiffs' unverified compilation of the data). Unlike Muñoz, where despite numerous revisions of the expert's report and extensive deposition testimony that failed to account for the errors ( see id. at 302 n. 4), the purported flaws in Siskin's analyses do not, in their totality, cast doubt on the correctness of his methodologies or materially affect the results of his analyses. Compare also Lewis, 150 F. Supp. 2d at 92 (deeming inadmissible promotion tests, where the initial tests contained an error that affected the results, and where correcting the error rendered the results not statistically significant).
Moreover, focusing on errors Siskin may have made earlier during years of data gathering and refining of analyses misses the mark, for plaintiffs have shown that the end products on which they now rely meet the Daubert standard. Neither the record nor the cases cited by defendant support the contention that Siskin's testimony should be excluded because, even though his final products are reliable, his alleged carelessness earlier in the process disqualifies him as an expert.
In sum, any errors in Siskin's analysis do not rise to the level of warranting exclusion, but rather bear only on the probativeness of his reports and his credibility. Despite defendant's arguments to the contrary, its contention that Siskin's statistical methodology is "so flawed" that it should be deemed inadmissible as a matter of law is rejected; the issue is one of the evidence's weight, not its admissibility. See Diehl v. Xerox Corp., 933 F. Supp. 1157, 1167 (W.D.N.Y. 1996); see also Murphy v. General Elec. Co., 245 F. Supp. 2d 459, 467 n. 4 (N.D.N.Y. 2003) (in a disparate treatment claim, failure to perform a multiple regression analysis is not a basis for Daubert exclusion of evidence, but instead goes to the weight of the evidence).
Similarly, the Court rejects defendant's contention that Siskin's testimony must be excluded because he is untrustworthy, as allegedly evidenced by his "willing[ness] to mislead the Court." (Tr. at 6.) Defendant maintains that, regardless of Siskin's experience and expertise, contradictions between his depositions and declarations show that he would also be willing to mislead a jury. ( Id. at 3, 6.) The Court notes as a threshold matter that the purported inconsistencies and misstatements are not in fact necessarily contradictory or false. ( See, e.g., id. at 6 (criticizing Siskin for stating in his deposition that he could not "do a good job" of reviewing computer programs to ascertain whether the programming was proper, but stating in his subsequent declaration that he can determine from the resultant computer output whether the underlying programming was flawed).) Further, these are the type of criticisms that go to credibility, rather than Daubert's standard of admissibility, particularly where defendant is unable to show that any such alleged misstatements and mistakes have affected the bottom-line reliability of Siskin's analyses and testimony.
D. Siskin's Definition of Promotion Within Sodexho
Defendant contends that Siskin improperly defined promotion in generating data for his analyses, and that because his definition does not comport with the evidence, his analyses must be rejected. (Def.'s Mot. at 30, citing, inter alia, McCleskey v. Kemp, 481 U.S. 279, 288 n. 6, 107 S.Ct. 1756, 95 L.Ed.2d 262 (1987).) Defendant sees Siskin's definition of promotion as both a moving target, and as flatly incorrect. But as with defendant's complaints about Siskin's purported errors, defendant does not explain the relevance of this purported deficiency by showing how the change in definitions materially affected Siskin's results.
Moreover, many of the supposed flaws are rebutted by Siskin or were addressed by his subsequent reports. For instance, defendant argues that Siskin did not include certain job moves that individual plaintiffs claim should have been analyzed as promotions. (Def.'s Mot. at 29.) As such, according to defendant, his analysis is based on assumptions that do not match the evidence in the case. In response, plaintiffs explain that Siskin did include the very promotions that defendant identifies. (Pls.' Opp. at 15.) Plaintiffs further argue that Siskin properly arrived at a consistent definition of promotion for statistical purposes instead of relying upon what a particular individual considered to be a promotion. ( Id. at 16.)
Defendant also claims that in Siskin's October 2001 report, he failed in some instances to use the Sodexho MARRPAY computer database's definition of promotion (certain job moves coded as 3 or 4). (Def.'s Mot. at 31.) As shown by plaintiffs, whether he failed to look at the two codes is irrelevant, because, based on those codes, only nine promotions were included in Haworth's data that were not in Siskin's. (Pls.' Opp. at 15.) Defendant likewise criticizes Siskin for not including as a promotion any increase in grade, but rather identifying a promotion only where an employee's job title changed. (Def.'s Mot. at 30-31.) But this is a mischaracterization — Siskin's analysis was based on occupation codes, not job titles. (Pls.' Opp. at 15.) Furthermore, defendant contends that Siskin failed to count many promotions because he created files by identifying employees present at work at the beginning of each year, and as a result, those employees who had been promoted but then left before the beginning of the next year were not included. (Def.'s Mot. at 31.) Additionally, if an employee had been promoted more than once in a year, she would be counted as a single promotion. ( Id.) But these contentions are largely irrelevant because they ignore the fact that it was only Siskin's first report that relied on yearly data, and by the time of his November 2001 report, the data included month-to-month changes. (Pls.' Opp. at 16.)
Generally, plaintiffs explain the inconsistencies identified by defendant by reference to the fact that Siskin's initial October 2001 report analyzed data available to him at that time, and that after he received new files from defendant containing more complete data, his report the next month took that data (Market Reference Rates ("MRRs")) into account, confirming the promotion shortfall he had initially observed. ( Id.) Thereafter, his September 2003 report was again refined to respond to additional evidence that had not been in Sodexho's computer records but which Haworth had pulled from personnel files. As contended by plaintiffs, "Dr. Haworth's analysis unjustifiably included all of these undocumented events as promotions, and Dr. Siskin responded by showing that a much more reasonable adjustment to account for these events would still have shown promotion differences that are statistically significantly adverse." ( Id.) Thus, he was "not changing his definition of promotion but simply demonstrating that even a liberal interpretation of these undocumented events as representing promotions would not alter his findings in this case." ( Id. at 16-17.)
More specifically, plaintiffs have proffered evidence to raise substantial factual issues as to whether defendant's expert correctly defined promotion for purposes of this litigation, thereby precluding any finding that Siskin's definition is "wrong." ( Id. at 14-15.) The term "promotion" is defined in Sodexho's official personnel manuals to mean either a move from one grade or band to the next higher grade or band, or, more ambiguously, "career moves" to positions within the same band but with higher MRRs. Plaintiffs (and Siskin) think that the most accurate way of capturing this definition when working with Sodexho's computerized MARRPAY data is to define a promotion as an increase in grade or band, or a change in job or unit with either a salary change code of 3 or 4 or an increase in MRR. Plaintiffs contest Haworth's definition of promotion on the grounds that it does not mirror Sodexho's actual operations because she considers an employee who remains in the same job, in the same unit, at the same pay and same salary band but with an increase in MRR (which may simply be due to a new market survey, which reflects external factors rather than any factor related to an individual employee) as having been "promoted." ( Id. at 14 n. 11.) In sum, while Siskin concedes that his definition of promotion may be somewhat underinclusive, he persuasively demonstrates that Haworth's definition may be overinclusive. (Def.'s S.J. Mot. Ex. 45 [12/03 Siskin report] at 2.)
Sodexho's 1999 Management Career Structure Salary Administration Guidelines explain MRRs as follows: "An MRR represents the competitive or `going' rate for a position within the national market." ( See Pls.' Opp. to Def.'s S.J. Mot. Ex. 105.)
When the factual underpinnings of an expert's opinion are in dispute, it is not the role of the court to determine the correctness of the facts underlying the expert's testimony.
Defendants confuse the requirement for sufficient facts and data with the necessity for a reliable foundation in principles and method, and end up complaining that Fiorito's testimony was not based on "reliable facts." The parties disputed many of the facts relevant in determining a reasonable royalty. . . . When, as here, the parties' experts rely on conflicting sets of facts, it is not the role of the trial court to evaluate the correctness of facts underlying one expert's testimony.Micro Chemical, Inc. v. Lextron, Inc., 317 F.3d 1387, 1392 (Fed. Cir. 2003). As noted in Micro Chemical, the Advisory Committee note to Federal Rule of Evidence 702 is particularly instructive in this regard:
When facts are in dispute, experts sometimes reach different conclusions based on competing versions of the facts. The emphasis in the amendment on "sufficient facts or data" is not intended to authorize a trial court to exclude an expert's testimony on the ground that the court believes one version of the facts and not the other.Id. See also Pipitone v. Biomatrix, Inc., 288 F.3d 239, 249-50 (5th Cir. 2002).
Indeed, as the Supreme Court stated in Daubert, "[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence." 509 U.S. at 596. Thus, whether Siskin uses the same definition as defendant's expert or whether Siskin has analyzed a greater or smaller number of promotions is hardly grounds for exclusion; rather, the fact question of what constitutes a promotion at Sodexho will be left for the jury.
E. The Time Period Covered by Siskin's Analyses
Defendant takes issue with the fact that Siskin considered data from 1995-2001 in his analysis. Defendant argues that statistics are invalid on their face if they fail to exclude acts of discrimination that are outside the class period (March 27, 1998 through July 1, 2001). (Def.'s Mot. at 34.)
Defendant's argument is, however, wrong as a matter of law. Cases in this Circuit clearly permit analysis of employment data that predates the class time frame. See, e.g., Palmer v. Shultz, 815 F.2d 84, 112 n. 20 (D.C. Cir. 1987) (quoting Bazemore v. Friday, 478 U.S. 385, 402 n. 13, in referring to such evidence as "quite probative"); Valentino v. United States Postal Serv., 674 F.2d 56, 66 n. 11 (D.C. Cir. 1982) (citing Hazelwood Sch. Dist. v. United States, 433 U.S. 299, 309 n. 15 (1977)); Trout v. Lehman, 652 F. Supp. 144, 146-47 n. 3 (D.D.C. 1986) (after Bazemore, the use of data from time periods that are not legally actionable is clearly acceptable); see also Eldredge v. Carpenters 46 N. Cal. Counties JATC, 833 F.2d 1334, 1339 n. 7 (9th Cir. 1987) ("Where, as here, `policies have remained unchanged over a period of time and there have been no substantial changes in the conditions determining their application, it would be unreasonable to require a plaintiff to break his or her data into year by year subgroups.' D. Baldus J. Cole, Statistical Proof of Discrimination § 7.1 (1986 Supp.). Accordingly, we find that the aggregated data are the most probative and are certainly sufficient in this case.").
F. Siskin's Destruction of Underlying Documents
Defendant complains that Siskin's process "cannot be replicated" because he destroyed the very documents from which he obtained the numbers for his reports. (Def.'s Mot. at 7-8 (accusing Siskin of throwing away or otherwise discarding the programs and output upon which he relied).) He produced "computer programs and output" for reports that are dated after the reports themselves, and thus, defendant does not have the handwritten notations he initially made about the programs. ( Id. at 8.) Defendant also complains that Siskin failed to produce drafts of his reports that were sent to counsel and counsel's comments. ( Id. at 35.) Significantly, defendant does not claim that it has been prejudiced by this alleged failure to produce. It claims only that the "process cannot be replicated." ( Id. at 35.)
Plaintiffs claim that they turned over drafts of Siskin's reports in his files and in counsel's files and that Siskin provided Haworth with "exact replicas" of the documents he reviewed to prepare his reports. (Pls.' Opp. at 8.) According to plaintiffs, Siskin's practice was to review printouts from programs he instructed his staff to run, which printouts were subsequently discarded. However, the underlying data was preserved in original form so identical printouts could be and were created for Haworth; the printouts were even annotated, explaining which data corresponded to which table in Siskin's reports. (Pls.' Opp. at 9.) Haworth, by contrast, only provided her back-up data to Siskin in electronic form. ( Id. at 9 n. 7.)
Fed.R.Civ.P. 26(a)(2)(B) requires that an expert produce "the data or other information considered by the witness in forming the opinions." It is far from clear that Siskin did not meet this standard, and in fact, it appears that if he did not, Haworth's noncompliance was even more striking. But more importantly, any failure to produce documents is not a basis for invoking exclusion under Daubert.
G. Siskin's Segregation Analyses
Defendant claims that Siskin is not an expert on segregation analyses, and he is therefore not qualified to offer such an analysis into evidence. (Def.'s Mot. at 36, Def.'s Reply at 16.) During his deposition, Siskin could not recall two segregation analysis-related arithmetical terms (Def.'s Mot. at 37), even though they appeared in an article that Siskin had read, but in the same deposition Siskin also explained that he performed the exact analyses that those terms describe. (Pls.' Ex. A [5/04 Siskin decl.] at ¶ 28; Def.'s Ex. A(1) [Siskin dep.] at 495-97.) Although Siskin has never before offered a segregation analysis into evidence, he has performed such analyses during his career. (Pls.' Ex. A [5/04 Siskin decl.] at ¶ 27.) Further, as a well-regarded statistician, Siskin is qualified to perform segregation analyses, which, as pointed out by plaintiffs, are "not qualitatively different from the statistical studies he has performed and analyzed over the last 30 years." (Pls.' Opp. at 24.)
"The Daubert test must be applied with due regard for the specialization of modern science. A scientist, however well credentialed he may be, is not permitted to be the mouthpiece of a scientist in a different specialty." Dura, 285 F.3d at 614. That said, the Court is unpersuaded that an expert statistician cannot perform a segregation analysis. The article defendant introduced to demonstrate Siskin's apparent unfamiliarity with the two terms of art begins by pointing out just how common such analyses are, and in no way suggests that segregation analyses are so specialized that an experienced Ph.D. statistician could not perform them properly. See W. Carrington K. Troske, On Measuring Segregation in Samples With Small Units, 15 J. Bus. Econ. Stat. 402, 402 (1997) ("Economists and other social scientists often assess the extent to which demographic groups are segregated from each other.").
Defendant also attempts to cast doubt on Siskin's segregation analysis because of the small size of his sample. (Def.'s Mot. at 36.) Although problems may arise when segregation analyses are conducted for small samples if gauged against "evenness," that is not the approach Siskin used. (Carrington Troske, at 405; Def.'s Ex. A1 [Siskin dep.] at 353.) Rather, he followed the approach proposed by Carrington and Troske, which measures segregation by looking at deviations from randomness and which is more accurate with small samples. (Pls.' Ex. A [5/04 Siskin decl.] at ¶ 27.) As such, the Court sees no reliability problem in admitting this evidence and leaving it to the jury to determine its probativeness.
It is noteworthy that, of defendant's numerous complaints about Siskin's segregation analysis, Sodexho does not actually claim that Siskin's segregation analysis was improperly performed, and apparently Haworth dropped her major objections to it after Siskin pointed out that her main criticism was erroneous. (Pls.' Ex. A [5/04 Siskin decl.] at ¶ 29.)
H. The Utility of a Breslow-Day Analysis
At the oral argument on this motion on December 3, 2004, defendant presented for the first time in nearly four years of litigation a "Breslow-Day" analysis that it claims is vital to establishing the reliability of Siskin's aggregated pools analysis. (Tr. at 41, 136.) The Court refused to consider the results of Breslow-Day analysis since neither the Court nor plaintiffs' counsel had seen this analysis prior to the hearing. ( Id. at 124-25.) But defendant nonetheless argued at the hearing that plaintiffs bear the burden of showing that Siskin's pools analyses are sufficiently homogeneous to be admitted. ( Id. at 136-37.) While this is a correct statement of law, see Meister, 267 F.3d at 1127 n. 9, the Court is unable to credit defendant's contention that the admissibility of Siskin's pools analysis, and much of plaintiffs' prima facie case, must be supported by a test that was not done by defendant until "very, very recently at [counsel's] request." (Tr. at 43.) Siskin has previously explained at length the methodology underlying his pools analyses, and the Court is satisfied that these analyses satisfy Daubert. ( See, e.g., Pls.' Opp. to Def.'s S.J. Mot. Ex. 160 [11/01 Siskin report] at §§ A, B.)
Defendant also complained at the hearing that Siskin re-ran his multiple regression analysis following the close of discovery. ( See, e.g., Tr. at 129-30.) But the evidence in question had been before the Court since May 2004 and defendant responded to his new regression analyses in its reply brief and in Haworth's accompanying declaration. ( See, e.g., Def.'s S.J. Mot. Ex. 70 [5/17/04 Haworth decl.] at ¶¶ 10-12.) Moreover, Siskin's introduction of analyses merely modifying preexisting results a few months after the close of discovery, at a time when defendant had ample time to respond in whatever manner it saw fit, is not comparable to defendant's attempted introduction of an entirely new form of analysis during oral argument.
Similarly, the Court will not consider the computer codes introduced by defendant for the first time at oral argument, which purportedly show that Siskin did not code various analyses in the same manner as Haworth. (Tr. at 141.)
I. Prior Rulings Regarding Siskin's Testimony
Finally, defendant insinuates that this Court should deem Siskin an unqualified witness, because other courts have previously discounted his opinions. See, e.g., Calloway v. Westinghouse Elec. Corp., 642 F. Supp. 663, 690 (M.D. Ga. 1986); Chang v. Univ. of R.I., 606 F. Supp. 1161, 1207 (D.R.I. 1985); Carpenter v. Boeing, No. 02-1019 (D. Kan. Feb. 24, 2004). (Def.'s Mot. at 20-21.) Unsurprisingly, plaintiffs' retort consists of citations to numerous cases where Siskin has been described in glowing terms. (Pls.' Opp. at 3, citing, inter alia, Penk v. Or. State Bd. of Higher Educ., No. 80-436,1985 WL 25631, at *14 (D. Or. Feb. 13, 1985); Vivone v. Acme Markets Inc., 687 F. Supp. 168, 171 (E.D. Pa. 1988); Bolden v. Pa. State Police, 491 F. Supp. 958, 966 (E.D. Pa. 1980).) Plaintiffs further note that Siskin's testimony has never been excluded under the Federal Rules of Evidence or Daubert, and the cases cited by defendant relate to the issue of the credibility of his testimony, not its admissibility. (Pls.' Opp. at 5-8.) In fact, in the only case cited to the Court where Siskin was the subject of a Daubert motion, which was brought by some of the very same counsel who represent the defendant here and which made similar arguments to those presented here, the motion was denied. (Pls.' Opp. at 4, citing Beck v. Boeing Co., No. 00-301P (W.D. Wash. May 11, 2004) (Pls.' Ex. B); see also Boeing's Def.'s Mot. in Limine to Exclude Report, Opinions, and Testimony of Plaintiffs' Expert Bernard Siskin, Ph.D., Beck v. Boeing Co., No. 00-301P (W.D. Wash.) (Def.'s Mot. filed Apr. 12, 2004).) See also Anderson v. Boeing Co., 222 F.R.D. 521, 526-30 (N.D. Okla. 2004) (denying Daubert motion challenging Siskin's testimony).
Of course, the question is not whether courts have admitted Siskin's testimony in the past, but rather whether his testimony in the instant case is sufficiently reliable and relevant as to warrant admission here. Both Siskin and Haworth have testified numerous times in Title VII cases, including on the same side ( see, e.g., Green v. United States Steel Corp., 570 F. Supp. 254, 275 (E.D. Pa. 1983) and 640 F. Supp. 1521, 1544 (E.D. Pa. 1986)), and both have been the subject of court criticism. See, e.g., Dukes v. Wal-Mart, Inc., 222 F.R.D. at 197-98 n. 8 (criticizing Haworth's conclusory assertions regarding the reliability of the data she relied upon and striking a portion of her declaration). As would be expected of experts with such extensive litigation experience, each has been on the losing side and the prevailing side numerous times. Looking to past decisions — whether they admitted Siskin's testimony while nonetheless according it little weight ( see, e.g., Calloway, 642 F. Supp. at 690) or whether they lauded his work and gave it great weight ( see, e.g., Pennsylvania v. O'Neill, 431 F. Supp. 700, 714 (E.D. Pa. 1977)) — hardly advances this Court's Daubert analysis. "Because the fate of disparate treatment claims is heavily dependent on the setting, facts, and circumstances of the particular case, decisions in such cases have limited precedential value." Valentino, 674 F.2d at 69 (citation and internal quotation marks omitted). Instead, in deciding whether to admit Siskin's testimony, the Court has considered the parties' arguments, Siskin's statements, and the record evidence, and on that basis, the Court deems Siskin's testimony to be sufficiently relevant and reliable as to survive defendant's challenge under Daubert. See Kumho Tire, 526 U.S. at 149-150.
CONCLUSION
The instant motion has done much to bring home the truth of the Supreme Court's observation in Teamsters: "[S]tatistics . . . come in infinite variety. . . . [T]heir usefulness depends on all of the surrounding facts and circumstances." 431 U.S. at 340. As noted, much concerning Siskin's analyses is hotly disputed. The same can also be said about Haworth's work, as is revealed by the parties' summary judgment pleadings. But none of defendant's many criticisms rises to the level of warranting the exclusion of Siskin's testimony. Under the standards set forth in Daubert, defendant's motion will therefore be DENIED.