Opinion
Civil Action No. 99 C 3356.
September 17, 2002
MEMORANDUM AND ORDER ON DEFENDANT'S MOTION IN LIMINE TO EXCLUDE PLAINTIFF'S EXPERT TESTIMONY
Dr. Louise F. Fitzgerald was engaged by EEOC as an expert in this case. The Table of Contents of her report dated May 1, 2000, lists the subjects covered: I. Introduction and Overview; II. Summary of Scientific Knowledge Concerning Sexual Harassment; III. Brief description of survey; IV Analysis of Work Environment at Aurora-Dial; and V. Conclusions and Opinions. (Hereafter Fitzgerald Report).
Five opinions are expressed in V., as follows:
1) The Aurora plant of the Dial Corporation is historically permeated by a high level of hostile and degrading sexualized behavior directed towards women as a group, as well as towards individual women. Such behavior was frequent, serious, and of long duration. It is my opinion that it was for many years virtually impossible to work "on the floor" at Dial without being exposed to these conditions on a regular basis. Although efforts have recently been made to correct the most obvious problems (e.g., pornography, sexualized visuals), such efforts are only the smallest of remedial steps, and leave untouched the more deeply rooted problems of which such materials are symptomatic.
2) The events alleged could reasonably be expected to lead to a variety of emotional and health-related problems for women who encountered them. Reactions ranging from anxiety, depression, decreased job satisfaction, and increased job stress to nightmares, panic attacks, and major depressive reactions are reasonably foreseeable, documented in the case materials, and consistent with those documented in the scientific literature as the outcomes of sexual harassment; they are the natural and expectable outcome of exposure to noxious sexualized behavior at work. The survey data provide further support for the contention that a substantial proportion of the female workforce at Dial Aurora was negatively affected by pervasive atmosphere of offensive sexual behavior.
3) Dial-Aurora appears characterized by a pervasive culture of permissiveness accepted and modeled by supervisors and management up to the highest executive levels of the plant. The documentation in the case presents considerable evidence of inappropriate behaviors engaged in, ignored by, and/or acquiesced to at every level of management. This normative culture contributed to the perception of lower level employees that such behavior was acceptable at Aurora-Dial.
4) In addition to a permissive organizational culture and lax management norms, the Dial Corporation exhibited a high level of tolerance of sexually harassing behavior in its Aurora plant as evidence by its minimal attempts to sensitize its employees to the issue; failure to train its HR professionals; failure to maintain adequate centralized records; failure to follow up with complainants; and reluctance to impose meaningful sanctions on offenders. There is considerable evidence, both testimony and survey, that reports of harassment were often not taken seriously and that female employees were unwilling to bring forward complaints for fear of serious consequences.
(Fitzgerald Report, p. 83).
The report then says that the opinions are to a "reasonable degree of scientific certainty."
The defendant's Motion In Limine says:
Defendant, the Dial Corporation ("Dial") by its attorneys and pursuant to Federal Rules of Evidence 402, 403 and 702 moves in Limine to exclude the testimony of EEOC's expert, Dr. Louise Fitzgerald, arguing:
1. Dr. Fitzgerald's testimony is irrelevant, unreliable, and prejudicial and, therefore, is inadmissible under Federal Rules of Evidence 702, 704 and 703.
2. The bases for this motion are more fully set forth in Dial's accompanying Memorandum of Law.
(Hereafter Dial Memo.)
Thus, the motion asks that all Dr. Fitzgerald's testimony be disallowed at trial. Her deposition (Fitzgerald Dep.), which I have read, spanned three days and consumes 634 pages. The motion will be granted in part and otherwise denied.
The opinions of Dr. Fitzgerald rely heavily, although not at all exclusively, upon a Workplace Environment Survey (hereafter WES) and a Sexual Experiences Questionnaire (SEQ). Many questions about both the WES and the SEQ are appropriate and many flaws in them have been noted in the Report of Dr. Barbara A. Gutek, an expert engaged by EEOC to analyze Dr. Fitzgerald's report. I shall not recite all the criticisms that Dr. Gutek has for Dr. Fitzgerald's report and conclusions, but there are some that for the purposes of the case in its present posture are so substantial that I, as the gatekeeper, should say that the opinions resting in such significant degree upon the WES and the SEQ must be kept from the jury. Neither those surveys nor the opinions substantially based upon them are relevant or valid or useful in assisting "the trier of fact to understand the evidence or determine a fact in issue." as required in Rule 702 of the Federal Rules of Evidence.
I. STANDARD OF REVIEW
The admissibility of testimony by experts is governed by Federal Rule of Evidence 702, which states as follows:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
Fed.R.Evid. 702. The Advisory Committee Note accompanying Rule 702 indicates that in the year 2000, Rule 702 was amended in response toDaubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993) and its progeny. Fed.R.Evid. 702 advisory committee's note. In Daubert, the Supreme Court "held that Federal Rule of Evidence 702 imposes a special obligation upon a trial judge to `ensure that any and all scientific testimony . . . is not only relevant, but reliable.'" Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 147 (1999) (quoting Daubert, 509 U.S. at 589). The amendment to Rule 702 affirms the Supreme Court's casting of the trial court in the role of a "gatekeeper," Fed.R.Evid. 702 advisory committee's note, who must "ensure the reliability and relevancy of expert testimony" and "make certain that an expert, whether basing testimony upon professional studies or personal experience, employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field." Id. at 152. The Court has listed certain factors that may be used by the trial court in connection with its reliability determination:
— Whether a "theory or technique . . . can be (and has been) tested"; — Whether it "has been subjected to peer review and publication"; — Whether, in respect to a particular technique, there is a high "known or potential rate of error" and whether there are "standards controlling the technique's operation"; and — Whether the theory or technique enjoys "`general acceptance'" within a "`relevant scientific community.'"Kumho, 526 U.S. at 149-50 (quoting Daubert, 509 U.S. at 592-94). The trial court should consider these factors "where they are reasonable measures of the reliability of expert testimony," but, at the same time, the trial court is granted "considerable leeway" to determine "how to test an expert's reliability." Id. at 152.
II. ANALYSIS
Dr. Fitzgerald has prepared a report that sets forth four substantive conclusions (see Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. B, "Expert Report" (hereinafter "Fitzgerald Report") at 83) that she claims are supported by her review of the "Scientific Knowledge Concerning Sexual Harassment" and her "Analysis of [the] Work Environment at Aurora — Dial," (id. at 2). Dr. Fitzgerald's "work environment" analysis includes a survey of women employed at the defendant's Aurora plant, which in turn contains a rather lengthy set of questions that have been grouped into scales. (See id. at 21.) One of these scales purports to measure the survey respondent's "offensive sex-related experiences at work," (id. at 21), and has been dubbed the "SEQ," (see, e.g., id. at 27 n. 8 and accompanying text). Defendant The Dial Corporation (Dial) argues that "Dr. Fitzgerald's testimony cannot satisfy the [Rule 702] reliability requirement because her methodology is flawed." (Def.'s Redacted Mem. of Law in Supp. of Its Mot. in Limine to Exclude PL.'s Expert Testimony (hereinafter Def.'s Br.) at 10.) Specifically, the defendant argues that: 1) Fitzgerald's SEQ is not a valid measure of unlawful sexual harassment; 2) the survey sample size is too small to permit the survey findings to be generalized to the entire relevant population of female employees at Dial; 3) the survey fails to confine its focus to the relevant time frame; and 4) the survey is biased against Dial. I shall review each of the defendant's arguments in turn.
The survey was twenty-seven pages long. (See Fitzgerald Report at 20.)
A. The Validity of the SEQ
The defendant first argues that because the SEQ does not measure "sexual harassment" within the meaning of Title VII, it is not valid. The first component of the defendant's argument seems to have been conceded, because the plaintiff admits that "[t]he SEQ measures sexual harassment under the social science definition . . . ." (Pl.'s Response to Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony (hereinafter "Pl.'s Br.") at 12.) In addition, Fitzgerald recognizes in her report that "[s]exual harassment is a legal term of art," but claims that the term "is used in [her] report in its commonly understood sense of sex-related behavior that is unwanted and unreciprocated by the recipient." (Fitzgerald Report at 5 n. 1.) However, although it is true that the SEQ does not measure "legal" sexual harassment, I do not believe that this fact, standing alone, requires a finding that the SEQ is invalid.
Fitzgerald and the defendant's expert, Dr. Barbara Gutek, have each discussed the importance of validity in survey research. (See Fitzgerald Report at 21-22; Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. D, "Report of Dr. Barbara A. Gutek" (hereinafter "Gutek Report") at 22.) Dr. Fitzgerald's report states that "[t]he issue of validity is critical in all research," and notes that collecting data within the context of a lawsuit "presents special challenges." (Fitzgerald Report at 21.) Her report also indicates that "every effort" was made to avoid biasing responses and that "objective measures" were examined "wherever possible" to provide confidence in the results. (Id.) However, Fitzgerald admits that a significant portion of the survey respondents are members of the class of persons with claims against Dial in the present lawsuit, and that her data collection methods may have "introduced a certain amount of nonsystemic variance into the data." (Id. at 22.) I take the latter statement to mean that Dr. Fitzgerald's research methods, as opposed to the variables that she attempted to measure using those methods, may have influenced the results she obtained. However, Dr. Fitzgerald opines without elaboration that her "examination of the pattern of results, the sample demographics, and the various objective measures available suggest that any such variance has minimal impact on the results reported here." (Id.)
Dr. Gutek's report initially presents a broad definition of "validity." (See Gutek Report at 22.) Specifically, it states that "[a] measure like the SEQ is valid if it measures what it claims to measure." (Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. D, "Report of Dr. Barbara A. Gutek" (hereinafter "Gutek Report") at 22. The report then provides more detailed definitions of different types of validity (see id. at 22-27), which I shall address momentarily. In this case, the SEQ claims to measure "offensive sex-related experiences at work." (Fitzgerald Report at 21.) Given the general definition of validity provided by the defendant's expert, it appears that if the SEQ measures offensive sex-related experiences at work, which is what it claims to measure, it could be considered valid. Therefore, the fact that the SEQ does not measure "legal" sexual harassment does not necessarily affect its validity. However, the fact that the SEQ measures something other than "legal" sexual harassment does raise the possibility of jury confusion. Although it is possible to define sexual harassment in its legal and its social science or non-legal meaning, the risk of the jury's failure to keep the distinction always in mind diminishes the usefulness of testimony and a survey built upon the non-legal definition. That is particularly true during a lengthy trial, which this one promises to be.
Although the fact that the SEQ purports to measure sexual harassment in some "non-legal" sense may not, in and of itself, render the SEQ invalid, it remains to be determined whether the SEQ is capable of measuring what it claims to measure — that is, whether it truly measures offensive sex-related experiences at work. My examination of the experts' reports suggests that the answer to this question is "no."
Dr. Gutek's report describes three "kinds" of validity: content validity, construct validity, and criterion validity. (See Gutek Report at 22.) The SEQ's content validity can be assessed by examining the survey's questions to determine whether the content of the questions relates to the "construct being measured," (id.), which, in this case, is "offensive sex-related experiences at work." (Fitzgerald Report at 21.) Dr. Gutek does not argue strongly that the SEQ lacks content validity. (See id. at 22-24.) However, she makes other observations in her report that seem to me to raise serious concerns about whether the questions included in the SEQ truly measure "offensive sex-related experiences at work." First, it must be noted that the "offensive sex-related experiences at work" that are asked about are those of "women employed at the Aurora plant between January, 1988, and October, 1999" (Fitzgerald Report at p. 20.). No identifiable period of time when these experiences occurred is targeted. Dr. Fitzgerald's survey methods have not resulted in a valid measurement of women's sex-related experiences at Dial during any common or meaningful frame of time. The defendant has raised this issue in a separate section of its brief in support of its motion in limine, and I shall therefore revisit this issue below. However, it should be noted that the SEQ's questions' failure to focus on an appropriate time frame seems to qualify as a validity shortcoming.
Second, the wording of several specific SEQ items seems ill-suited to the measurement of "offensive sex-related experiences at work" during whatever time frame the survey was supposed to test. The instructions (or "lead in," (Gutek Report at 18)) preceding the SEQ read as follows:
35. Now please tell us about your experiences working at Dial Corporation. Remember that your answers are COMPLETELY CONFIDENTIAL.
DURING THE TIME YOU WORKED AT DIAL CORPORATION, HAVE ANY OF YOUR SUPERVISORS OR COWORKERS . . .
(Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. A at 13.) This lead-in is followed by a number of separate items, such as, "told dirty stories or jokes?" (Id.) Accompanying each separate item is a scale ranging from zero to four, such that zero is associated with the response "Never," one with "Once or Twice," two with "Sometimes," three with "Often," and four with "Many Times." (Id.) The lead-in fails to limit the responses to incidents that were offensive or that were performed by male supervisors or coworkers, which means that several SEQ items could be answered with "threes" and "fours" by respondents who may actually have experienced nothing offensive at work. For example, a respondent who enjoyed joking with her female coworkers "many times" would enter a "four" on the first item of the SEQ, even though she did not view this activity as an offensive sex-related experience at work. In addition, it seems that if a coworker or supervisor "tried to get [the respondent] to talk about personal . . . things," (Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. A at 13), regardless of whether his or her efforts were offensive or sexual in nature, the respondent will accumulate points on the SEQ. While I do not find that the "content validity" of each item in a lengthy survey must be facially apparent in order to render that survey "reliable" for the purposes of Rule 702, it is important to note that the accumulation of a single SEQ point means that the respondent experienced "sexual harassment" as Dr. Fitzgerald defines the term (see, e.g., Fitzgerald Report at 65 (indicating that respondents "who reported at least one incident of offensive behavior (i.e., SEQ score 0)" were "harassed" by certain supervisors or co-workers)), and this in turn raises serious doubts about the validity of her analysis and her conclusions. As Dr. Gutek points out, it is possible that a respondent who was asked "How is your family?" by a supervisor could be identified as a victim of "offensive sexual harassment" by Dr. Fitzgerald's instrument. (See Gutek Report at 20.)
Note that the lead in defines the relevant time period as "the time you worked at Dial Corporation,' as opposed to the years from 1988 through 1999. Gutek has observed that many of the survey respondents were employed by Dial outside of this time period. (See Gutek Report at 49-50.) Indeed, Fitzgerald reports that the respondent's average tenure at Dial exceeded 14 years (Fitzgerald Report at 21), which clearly exceeds the twelve-year period of interest.
There is a separate question following the SEQ that asks respondents to indicate the extent to which "any" of the "situations" listed in the SEQ were "annoying," "offensive," "disturbing," "threatening," "embarrassing," "upsetting," or "frightening." (See id. at 14.) This arrangement does not appear to allow Fitzgerald to claim that every item scored with a value greater than one on the SEQ was viewed as offensive by a given respondent.
Dr. Gutek describes two other types of validity in her report: "construct validity" and "criterion validity." Although the terms are not clearly defined in the report, it seems that in a general sense, construct validity refers to the extent to which an instrument measures the "construct" that it is designed to measure (see Gutek Report at 24 n. 37), while criterion validity refers to the degree to which a measure such as the SEQ relates to a "criterion question," such as, "have you ever been sexually harassed?" (id. at 25-26). It seems to me that Gutek's report raises a key question confronting me in my role as "gatekeeper" under Rule 702: What precisely does an SEQ score mean?
In figure 1 of Dr. Fitzgerald's report, the respondents' SEQ score is interpreted as a measure of the "Frequency of Unwanted Sexual Attention" that those respondents experienced. Dr. Fitzgerald reports that the average SEQ score for the overall sample of respondents was 18.95. She does not indicate whether this is a high, low, or average frequency in any objective sense, nor does she compare this frequency to the frequencies observed in other working environments. Dr. Gutek argues rather persuasively that, for various reasons, comparisons of SEQ scores across different contexts and research studies are "impossible." (See Gutek Report at 17-20.) This lack of comparability of SEQ scores seems problematic to me, because it seems to render the SEQ scores devoid of any objective meaning. For example, in reviewing Dr. Fitzgerald's report, I have considered her "Figure 1" and its accompanying discussion. (See Fitzgerald Report at 27-28.) Here Dr. Fitzgerald divides the respondents into three groups based upon each respondent's relative score on a measure designed to assess how "male dominated" her work environment is or was. (See id. at 27.) She then presents each of the three groups' SEQ score on a bar graph, and notes that the mean SEQ score for members of "Group 1," who had the least "male dominated" working environment, was 11.69, while groups 2 and 3 had mean SEQ scores of 25.18 and 29.48, respectively. (See id. at 27-28.) Dr. Fitzgerald suggests that this analysis supports the claim that "survey respondents who were one of the first, the only, or one of a very few women in their job category at Dial Aurora experienced more offensive sex-related behavior, rated it as more severe, and considered it a bigger problem in the workplace than other female respondents." (Id. at 27.) The analysis associated with Figure 1 does seem to support the conclusion that survey respondents who worked in more "male dominated" working environments had higher SEQ scores than those respondents who worked in less "male dominated" environments. However, there is no evidence to suggest that a SEQ score of 29.48, or any other score, is high in any objective sense.
As I noted previously, the SEQ does not seem capable of indicating which incidents were "unwanted." See supra note 3 and accompanying text.
Despite acknowledging its "impossibility," Gutek has attempted a cross-contextual comparison of SEQ scores by making certain statistical assumptions and assigning weights to the respondents' scores. (See id. at 48-52.) Gutek admits, however, that her adjustments cannot account for the unique questions added to the SEQ by Fitzgerald in this case and do not account for the bias introduced by including class members in the survey results (see infra Part III.B). (See Gutek Report at 51-52.)
This ambiguity, along with the lack of comparability of SEQ scores across studies, limits the overall meaningfulness of Dr. Fitzgerald's conclusions. In other words, certain group comparison analyses offered by Dr. Fitzgerald, such as her opinion that survey respondents who worked in more "male dominated" working environments had "higher" SEQ scores than those respondents who worked in less "male dominated" environments, are not affected by the SEQ scores' lack of objective meaning. However, this lack of objective meaning renders some of her overall conclusions "unreliable" within the meaning of Rule 702. For example, Fitzgerald offers the following "overall" conclusion and opinion:
1) The Aurora plant of the Dial Corporation is historically permeated by a high level of hostile and degrading sexualized behavior directed towards women as a group, as well as towards individual women. Such behavior was frequent, serious, and of long duration. It is my opinion that it was for many years virtually impossible to work "on the floor" at Dial without being exposed to these conditions on a regular basis.
(Fitzgerald Report at 83.) The conclusion that the plant was "permeated by a high level of hostile and degrading sexualized behavior," (id. at 83 (emphasis added)), suggests that the extent of "hostile and degrading sexualized behavior"at Dial was compared to the extent of similar behaviors occurring at other locations and determined to be "high," when in fact no such comparison is made in the report. By claiming that Dial was permeated by a "high" level of sexual behavior, Dr. Fitzgerald is, in effect, stating that the average SEQ score of her sample (18.95) is "high." Yet Dr. Fitzgerald does not explain how an SEQ score of 18.95, 29.48, or any other value can be said to be "high" without some objective point of comparison, such as comparisons of the scores obtained in this case with those obtained in other studies. Indeed, since Dr. Gutek argues that such comparisons may be impossible (see Gutek Report at 17-20, 47-52), and since Dr. Fitzgerald herself testified that she does "not [make] any representations here about Dial being better or worse than anyone else," (Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. C, Fitzgerald Dep. (hereinafter "Fitzgerald Dep.") at 566:12-14), it seems that the SEQ cannot support Fitzgerald's overall conclusion that Dial is or has been permeated by a "high level" of "sexualized behavior." This nicely illustrates the limitation caused by the SEQ scores' lack of objective meaning, or lack of "baseline,"(see Gutek Report at 19), which seems to have been caused in part by the fact that no standard, established version of the SEQ has been developed. (See Gutek Report at 16-19, 52 n. 78.)
Similarly, the claim that the "sexualized behavior" was "frequent" (Fitzgerald Report at 83) is not supported by the SEQ. While the SEQ provides a measure that may allow for relative comparisons to be made among various groupings of the survey respondents, it cannot be used to support the conclusion that the behaviors occurred "frequently" for the same reasons that it cannot support the conclusion that levels of offensive behavior were "high."
In addition, I cannot see how the SEQ, or the survey as a whole for that matter, could reliably contribute to the conclusion that "[t]he Aurora plant of the Dial Corporation is historically permeated by a high level of hostile and degrading sexualized behavior directed towards women as a group," (id. (emphasis added)), since Dr. Fitzgerald admits that the survey results should not be generalized to "the entire population of female employees of the Dial Aurora plant during the time at issue." (Fitzgerald Report at 20.) This means that she cannot claim that the survey results accurately reflect the responses of the entire set of relevant Dial employees. Therefore, not only is it impossible to assign an objective value or meaning to an SEQ score due to the comparability problem explained by Dr. Gutek, but the scores do not even represent the experiences of all of the female employees at Dial.
Also, I note in passing that Dr. Fitzgerald reported that of the 129 women who were included in her "final sample," (id. at 20), only 102 had an SEQ score greater than zero (id. at 65). This suggests that approximately 21% of the final sample did not report experiencing any of the "sexualized behaviors" included on the SEQ. Although Dr. Fitzgerald testified that 90% of her SEQ respondents experienced one or more of the behaviors listed in the SEQ, (see Fitzgerald Dep. at 565:18-20; see also Fitzgerald Report at 25 (indicating that 91.7% of respondents indicated that they "ever" heard a supervisor or coworker tell a dirty story or joke)), the discrepancy might be explained by "missing data," which appear to have influenced the percentages of "sex-related behavior" reported in Table 1 of Dr. Fitzgerald's report. (See Fitzgerald Report at 25 (noting that the number of participants included in certain calculations varied between 117 and 129 due to missing data).) At any rate, Dr. Fitzgerald's own data, which suggest that over one-fifth of her final set of respondents reported no incidents, belie her opinion that "it was for many years virtually impossible to work `on the floor' at Dial without being exposed to these conditions on a regular basis." (Fitzgerald Report at 83).
In sum, it seems to me that certain inherent shortcomings in SEQ, combined with particular problems associated with the administration and use of the SEQ in this case, raise serious questions concerning the SEQ's "validity," as that term is used in the social sciences. Although the fact that the SEQ does not measure "legal" sexual harassment per se does not render it invalid, it seems to me that the wording of certain SEQ items and their instructions suggest that the SEQ does not measure the "construct" specified by Fitzgerald. The SEQ does not focus upon any particular time frame of sexual occurrences, does not ensure that only "offensive" behaviors are reported, does not ensure that only "sexualized" behaviors are reported, and yields scores that cannot support universal claims regarding the extent of the "harassment" that occurred at Dial, due to the unavailability of baseline data. I find that under Federal Rule of Evidence 702, the validity problems associated with the use of the SEQ in this case weigh against receiving into evidence the portion of Fitzgerald's report that relies upon that instrument.
B. Sample Size
The defendant also argues that the number of employees who responded to the survey, or sample size, is too small to allow one to draw conclusions about the experiences of all of the relevant employees based upon the results obtained from the survey. Indeed, as I have noted above, Dr. Fitzgerald admits as much in her report. (See Fitzgerald Report at 20.) However, despite Dr. Fitzgerald's statement that she does not use the survey to make any formal statistical generalizations, her report contains statements that call her commitment to that position into question. For example, she claims as follows:
Length of tenure was found to be significantly associated with all forms of psychological distress, as well as with negative health status. These results provide strong inference [sic] that the working environment at Dial Aurora had a pervasive pernicious effect on the women who worked there.
(Fitzgerald Report at 57 (emphasis added).) The second sentence is an inappropriate generalization according to Dr. Fitzgerald herself. (Compare id. with id. at 20.) The sentence refers to a pervasive effect upon "the women who worked" at Dial, even though conclusions derived from the survey results are to be limited to the subset of women who completed the survey. Similar examples can be found at various points throughout Dr. Fitzgerald's report. Some of these generalizations seem to be based solely upon the survey results (see, e.g., id. at 58 ("it is reasonable to conclude that women employees as a group suffered and continue to suffer considerable emotional distress as a result of their experiences at the Aurora plant.")) while others appear to be based primarily upon Dr. Fitzgerald's review of certain claimants' depositions (see, e.g.,id. at 58; id. at 62; id. at 64-65; id. at 76; id. at 78; id. at 79 ("[S]uch incidents fostered a general perception among Aurora-Dial female employees that it was risky to complain about sexual harassment.")). Some of Dr. Fitzgerald's "Overall Conclusions and Opinions" also seem to disregard her earlier warning about the risks of generalizing her survey results. (See id. at 83.) Specifically, her first, third, and fourth set of "conclusions" appear to be phrased in a manner that suggests that the experiences of the women who participated in her research can be used to characterize the overall environment at Dial. (See id.)
Dr. Fitzgerald's report does not explain whether or not the information she gleaned from her review of the depositions may be "generalized," but I believe that I may safely assume that they may not be. Dr. Fitzgerald seems to use the depositions primarily to illustrate her opinions with detailed examples taken from the testimony of certain claimants, (see, e.g., Fitzgerald Report at 75), and absent some explanation from Dr. Fitzgerald, I find I cannot assume that the experiences described by a single deponent allow one to draw formal conclusions about the experiences of the entire relevant population of employees. Indeed, if the combined survey results of 129 employees cannot be generalized to the entire group, it seems unlikely that the experiences of a single deponent can be so generalized.
In addition to this generalizing, which seems to be impermissible under Dr. Fitzgerald's own standards, Dr. Fitzgerald recognizes that a substantial number of claimants, or "class members" are included among the survey respondents. (See Fitzgerald Report at 21-22.) Dr. Gutek has responded with an analysis that demonstrates the bias that has been introduced into the survey results due to the inclusion of the class members. (See Gutek Report at 27-37.) The class members are highly motivated to return surveys that enhance their chance for recovery, and it seems that they may have done precisely that. (See id.) The difference between the class member's responses and the other employees' responses was pronounced, and Dr. Gutek claims that the survey materials can actually be used to predict accurately whether a given respondent was a class member or not. (See id.)
I find that the use to which the survey results have been put, namely, to generalize about the overall conditions at Dial, is not consistent with the inherent limitations of Dr. Fitzgerald's research design — limitations that Dr. Fitzgerald herself expressly recognizes in her own report. Furthermore, the inclusion of a large number of class members in the survey appears to have strongly influenced the overall results, which further supports the defendant's position that the survey data do not reliably reflect the views or experiences of the overall population of relevant employees. Therefore, I find that Dr. Fitzgerald's conclusions are unreliable within the meaning of Rule 702, to the extent that those conclusions are based upon generalizations.
C. The Relevant Time Frame
The defendant next argues that Dr. Fitzgerald's survey is unreliable because it fails to focus upon a relevant time frame. The plaintiff responds only with an unsupported assertion that the defendant's argument "goes to the weight of Dr. Fitzgerald's testimony, not its admissibility." (Pl.'s Br. at 12.) I disagree with the plaintiff's response. Under Rule 702, I am charged with determining whether Fitzgerald's evidence "is the product of reliable principles and methods" and whether she has "applied the principles and methods reliably to the facts of the case." Fed.R.Evid. 702. If the plaintiff's survey fails to identify the relevant time frame, its design is flawed in a manner that directly affects its relevance to this case.
The defendant argues that no question in the entire survey limits the respondent's answers to a 1988-1999 time frame. I again note that the survey was of "women employed . . . between January, 1988, and October, 1999." (Fitzgerald Report at p. 20) (Emphasis added.) There is no limitation of a time frame for occurrences of sexual events. In addition, I have already noted that the respondents' average tenure at Dial exceeded fourteen years (see supra note 2), which suggests that many respondents may have included descriptions of events and experiences in their surveys that occurred beyond 1988 and 1999 or any other identifiable time frame. The survey's failure to measure reliably the events of a relevant time frame weighs against its admissibility under Rule 702.
In addition, the defendant points out that the survey's failure to limit itself to particular years, combined with the response options available to the respondents who completed the SEQ, interferes with the survey's reliability in yet another way. For example, a respondent who worked at Dial for two years and who heard a "dirty joke" once per year would select the "once or twice" response option on the appropriate portion of question number 35. (See Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. A at 13.) An employee who worked at Dial for 15 years and who heard a "dirty joke" once per year would likely select "many times" in response to the same question, even though she experienced the jokes at the same rate as the other respondent. (See id.) This creates an "artifact" that causes persons who worked at Dial for longer periods of time to present higher SEQ scores. (See Gutek Report at 29.) This artifact interferes with the survey data's ability to describe the rate at which the respondents experienced the "harassing" events listed in the survey. (See Gutek Report at 29-31.)
For these reasons, the failure of the survey to confine its focus to a relevant time frame renders it unreliable and not useful to a jury in this case within the meaning of Rule 702.
D. Bias
The defendant next argues that Dr. Fitzgerald's survey is unreliable because it is biased against Dial. Once again, the plaintiff responds only with its assertion that the defendant's argument "goes to the weight of Dr. Fitzgerald's testimony, not its admissibility." (Pl.'s Br. at 12.) I disagree with the plaintiff's response for the reasons stated above. (See supra Part III.C.) It seems to me that if Fitzgerald's survey is biased to a degree that renders it unreliable within the meaning of Rule 702, the bias affects the admissibility of her report.
I have already indicated that it is likely that the survey results are biased by the presence of the class members. (See supra Part III.B.) In addition, Dr. Gutek points out that the survey itself contains an overabundance of negative response options and negatively worded questions, which render it difficult for respondents to express a positive view of Dial. (Gutek Report at 7-12.) For example, the survey item that purports to measure the respondents' reactions to the "sex-related experiences at work" that they reported on the SEQ contains no option that allows the respondents to indicate that the event they experienced did not bother them. (See Def.'s Mot. in Limine to Exclude Pl.'s Expert Testimony, Ex. A at 14.) Also, question number 24 asks the respondent to report "What are/were the PEOPLE YOU WORK WITH AT DIAL like MOST OF THE TIME?", and includes the following nine options: "boring," "slow," "stupid," "responsible," "waste of time," "lazy," "unpleasant," "bother me," and "work well together." Apparently, some of the respondents not only noticed the survey's negative slant, but wrote comments about this obvious bias on the face of the survey itself. (See Gutek Report at 12 n. 12.)
Dr. Gutek also argues that the survey respondents were "primed." (Gutek Report at 12-14.) According to Dr. Gutek, "Priming refers to creating a situation that will influence someone's views or opinions." (Id. at 12.) "Good research avoids priming respondents, i.e., it avoids asking questions or presenting information that will influence the way respondents will answer subsequent questions." (Id.) Dr. Gutek claims that Dr. Fitzgerald's survey does not adequately address the priming problem. (See id. at 12-14. See also Def.s' Br. at 15.) The plaintiff does not dispute this argument.
Finally, Dr. Gutek's Report indicates that before the survey was distributed to the relevant group of employees, the EEOC sent a letter to the entire group indicating that efforts were being made to "identify any female employees who were affected by sex harassment and who may be entitled to recover in this lawsuit." (Gutek Report at 5.) It seems to me that this letter might not only have motivated respondents to complete the survey, but could have introduced an element of bias into their responses.
There are indications of bias against Dial in the survey instrument used by Dr. Fitzgerald, and there is reason to believe that many of the actual respondents were biased against Dial as well. Since the plaintiff has chosen not to dispute the existence of this bias or to argue that its effects are not severe enough to support a finding that reliable methods were not used to obtain the survey data, I can only conclude that the presence of the bias weighs against a finding that the survey report is reliable or helpful in assisting the factfinder within the meaning of Rule 702.
III. CONCLUSION
In conclusion, I find that the plaintiff's survey instrument presents inherent reliability problems, and that Dr. Fitzgerald has applied it to this case in an unreliable fashion. The instrument itself and the manner in which it has been used in this case present sampling and bias problems that render it incapable of passing muster under Rule 702, and the SEQ portion of the survey lacks validity. The survey, its results, and any conclusions based upon those results shall not be admitted into evidence in this case. This includes all the findings and opinions contained in Dr. Fitzgerald's report under the headings III, IV, and V, covering the "Brief description of survey," "Analysis of Work Environment at Aurora-Dial," and "Conclusions and Opinions." Such findings and opinions are so reliant upon the survey materials that they are too flawed to be useful in assisting the factfinder in this case. Also inadmissible is the material in heading I, which is identified as "Introduction and Overview," because it describes in a general way what Dr. Fitzgerald did in the "empirical survey of women employed at the Aurora Plant of The Dial Corporation between 1988 and 1999," the very material that I have concluded must be kept out of evidence.
IV. SECTIONS OF DR. FITZGERALD'S REPORT NOT DEALING DIRECTLY WITH THE AURORA PLANT
Portions of Dr. Fitzgerald's report not relating directly to the Aurora Plant are not challenged by the defendant, except by the broad expression that "Dr. Fitzgerald's testimony is irrelevant, unreliable, and prejudicial . . ." Untouched is the material in Dr. Fitzgerald's report under heading II, "Summary of Scientific Knowledge Concerning Sexual Harassment." It contains a general description of the nature, causes and consequences of harassment, together with factors and conditions that lead to psychological harm and various responses strategies. Those matters may well be helpful to the trier of fact to understand the evidence or to determine a fact in issue.
IT IS ORDERED that:
1. subjects, findings, and opinions developed and expressed in sections I, III, IV, and V of the Expert Report of Louise F. Fitzgerald, PhD, shall not be received into evidence in this case, and to that extent "the Defendant's Motion In Limine to Exclude Plaintiff's Expert's Testimony, filling 238, is granted; and
2. otherwise, the Defendant's Motion In Limine to Exclude Plaintiff's Expert's Testimony, filing 238, is denied.