In re Paraquat Prods. Liab. Litig.

Full title: In re PARAQUAT PRODUCTS LIABILITY LITIGATION This Document Relates to…

Court: United States District Court, Southern District of Illinois

Date published: Apr 17, 2024

MDL 3004 (S.D. Ill. Apr. 17, 2024)

Opinion

MDL 3004 3:21-md-3004-NJR

04-17-2024

In re PARAQUAT PRODUCTS LIABILITY LITIGATION This Document Relates to: Richter v. Syngenta AG, et al., No. 3:21-pq-571 Fuller v. Syngenta AG, et al., No. 3:21-pq-836 Burgener v. Syngenta AG, et al., No. 3:21-pq-1218 Coward v. Syngenta AG, et al., No. 3:21-pq-1560

NANCY J. ROSENSTENGEL, Chief U.S. District Judge.

MEMORANDUM AND ORDER

NANCY J. ROSENSTENGEL, Chief U.S. District Judge.

This is a multidistrict litigation (“MDL”) in which over 5,000 individual Plaintiffs allege that they developed Parkinson's disease because of their exposure to an herbicide, paraquat dichloride (“paraquat”). Paraquat is a restricted-use quaternary ammonium herbicide that is used to control weeds in farming operations and other settings around the United States. See 40 C.F.R. § 152.175 (permitting paraquat use only “by or under the direct supervision of a certified applicator.”). Defendants, Syngenta Crop Protection, LLC and Syngenta AG (collectively “Syngenta”), currently manufacture and distribute paraquat for use in the United States, whereas Defendant, Chevron U.S.A., Inc. (“Chevron”), manufactured and distributed paraquat until 1986.¹The Judicial Panel on Multidistrict Litigation consolidated these cases for pretrial proceedings in this Court to enable the Parties to explore “common factual issues concerning the propensity of paraquat to cause Parkinson's disease.” (Doc. 1 at 2). See 28 U.S.C. § 1407(a).

1 See Chevron's Motion for Summary Judgment (Doc. 4349 at 3) (“There is no dispute that, in 1986, Chevron left the paraquat market-entirely and for good.”).

Pending before the Court is Defendants' Motion to Exclude the Testimony of Plaintiffs' Expert, Dr. Martin Wells, under Federal Rule of Evidence 702 and Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). (Doc. 4355). Dr. Wells serves as Plaintiffs' sole expert witness on the critical issue of general causation, offering an opinion that occupational exposure to paraquat can cause Parkinson's disease. (Doc. 4355-2 at 26). See Higgins v. Koch Dev., Corp., 794 F.3d 697, 701 (7th Cir. 2015) (“General causation refers to whether the substance at issue had the capacity to cause the harm alleged.”) (internal quotation marks and citation omitted). Defendants' motion is brought in four of the six member cases that were selected for case-specific discovery in the Court's April 13, 2022 order. (Doc. 1317).² These four cases have gone through fact and expert discovery and now serve as this MDL's first set of trial selection cases.

2 Richter, et al. v. Syngenta AG, et al., No. 3:21-pq-571-NJR; Fuller, et al. v. Syngenta AG, et al., No. 3:21-pq-836-NJR; Burgener v. Syngenta AG, et al., No. 3:21-pq-1218-NJR; and Coward v. Syngenta AG, et al., No. 3:21-pq-1560-NJR. Plaintiffs voluntarily dismissed the other two cases that were selected for case-specific discovery (Walkington et al v. Syngenta AG et al., 3:21-pq-00601-NJR; and Marx v. Syngenta Crop Protection LLC et al., 3:21-pq-00922-NJR). On May 10, 2023, the Court entered judgment dismissing Walkington and Marx with prejudice. (Walkington (Doc. 25) & Marx (Doc. 14)).

The Court has carefully reviewed the Parties' briefs, expert reports, deposition testimony, and supporting exhibits to evaluate the admissibility of Dr. Wells' proffered opinions. The Court also held a four-day Daubert hearing, where Dr. Wells himself testified and the Parties offered extensive and compelling oral argument. The motion is thus fully briefed and ripe for disposition. (Docs. 4355, 4561, 4654, 4798 & 4799).³ For the following reasons, the Court grants Defendants' motion to exclude the testimony of Dr. Wells.

3 The Court thanks the Parties for their exceptional briefing and advocacy, which greatly aided its review of the evidence.

Background

Plaintiffs retained Dr. Martin Wells to “analyze the epidemiological evidence relating the association and causation of the occupational exposure of paraquat to the onset of Parkinson's disease.” (Doc. 4355-2 at 2). To accomplish this task, Dr. Wells conducted a meta-analysis of seven epidemiological studies that measured a potential association between paraquat and Parkinson's disease. Dr. Wells determined, based on this meta-analysis, that there was a “near tripling of PD occurrence in [study] participants occupationally exposed to paraquat.” Id. at 18.

After establishing a positive association between occupational exposure to paraquat and Parkinson's disease, Dr. Wells conducted a weight of the evidence review to determine whether the association was attributable to a causal relationship. He found that it was and drew the following conclusions: (i) the available epidemiological evidence supports a causal relationship between paraquat and Parkinson's disease; and (ii) the trial selection plaintiffs fit the exposure and diagnostic criteria of the seven studies in his metaanalysis, meaning that they were at “near tripl[e]” the risk of developing Parkinson's disease. Id. at 18, 26-27.

Jurisdiction

Before reaching the merits of Defendants' motion to exclude Dr. Wells, the Court has an “independent obligation to determine whether subject-matter jurisdiction exists.” Arbaugh v. Y&H Corp., 546 U.S. 500, 514 (2006). Subject matter jurisdiction can “never be forfeited or waived” because it concerns the Court's “power to hear a case.” United States v. Cotton, 535 U.S. 625, 630 (2002). Accordingly, a federal court's duty to critically evaluate its subject matter jurisdiction endures throughout the pendency of a case, “even in the absence of a challenge from any party.” Arbaugh, 546 U.S. at 514; Estate of Alvarez v. Donaldson Co., 213 F.3d 993, 995 (7th Cir. 2000).

Here, Plaintiffs contend that this Court has the power to hear their cases based on diversity of citizenship under 28 U.S.C. § 1332(a). If they are incorrect and the Court determines that any of the four trial selection cases lack the necessary jurisdictional predicate, the cases must be dismissed. Arbaugh, 546 U.S. at 514; see also Guilbeau v. Pfizer Inc., 880 F.3d 304, 307 n.1 (7th Cir. 2018) (MDL cases may only be adjudicated on the merits once “federal subject matter jurisdiction is secure.”). With these considerations in mind, the Court begins with an assessment of its subject matter jurisdiction over the four trial selection cases.

Federal courts have original jurisdiction over all actions between “citizens of different States” where the amount in controversy exceeds $75,000, exclusive of interest and costs. 28 U.S.C. § 1332(a). To proceed on this jurisdictional basis, the Court must satisfy itself that complete diversity of citizenship exists and that the amount in controversy requirement is satisfied. City of E. St. Louis, Illinois v. Netflix, Inc., 83 F.4th 1066, 1070 (7th Cir. 2023); Webb v. FINRA, 889 F.3d 853, 856 (7th Cir. 2018). “Complete diversity exists only if none of the defendants has the same citizenship as any plaintiff.” City of E. St. Louis, 83 F.4th at 1070. Complete diversity is not a problem here because, as discussed below, none of the Plaintiffs in the four trial selection cases has the same citizenship as any of the Defendants. See Page v. Democratic Nat'l Comm., 2 F.4th 630, 636 (7th Cir. 2021).

As natural persons, Plaintiffs are citizens of the state in which they are domiciled- i.e., where they have a “true, fixed home and principal establishment, and to which, whenever [they are] absent from the jurisdiction, [they have] the intention of returning.” 13E Charles Alan Wright & Arthur R. Miller, Federal Practice and Procedure § 3612 (3d ed. 2021); see also Gilbert v. David, 235 U.S. 561, 568-69 (1915) (discussing the concept of domicile). The pleadings in the four trial selection cases demonstrate Plaintiffs' citizenships in Illinois and Florida.⁴

4 Frederick Richter and his wife, Karen Richter (Illinois), Richter, et al. v. Syngenta AG, et al., No. 3:21-pq-571-NJR (Doc. 1 at 2); Keith Fuller and his wife, Diane Fuller (Illinois), Fuller, et al. v. Syngenta AG, et al., No. 3:21-pq-836-NJR (Doc. 44 at 2); Todd Burgener (Illinois), Burgener v. Syngenta AG, et al., No. 3:21-pq-1218-NJR (Doc. 1 at 2); and Matthew Coward (Florida), Coward v. Syngenta AG, et al., No. 3:21-pq-1560-NJR (Doc. 20 at 2).

Defendants, on the other hand, are institutional parties whose citizenship is determined by their corporate form. Page, 2 F.4th at 635. Chevron is a domestic corporation and therefore a citizen of its state of incorporation (Pennsylvania) and the state in which it maintains its principal place of business (California).⁵ See 28 U.S.C. § 1332(c)(1). Syngenta AG is a foreign corporation that is incorporated and has its principal place of business in Basel, Switzerland.⁶ See id. (explaining citizenship requirements of foreign corporations). The analysis becomes somewhat trickier for Syngenta Crop Protection, LLC, because as a limited liability company, its citizenship is determined by the citizenships of its members. Page, 2 F.4th at 635. Syngenta Crop Protection LLC's sole member is Syngenta Seeds LLC, another limited liability company. Syngenta Seeds LLC's sole member is Syngenta Corporation, which is incorporated and has its principal place of business in Delaware. Syngenta Crop Protection, LLC is therefore a citizen of Delaware for diversity purposes.⁷ See City of E. St. Louis, 83 F.4th at 1070 (a limited liability company's citizenship must be “traced through as many levels as necessary until reaching a natural person or a corporation.”). In sum, the Parties' respective citizenships support diversity jurisdiction over the four trial selection cases because Plaintiffs are citizens of Illinois and Florida, and their party opponents are citizens of Pennsylvania, California, Delaware, and Switzerland. Although this satisfies complete diversity, the Court's jurisdictional analysis does not end here. To close the loop, the Court also must assess whether the four trial selection cases meet the amount in controversy requirement set out in 28 U.S.C. § 1332(a).

5 See Chevron's Answer to Plaintiffs' complaint in Fuller, et al. v. Syngenta AG, et al., No. 3:21-pq-836-NJR (Doc. 22 at 3).

6 See Syngenta's Answer to Plaintiffs' complaint in Fuller, et al. v. Syngenta AG, et al., No. 3:21-pq-836-NJR (Doc. 21 at 3). The fact that Syngenta AG is a citizen of a foreign state does not alter the Court's diversity analysis because federal courts have original jurisdiction over civil actions between “citizens of a State and citizens or subjects of a foreign state,” provided the amount in controversy requirement is met. 28 U.S.C. § 1332(a)(2).

7 See Syngenta's Answer to Plaintiffs' complaint in Fuller, et al. v. Syngenta AG, et al., No. 3:21-pq-836-NJR (Doc. 21 at 2); see also Gorman, et al. v. Syngenta Crop Protection, LLC, et al., 3:23-pq-00369-NJR (Doc. 1 at 4) (Syngenta Crop Protection, LLC's notice of removal in related litigation, explaining its membership structure for purposes of establishing diversity jurisdiction under 28 U.S.C. § 1332).

To begin, the Court will accept Plaintiffs' allegations concerning the amount in controversy unless “from the face of the pleadings, it is apparent, to a legal certainty, that [they] cannot recover the amount claimed or if . . . the proofs . . . [demonstrate] to a like certainty that [they were] never . . . entitled to recover that amount.” St. Paul Mercury Indem. Co. v. Red Cab Co., 303 U.S. 283, 289 (1938). This standard is “not onerous,” and the Court will find that it has jurisdiction “unless an award for the jurisdictional minimum would be legally impossible.” Sykes v. Cook Inc., 72 F.4th 195, 205 (7th Cir. 2023).

Because neither party challenges this Court's jurisdiction over the four trial selection cases, the Court views Plaintiffs' jurisdictional allegations “in the light most favorable to finding jurisdiction.” Id. at 206. Moreover, unless jurisdiction is contested, courts “ordinarily do not look beyond the complaint to assess the amount in controversy.” Id. at 209.

Here, the Court is satisfied that Plaintiffs in the four trial selection cases have alleged the necessary amount in controversy in good faith. They claim they are entitled to more than $75,000 in damages, exclusive of interest and costs, because they suffer from Parkinson's disease as a result of their exposure to paraquat. This disease, they contend, has caused them severe physical pain, the loss of motor functions, mental anguish, and past and future medical expenses, among other damages. They further allege that their medical conditions and symptoms are permanent because there is no known cure for Parkinson's disease and because Parkinson's disease is a progressive neurodegenerative disorder that becomes less susceptible to treatment over time. The disease's symptoms are myriad and can include bradykinesia, rigidity, muscle spasms, difficulty swallowing, drooling and slurred speech, to name a few. Accepting the truth of Plaintiffs' damages allegations, the Court has no trouble concluding that an award over the jurisdictional minimum would not be excessive under the laws of Illinois and Florida (which govern the substantive claims in these trial selection cases). See e.g., Kerrivan v. R.J. Reynolds Tobacco Co., 953 F.3d 1196, 1203-08 (11th Cir. 2020) (applying Florida law and affirming $15.8 million jury award to smoker who suffered from chronic obstructive pulmonary disease); Phelps v. Chicago Transit Auth., 586 N.E.2d 352, 354, 356 (Ill. Ct. App. 1991) ($120,000 in damages for fractured jaw not excessive).

Having established that the Parties in the four trial selection cases are completely diverse and that all four cases meet the amount in controversy requirement, the Court concludes that it has subject matter jurisdiction to hear these cases under 28 U.S.C. § 1332(a). Accordingly, the Court proceeds to analyze Dr. Wells' proffered opinions to determine whether they are admissible at trial.

Legal Standard

The admissibility of expert testimony under the Federal Rules of Evidence is governed by the well-known and oft-cited standards of Federal Rule of Evidence 702 and Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). Under Rule 702, “[a] witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if the proponent demonstrates to the court that it is more likely than not that:

(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;

(b) the testimony is based on sufficient facts or data;

(c) the testimony is the product of reliable principles and methods; and

(d) the expert's opinion reflects a reliable application of the principles and methods to the facts of the case.”

FED. R. EVID. 702.⁸ To implement this evidentiary standard, the district court acts as a “gatekeeper” to “ensure that any and all scientific testimony . . . is not only relevant, but reliable.” Daubert, 509 U.S. at 589. Daubert provides four non-exhaustive considerations that bear on this inquiry: (1) whether a “theory or technique . . . can be (and has been) tested;” (2) “whether the theory or technique has been subjected to peer review and publication;” (3) the technique's “known or potential rate of error . . . and the existence and maintenance of standards controlling the technique's operation;” and (4) whether the theory or technique has found “general acceptance” within the scientific community. Id. at 593-94. This standard applies to all forms of expert evidence, Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147 (1999), and the proponent of the evidence “bears the burden of demonstrating that [it] satisfies [Daubert] by a preponderance of the evidence.”⁹ Krik v. Exxon Mobil Corp., 870 F.3d 669, 673 (7th Cir. 2017). Ultimately, “a district judge asked to admit scientific evidence must determine whether the evidence is genuinely scientific, as distinct from being unscientific speculation offered by a genuine scientist.” Rosen v. Ciba-Geigy Corp., 78 F.3d 316, 318 (7th Cir. 1996).

8 This amended version of Rule 702 took effect on December 1, 2023, after the Parties submitted their briefing. The Advisory Committee on the Rules of Evidence explained that the amendment does not “impose[] any new, specific procedures.” FED. R. EVID. 702 advisory committee's note to 2023 amendments. Rather, the amendment emphasized that the proponent bears the burden of demonstrating compliance with Rule 702 by a preponderance of the evidence, and that “each expert opinion must stay within the bounds of what can be concluded from a reliable application of the expert's basis and methodology.” Id.; see also In re Acetaminophen - ASD-ADHD Prod. Liab. Litig., - F.Supp.3d -, 2023 WL 8711617, at *3, *16 & n.27 (S.D.N.Y. 2023) (applying amended version of Rule 702 even though briefing was completed before amendment took effect).

9 The Advisory Committee cautions that “expert testimony may not be admitted unless the proponent demonstrates to the court that it is more likely than not that the proffered testimony meets the admissibility requirements set forth in [Rule 702].” FED. R. EVID. 702 advisory committee's note to 2023 amendments (emphasis added). In providing this instruction, the Advisory Committee noted that some courts had “incorrect[ly]” held that an expert's basis of opinion and application of her methodology were questions of weight, not admissibility. Id. The Advisory Committee thus appears to have found that courts had erroneously admitted unreliable expert testimony based on the assumption that the jury would properly judge reliability by assigning appropriate weight to an expert's opinion. See Schmidt v. Int'l Playthings, LLC, 536 F.Supp.3d 856, 888 n.36 (D.N.M. 2021) (noting that courts are in “conflict” over whether sufficiency of an expert's basis and application of methodology concern weight or admissibility; concluding that, under Rule 702, those issues concern admissibility). Mindful of its role as the witness stand's “vigorous gatekeeper,” the Court will closely scrutinize the reliability of proffered expert testimony before permitting an expert to share her opinion with the jury. Robinson v. Davol, Inc., 913 F.3d 690, 696 (7th Cir. 2019); Am. Honda Motor Co. v. Allen, 600 F.3d 813, 819 (7th Cir. 2010) (per curiam) (“[E]xpert testimony that is not scientifically reliable should not be admitted.”). The gatekeeping function, after all, “requires more than simply ‘taking the expert's word for it.'” FED. R. EVID. 702 advisory committee's note to 2000 amendments.

District courts in the Seventh Circuit conduct a three-step analysis before admitting expert testimony: They “must determine whether the witness is qualified; whether the expert's methodology is scientifically reliable; and whether the testimony will assist the trier of fact to understand the evidence or to determine a fact in issue.” Gopalratnam v. Hewlett-Packard Co., 877 F.3d 771, 779 (7th Cir. 2017) (internal quotation marks omitted). In short, the Court must evaluate (i) the proffered expert's qualifications, (ii) the reliability of her methodology, and (iii) the relevance of her testimony. Id.

Step 1 of this analysis-qualification-addresses the expert's ability to offer something of value to the resolution of a disputed issue in the case. “[A] court should consider a proposed expert's full range of practical experience as well as academic or technical training when determining whether that expert is qualified to render an opinion in a given area.” Smith v. Ford Motor Co., 215 F.3d 713, 718 (7th Cir. 2000). “Whether a witness is qualified as an expert can only be determined by comparing the area in which the witness has superior knowledge, skill, experience, or education with the subject matter of the witness's testimony.” Gayton v. McCoy, 593 F.3d 610, 616 (7th Cir. 2010) (internal quotation marks omitted). “The question . . . is not whether an expert witness is qualified in general, but whether his qualifications provide a foundation for him to answer a specific question.” Id. at 617.

Step 2 of the admissibility analysis-reliability-is the primary focus of the Parties' briefing. To assess the reliability of a qualified expert's testimony, the Seventh Circuit instructs district courts to consider, at a minimum, four factors that track the Daubert considerations mentioned above: “(1) whether the proffered theory can be and has been tested; (2) whether the theory has been subjected to peer review; (3) whether the theory has been evaluated in light of potential rates of error; and (4) whether the theory has been accepted in the relevant scientific community.” Baugh v. Cuprum S.A. de C.V., 845 F.3d 838, 844 (7th Cir. 2017). The Seventh Circuit also has endorsed six additional factors, taken from the advisory notes of the 2000 amendment to Rule 702, that bear on the reliability of an expert's testimony:

(5) whether maintenance standards and controls exist; (6) whether the testimony relates to matters growing naturally and directly out of research they have conducted independent of the litigation, or developed expressly for purposes of testifying; (7) whether the expert has unjustifiably extrapolated from an accepted premise to an unfounded conclusion; (8) whether the expert has adequately accounted for obvious alternative explanations; (9) whether the expert is being as careful as he would be in his regular professional work outside his paid litigation consulting; and (10) whether the field of expertise claimed by the expert is known to reach reliable results for the type of opinion the expert would give.

Gopalratnam, 877 F.3d at 779-80 (cleaned up). Critically, these factors are neither exhaustive, nor is any single one of them mandatory in a Daubert analysis. Kumho Tire, 526 U.S. at 150; see also Krik, 870 F.3d at 674 (“Despite the list, we have repeatedly emphasized that no single factor is either required in the analysis or dispositive as to its outcome.”) (internal quotation marks omitted). Thus, in its discussion below, the Court will consider the factors that are most probative of the reliability of Dr. Wells' opinions.

Finally, step 3 of the admissibility analysis-relevance-determines whether the expert's proffered testimony “will help the trier of fact to understand the evidence or to determine a fact in issue.” FED. R. EVID. 702(a). Daubert requires the proffered testimony to “fit” with a disputed issue in the case because “scientific validity for one purpose is not necessarily scientific validity for other, unrelated purposes.” Daubert, 509 U.S. at 591. Stated differently, there must be a “valid scientific connection to the pertinent inquiry.” Id. at 592.

Ultimately, the focus of a Daubert inquiry is not the correctness of the expert's conclusions, but rather, the principles and methodology she relied on to arrive at her conclusions. Daubert, 509 U.S. at 595; Smith, 215 F.3d at 719; see also Manpower, Inc. v. Ins. Co. of Pa., 732 F.3d 796, 806 (7th Cir. 2013) (“Reliability . . . is primarily a question of the validity of the methodology employed by an expert, not the quality of the data used in applying the methodology or the conclusions produced.”); Milward v. Acuity Specialty Prods. Grp., Inc., 639 F.3d 11, 22 (1st Cir. 2011) (“[T]he fact that another explanation might be right is not a sufficient basis for excluding [the expert's] testimony.”); Tamraz v. Lincoln Elec. Co., 620 F.3d 665, 675 (6th Cir. 2010) (“The important thing is not that experts reach the right conclusion, but that they reach it via a sound methodology.”). Accordingly, the Court will evaluate Dr. Wells' proffered opinion to determine whether he formed it with the requisite “soundness and care.” Schultz v. Akzo Nobel Paints, LLC, 721 F.3d 426, 431 (7th Cir. 2013).

Relevant Scientific Concepts

Defendants' motion to exclude Dr. Wells' proffered testimony raises complex issues related to the study of epidemiology and the scientific methodologies of systematic review and meta-analysis. Although “[t]otal immersion in the complexities of these disciplines is neither required, nor possible,” In re TMI Litig., 193 F.3d 613, 629 (3d Cir. 1999), a high-level discussion of relevant scientific concepts and methodologies is appropriate to provide the analytical framework for the Court's admissibility analysis. Thus, drawing extensively on the Federal Judicial Center's Reference Manual on Scientific Evidence and other relevant sources, the Court “offer[s] the following overview of the controlling principles with an awareness that doing so stretches the boundaries of [its] institutional competence, and with a recognition of [its] need to borrow heavily from others in academic disciplines far from the familiar confines of the law.” Id.; see also Lapsley v. Xtek, Inc., 689 F.3d 802, 811 (7th Cir. 2012) (recognizing the need for judges and lawyers to understand scientific and technical topics that are increasingly relevant in litigation).

I. The Study of Epidemiology

Epidemiology is the study of the “incidence, distribution, and etiology of disease in human populations.” Michael D. Green et al., Reference Guide on Epidemiology, in Reference Manual on Scientific Evidence 551 (Fed. Jud. Ctr. 3d ed. 2011) (hereinafter cited as “RMSE”). Epidemiology seeks to answer questions of disease causation and how to prevent or minimize the risk of disease in humans. Id. Broadly, an epidemiological causation analysis proceeds in two steps: (i) the determination of whether an association between the agent and the disease exists-i.e., do they occur more frequently together than one would expect by chance; and (ii) if a positive association is found, whether it is the result of a causal relationship. Id. at 566, 597. Although one of its goals is the assessment of disease causation, epidemiology is not a one-stop solution to resolving all complex causation issues in a toxic tort case. Most importantly, “[e]pidemiology focuses on the question of general causation (i.e., is the agent capable of causing disease?) rather than that of specific causation (i.e., did it cause disease in a particular individual?).” Id. at 552.

When epidemiologists analyze whether an agent might be associated with a certain disease, the best practice is to “observe a group of individuals who have been exposed to an agent of interest, such as . . . an industrial chemical and compare them with another group of individuals who have not been exposed.” Id. at 556. To do so, the investigator compares the rate of disease in individuals who have been exposed to the agent with that of individuals who have not been exposed. Id. An epidemiological study applying this methodology is known as an “observational” study. Observational studies are generally designed as case-control, cohort, or cross-sectional studies. A case-control study compares study participants who have a disease of interest to a control group that does not have the disease. Id. at 620. From there, the study looks back at potential causes of the disease. Id. Cohort studies analyze whether a given exposure to an agent influences the incidence of disease. Id. at 621. The exposed group and the control group (which was not exposed to the agent) are observed to “to find out if the exposed group is more likely to develop disease.” Id. In a cross-sectional study, participants are interviewed, and their exposure and disease statuses are determined at a single point in time. Id. at 560. As a result, a cross-sectional study does not account for temporality-the requirement that exposure precede the development of the disease in order to be causal. Id. at 560-61.

Observational studies play a critical role in an epidemiological assessment of a disease because randomized clinical trials in which a group of subjects is exposed to an agent to compare its effects against a non-exposed control group-the “gold standard” in assessing an agent's relationship to a health outcome-are not possible when the agent is suspected of being harmful to humans. Id. at 555. Accordingly, observational studies are the next best thing available to epidemiologists, even though their “Achilles heel” is “the possibility of differences in the two populations being studied with regard to risk factors other than exposure to the agent.” Id. at 556.

Epidemiologists also draw on toxicology models using live animals to assess toxicity in humans. Animal studies solve for some of the disadvantages of observational studies because “[e]xposure can be carefully controlled and measured” and “researchers control all aspects of the animals' lives.” Id. at 563. However, animal studies suffer from significant inherent disadvantages, which can undermine their reliability as scientific evidence in a toxic tort case. Most notably, animal data must be extrapolated to another species (humans), an exceedingly difficult task because the investigator must account for “differences in absorption, metabolism, and other factors, [which] may result in interspecies variation in response.” Id. Moreover, the “high doses customarily used in animal studies” can mask a threshold no-effect dose, making a causation inference for humans problematic.¹⁰ Id. Animal studies, therefore, at best, “play a complementary role to epidemiology by assisting researchers in framing hypotheses and in developing study designs for epidemiological studies.” Id.

10 Plaintiffs and Defendants in the four trial selection cases fault each other's experts for improperly relying on in vivo (live) animal studies to form their opinions. Indeed, considerable ink was spilled on both sides to articulate the reasons why in vivo animal studies have minimal value, if any, in resolving the issues in this case. Although the shortcomings of animal studies are myriad, the notable methodological concerns regarding interspecies extrapolation and dose-response relationship tend to fundamentally weaken their utility in a toxic tort case. Numerous courts around the country have expressed similar concerns. See e.g., Gen. Elec. Co. v. Joiner, 522 U.S. 136, 144 (1997) (proffered experts' causation opinions based on infant mice studies involving “highly concentrated” injections into animals' stomachs properly excluded because experts failed to explain how “these seemingly far-removed animal studies” could be extrapolated to humans); C.W. ex rel. Wood v. Textron, Inc., 807 F.3d 827, 836 (7th Cir. 2015) (recognizing risk of improper causation inferences based on animal studies); Daniels-Feasel v. Forest Pharmaceuticals, Inc., No. 17 CV 4188-LTS-JLC, 2021 WL 4037820, at *14 (S.D.N.Y. Sept. 3, 2021) (“The unreliability of animal studies is particularly apparent where there is overwhelming contradictory epidemiological evidence.”); In re Prempro Prod. Liab. Litig., 738 F.Supp.2d 887, 894 (E.D. Ark. 2010) (“Federal courts have consistently cautioned against extrapolation of human effects from animal studies.”); Magistrini v. One Hour Martinizing Dry Cleaning, 180 F.Supp.2d 584 (D.N.J. 2002) (“[A]nimal bioassays are of limited use in determining whether a particular chemical causes a particular disease....”). A healthy degree of skepticism is therefore warranted when evaluating causal inferences based on in vivo animal studies.

Because “[t]he observational nature of epidemiologic studies virtually always results in concerns about the results being skewed,” researchers must account for potential sources of error. RESTATEMENT (THIRD) OF TORTS: PHYS. & EMOT. HARM § 28 cmt. c(3) (AM. L. INST. 2010). The three most common forms of error are random error, bias, and confounding. RMSE at 572. Random error refers to the possibility that a result was reached purely by chance. The most common way to reduce the risk of random error in an epidemiological study is by increasing the number of participants because it is less likely that random error will taint the results of a study with 1,000 participants vs. a study with 10 participants. Id. at 576. Once a study is completed, the possibility of random error can be assessed by testing for statistical significance¹¹and/or specifying a confidence interval.¹² Id. at 573.

11 A statistically significant result is one that is unlikely the product of chance. To test for statistical significance, a researcher calculates a p-value representing the probability that an observed positive association resulted from random error. RMSE at 576. A p-value of .1 means that there is a 10% chance that values at least as large as the observed result could have been the product of random error. Id. Thus, the lower the p-value, the less likely it is that random error produced the observed result. Id. at 626. To minimize the likelihood of a false positive result, epidemiologists typically require the study's p-value to fall below a certain threshold (known as “alpha”). In most cases, the alpha is set at .05, meaning that the probability that one would observe an association at least as large as the one found in the study is 5%, when in truth no association was present. Id. at 577.

12 A confidence interval is a study's “margin of error.” In re Viagra (Sildenafil Citrate) and Cialis (Tadalafil) Prod. Liab. Litig., 424 F.Supp.3d 781, 787 (N.D. Cal. 2020). It indicates the “range of possible values calculated from the results of a study.” RMSE at 580. Researchers ordinarily assert a 95% confidence interval, meaning that “there is a 95% chance that the “true” odds ratio value falls within the confidence interval range.” In re Zoloft (Sertraline Hydrochloride) Prod. Liab. Litig., MDL No. 2342, 2015 WL 7776911, at *2 (E.D. Pa. Dec. 2, 2015).

Bias is another significant source of potential error in an epidemiological study. Two types of bias bear brief mention: selection bias and information bias. Selection bias involves a method of selecting study participants that produces an error in the observed association or result.¹³ Id. at 584. Selection bias also can occur when a researcher only selects data for a study that supports a given hypothesis. See Allgood v. Gen. Motors Corp., No. 102CV1077DFHTAB, 2006 WL 2669337, at *9-11 (S.D. Ind. Sept. 18, 2006) (Hamilton, J.) (discussing selection bias in the context of an expert's outcome-driven identification of data points). Information bias, on the other hand, involves inaccurate information from or about the study participants. RMSE at 585. A common form of information bias is the tendency of “individuals with disease (cases) . . . to recall past exposures more readily than individuals with no disease,” a phenomenon known as “recall bias.” Id. Finally, “[c]onfounding occurs when another causal factor (the confounder) confuses the relationship between the agent of interest and outcome of interest.” Id. at 591. Observational studies are particularly susceptible to confounding variables because the lack of randomization diminishes “the possibility that exposures other than the one under study are evenly distributed.” Id. at 592.

13 For instance, selection bias can occur if case-control study participants volunteered to join a study because of their exposure to the agent being studied. RMSE at 583. If volunteers join the study because they were exposed to the agent of interest, their selection could inflate the observed association. Id. at 584.

Before epidemiologists can draw conclusions about causation, they must assess whether there is an association between an exposure to the agent and the disease in question. Id. at 566. The two most common measures of the strength of an association are Relative Risk (“RR”) and Odds Ratio (“OR”). RR is defined as the “ratio of the risk of disease . . . among people exposed to an agent to the risk among the unexposed.” Id. at 627. For instance, if 10% of people who are exposed to an agent develop a disease compared to only 5% of people who were not exposed, then the disease occurs twice as frequently in people who were exposed. Id. The RR in this example would be 2 (an RR of 1 is used as a baseline to indicate no association between exposure and the disease).¹⁴ An OR is calculated by dividing the odds of a disease of interest occurring in an exposed person by the odds of the disease occurring in an unexposed person. Id. at 625. It is important to note that an association, even a statistically compelling one, “does not necessarily mean that there is a cause-and-effect relationship.” Id. at 566 (emphasis added). However, an association is the first step in an epidemiological causation analysis, and only after an association is found can an epidemiologist move on to assess whether the association might be causal.

14 When a confidence interval crosses the no-association threshold of 1 (thereby indicating no increased risk or even a decreased risk), the result is considered to be statistically insignificant. In re Viagra, 424 F.Supp.3d at 787.

Once a sufficiently strong association between an agent and a disease is found, epidemiologists consider whether the association could be the product of a cause-and-effect relationship. To judge causation, researchers ordinarily consider how the following nine “Bradford Hill” factors (named after the British epidemiologist and statistician, Sir Austin Bradford Hill)¹⁵ apply to an observed association:

15 See Austin Bradford Hill, The Environment and Disease: Association or Causation?, 58 Proc. Royal Soc'y Med. 295 (1965).

(1) temporal relationship (the exposure must occur before the disease develops);

(2) strength of association (the higher the RR or OR, the greater the likelihood that the relationship is causal);

(3) dose-response relationship (whether a higher dose increases the incidence or severity of the disease);

(4) replication of findings (whether research findings have been replicated in different populations with consistent results);

(5) biological plausibility (whether the association is consistent with current biological knowledge about the disease);

(6) consideration of alternative explanations (whether the research has properly accounted for bias and confounding variables);

(7) cessation of exposure (whether the cessation of exposure reduces the risk of disease);

(8) specificity of association (an association is specific and more likely to be causal if the exposure is associated with only a single or a small number of diseases); and

(9) consistency with other knowledge (whether a causal inference is consistent with relevant general knowledge or data).¹⁶

Id. at 599-607. “No algorithm exists for applying the [Bradford] Hill guidelines to determine whether an association truly reflects a causal relationship or is spurious.” RESTATEMENT (THIRD) OF TORTS: PHYS. & EMOT. HARM § 28 cmt. c(3). Thus, a causal relationship may exist even if one or more of the Bradford Hill factors is absent; similarly, a causal relationship does not necessarily exist by virtue of some factors being satisfied.¹⁷ RMSE at 600. Drawing causal inferences utilizing the Bradford Hill factors therefore always involves a degree of scientific judgment.

16 An example of this factor is the observed association between cigarette smoking and lung cancer. RMSE at 607. If an increase in lung cancer deaths followed a decrease in cigarette sales, such general knowledge might warrant skepticism about the existence of a causal relationship. Id.

17 The only exception is the first Bradford Hill factor-temporality. To cause a disease, the exposure to the agent necessarily must have occurred before the disease develops. RMSE at 601. Thus, temporality is the only necessary Bradford Hill factor. Id.

II. Systematic Reviews and Meta Analysis

Scientific studies of a given topic can (and often do) yield a range of findings and conclusions. As a result, the scientific community utilizes systematic reviews and metaanalysis to synthesize results and provide a more comprehensive overview of the state of the science. RMSE at 606-07. Systematic reviews seek to answer a defined research question by collecting the relevant scientific literature and summarizing the empirical evidence. Lisa A. Bero, Ph.D., Evaluating Systematic Reviews and Meta-Analyses, 14 J.L. & Pol'y 569, 570 (2006). Because systematic reviews require the reviewer to (a) search for relevant studies; and (b) decide which studies to include and exclude in the review, they can be corrupted by bias. To guard against this risk, “[a] good systematic review contains a focused question, an explicit and comprehensive search strategy [for the identification of relevant studies], explicit inclusion and exclusion criteria that are uniformly applied, a rigorous critical appraisal of each identified study and, if appropriate, a quantitative summary of the evidence.” Id. at 572. Reviewers should “develop a protocol for the review before commencement and adhere to the protocol regardless of the results of the review.” Id. at 575 (emphasis added). This requirement ensures that the reviewer is guided by scientific objectivity as she searches for and selects relevant studies for systematic review.

Meta analysis is a quantitative technique that allows a researcher to pool the results of multiple studies with the goal of arriving at a single figure (generally a numerical risk estimate) that represents the totality of the analyzed studies. RMSE at 607. Studies within the meta-analysis are assigned different weights based on “the sizes of their study populations and other characteristics.” Id. The more weight a study receives, the larger its influence will be on the final quantitative result. Thus, meta-analysis is a “quantitative approach to systematically combining the results of previous studies.” Bero, supra, at 570.

Although meta-analysis provides the benefit of data aggregation and synthesis, it comes with certain inherent vulnerabilities. As noted, when the relationship between a suspected harmful agent and a disease is at issue, randomized clinical trials involving human subjects are rarely (if ever) possible because ethical concerns prevent the researcher from deliberately exposing humans to the agent. As a result, epidemiological studies of such associations often rely on observational data as opposed to randomized clinical trials involving human subjects. Meta-analysis, however, “is most appropriate when used in pooling randomized experimental trials,” where most studies share core methodological characteristics. RMSE at 607. Observational studies, on the other hand, tend to vary more significantly in their respective methodological approaches, making it more challenging to reliably pool their results into a single risk estimate. Id. The Reference Manual on Scientific Evidence therefore cautions that “[p]eople often tend to have an inordinate belief in the validity of the findings when a single number is attached to them, and many of the difficulties that may arise in conducting a meta-analysis, especially of observational studies such as epidemiological ones, may consequently be overlooked.” Id. at 608.

Systematic reviews and meta-analyses can have a compelling effect on their audience because of their ability to offer an overarching conclusion or a single quantitative risk estimate in response to a complex problem. Thus, the treatment of varying results (heterogeneity) among the studies within a systematic review or metaanalysis warrants particular scrutiny. Id. The reason for this is apparent. When the results of the studies within the analysis are widely scattered, it becomes “harder to trust a single estimate of effect,” and “the reasons for such differences need at least be acknowledged and, if possible, explained.” Id.

PARAQUAT AND PARKINSON'S DISEASE

I. Relevant Epidemiological Studies

Many epidemiological studies have explored a potential association between paraquat and Parkinson's disease. Dr. Wells, for his part, reports that his research identified 36 studies that were responsive to the question of whether paraquat exposure was associated with Parkinson's disease. (Doc. 4355-3 at 45-46). Notwithstanding this extensive volume of epidemiological literature, Dr. Wells' opinions and Defendants' motion largely focus on the following nine studies.¹⁸

18 The Parties generally refer to studies by the principal author's last name. The Court will do the same unless the context clearly suggests otherwise.

Liou et al. (1997) ¹⁹ is a case-control study that examined the relationship between several environmental risk factors and Parkinson's disease in Taiwan. The study matched 120 patients with Parkinson's disease from the Movement Disorder Clinic of the National Taiwan University Hospital in Taipei with 240 control subjects from the same hospital. (Doc. 4355-22 at 3). The investigators interviewed study participants about their history of paraquat use, other herbicide and pesticide use, rural residence, source of drinking water, and other environmental risk factors. Id.

19 H.H. Liou et al., Environmental risk factors and Parkinson's disease; A case-control study in Taiwan, 48 Neurology 1583 (1997) (Doc. 4355-22).

Liou matched participants based on age and sex and found an odds ratio of 3.22, with a confidence interval of 2.41 - 4.31, in participants who had been exposed to paraquat “for at least 1 year before the onset of PD.” Id. at 4, 5. This result was based on a subset of 31 cases and 22 controls who reported paraquat exposure. Moreover, the reported odds ratio of 3.22 did not control for environmental risk factors that could have confounded the result. To address this issue, Liou reported multivariate adjusted odds ratios that isolated paraquat and herbicide and pesticide use from other environmental risk factors and adjusted for the length of a participant's exposure. Id. at 6. This analysis resulted in an odds ratio of 6.44, with a confidence interval of 2.41 - 17.2, for participants who reported 20 years or more of paraquat use. Id. Liou reported a substantially lower and statistically insignificant odds ratio of .96, with a confidence interval of .24 - 3.83, for participants with 1 to 19 years of paraquat use. Id. Based on this data, Liou concluded that “[h]aving a history of occupational herbicides/pesticides and paraquat use was associated with a significant increase in PD risk of about four to sevenfold after other environmental factors were adjusted.” Id. at 5.

Tanner et al. (2011) ²⁰ (also known as the Farming and Movement Evaluation (“FAME”) study) is a nested case-control study within the Agricultural Health Study (“AHS”), representing the AHS' second investigatory phase. (Docs. 4355-16 at 2; 4558-9 a 3). Tanner drew participants from the AHS population to assess their lifetime use of pesticides (including paraquat). (Doc. 4355-16 at 2). Diagnoses of Parkinson's disease were based on in-person examinations and required a consensus of two experts to minimize diagnostic misclassifications. Id.

20 Caroline M. Tanner et al., Rotenone, Paraquat and Parkinson's Disease, 119 Environmental Health Perspectives 866 (2011) (Doc. 4355-16).

Tanner found an odds ratio of 2.5, with a confidence interval of 1.4 - 4.7, among participants who had used paraquat. Id. at 5. This finding was adjusted for reference age, sex, state, and cigarette smoking using logistic regression. Id. Although Tanner adjusted for these potential confounders, the study authors acknowledged that “because most participants were exposed to many pesticides, we cannot confidently exclude effects of agents other than those studied or rule out the possibility that our results are attributable to combined exposures.” Id. at 7. Notwithstanding this potential limitation, Tanner concluded that its findings, in combination with earlier results, “suggest that paraquat use plays a role in human PD.” Id. at 6.

Hertzman et al. (1994) ²¹ was a community-based case-control study of environmental risk factors for idiopathic Parkinson's disease (“IP”) in the Onkanagan Valley in British Columbia. A neurologist diagnosed potential cases based on the presence of two of the four common symptoms of Parkinson's disease: bradykinesia, resting tremor, rigidity, and loss of postural reflexes. The study's participants included 127 confirmed cases. (Doc. 4355-21 at 2). The investigators recruited two control groups to address recall bias because “patients with chronic disease are introspective and more likely to recall remote exposures.” Id. at 3. The first control group consisted of 121 patients with cardiac disease; the second consisted of 124 individuals randomly selected from local voting rolls. Id. The study reported slightly elevated but statistically insignificant associations between paraquat exposure and Parkinson's disease. Parkinson's disease patients compared to patients with cardiac disease (the first control group) generated an odds ratio of 1.11, with a confidence interval of .32 - 3.87. Id. at 5. In comparison to the voter control group, Parkinson's disease patients generated an odds ratio of 1.25, with a confidence interval of .34 - 4.63.²² Id. Based on this data and other findings unrelated to paraquat, the investigators concluded that there was “an association between occupations, pesticide use, and IP, but we did not demonstrate a strong association between individual chemicals and IP.” Id. at 7.

21 Clyde Hertzman et al., A Case-Control Study of Parkinson's Disease in a Horticultural Region of British Columbia, 9 Movement Disorders 69 (1994) (Doc. 4355-21).

22 The reported data from Hertzman excluded women because “there were too few reports to permit an analysis of this variable.” (Doc. 4355-21 at 5). Dr. Wells calculated a combined odds ratio of 1.43, with a confidence interval of .33 - 7.04, which included women from the voting control group because he “believed this provided more complete information and . . . is more representative.” (Doc. 4355-3 at 24).

Kuopio et al. (1999) ²³ was a community-based case-control study in Finland that examined environmental risk factors for Parkinson's disease. The study matched 123 cases with 246 controls and tested for environmental risk factors, including exposure to domestic animals, smoking, occupation, and the use of herbicides and pesticides. (Doc. 4355-23 at 3). Only three cases and five controls reported having used paraquat. Id. at 5. Although Kuopio did not report a numerical risk estimate for the potential association between paraquat use and Parkinson's disease, Dr. Wells calculated an odds ratio of 1.21 with a confidence interval of .18 - 6.31, based on the data provided in the study. (Doc. 4355-2 at 19).²⁴

23 Anne-Maria Kuopio et al., Environmental Risk Factors in Parkinson's Disease, 14 Movement Disorders 928 (1999) (Doc. 4355-23).

24 Dr. Dominik Alexander, Defendants' expert on general causation, reached the same result, but concluded that it was “unsupportive of an increased risk of Parkinson's disease from paraquat exposure.” (Doc. 4355-6 at 66).

Rugbjerg et al. (2011) ²⁵ was a population-based case-control study in British Columbia that investigated potential associations between pesticide exposure and Parkinson's disease. The study matched 403 cases to 405 controls, but only three cases and three controls reported paraquat exposure. (Doc. 4355-26 at 3, 10). Although Rugbjerg did not report a risk estimate for the association between paraquat and Parkinson's disease, Dr. Wells calculated an odds ratio of 1.01, with a confidence interval of .13 - 7.55. (Doc. 4355-2 at 19).

25 Katherine Rugbjerg et al., Pesticide exposure and risk of Parkinson's disease - a population-based case-control study evaluating the potential for recall bias, 37 Scandinavian J. of Work, Env't & Health 427 (2011) (Doc. 4355-26).

Firestone et al. (2010) ²⁶ was a population-based case-control study of the risk of Parkinson's disease associated with various jobs and workplace exposures in western Washington State. Researchers identified potential cases from provider referrals or through diagnostic and pharmaceutical database searches. (Doc. 4355-25 at 3). A panel of neurologists confirmed diagnoses based on a review of medical records, requiring at least two of the four cardinal signs of Parkinson's disease (bradykinesia, resting tremor, cogwheel rigidity, and postural reflex impairment), one of which had to be bradykinesia or resting tremor. Controls did not have a history of Parkinson's disease or other progressive neurological conditions. All subjects were blinded to the study hypothesis to minimize recall bias, and men and women were evaluated separately because “their work activities are often dissimilar.” Id.

26 Jordan A. Firestone et al., Occupational Factors and Risk of Parkinson's Disease: A Population-Based CaseControl Study, 53 Am. J. of Indus. Med. 217 (2010) (Doc. 4355-25).

Although Firestone had a large participant pool (404 cases and 526 controls), only two cases and three controls reported paraquat exposure. The study revealed a statistically insignificant odds ratio of .9, with a confidence interval of .14 - 5.43. This result “d[id] not provide strong support for the hypothesis that exposure to any of these pesticides [including paraquat] affect the risk of PD.” Id. at 6. Overall, the study concluded that “[t]he growing scientific consensus is that PD is not a single disorder, but instead reflects a common pathological endpoint resulting from the interaction of various environmental and genetic risk factors.” Id. at 7.

Dhillon et al. (2008) ²⁷ was a case-control-study that examined potential associations between pesticide exposure and the development of Parkinson's disease in East Texas. Like Kuopio, Rugbjerg, and Firestone, Dhillon's study population only included a small number of participants who had been exposed to paraquat (four cases and one control). (Doc. 4355-24 at 8). As a result, Dhillon reported an odds ratio of 3.5, with a confidence interval of .4 - 31.6, indicating a wide margin of error. Id. at 5. Based on these findings, the authors conceded that “the numbers of subjects reporting [paraquat] exposure were small and the results were not statistically significant.” Id. at 10.

27 Amanpreet S. Dhillon, Pesticide/Environmental Exposures and Parkinson's Disease in East Texas, 13 J. of Agromedicine 37 (2008) (Doc. 4355-24).

Van der Mark et al. (2014) ²⁸ was a hospital-based case-control study in the Netherlands that explored potential associations between Parkinson's disease and occupational exposure to pesticides (including paraquat). (Doc. 4355-20 at 2). The study recruited participants from five hospitals across the Netherlands, matching 444 cases with 876 controls. Id. at 3. The investigators identified cases based on an initial diagnosis of Parkinson's disease by a healthcare provider and confirmed the diagnoses based on a medical records review by a neurologist. Id. Van der Mark relied on three different methods to estimate a participant's occupational exposure to the pesticides of interest: (i) a job exposure matrix that assigned exposure values based on a participant's occupational history; (ii) an algorithm that provided a more specific exposure estimation for participants with an occupational history of farming or gardening; and (iii) a cropexposure matrix that assigned exposure to specific active ingredients (including paraquat) based on the specific crops that a participant reported having cultivated. This exposure estimation approach was believed to generate more accurate exposure data because it was less susceptible to recall bias. Id. at 2-4.

28 Marianne van der Mark et al., Occupational exposure to pesticides and endotoxin and Parkinson's disease in the Netherlands, 71 J. of Occupational and Environmental Medicine 757 (2014) (Doc. 4355-20).

Van der Mark found an odds ratio of 1.27, with a confidence interval of .68 - 2.35, among participants with “low” amounts of exposure to paraquat, and an odds ratio of 1.03, with a confidence interval of .54 - 1.95, among participants with “high” exposure to paraquat. Id. at 7. After adjustment for potential confounders, including cumulative endotoxin exposure, the results remained roughly consistent, as participants in the “low” exposure tier generated an adjusted odds ratio of 1.42, with a confidence interval of .71 -2.85, and participants in the “high” exposure tier recorded an adjusted odds ratio of 1.01, with a confidence interval of .48 - 2.12. Id. Van der Mark concluded, based on these statistically insignificant results, that there was “no evidence for an association with pesticides and the functional subclasses: insecticides, herbicides and fungicides,” including paraquat. Id. at 7.

Shrestha et al. (2020) ²⁹ is a prospective cohort study of farming populations in North Carolina and Iowa. The study represents the third investigatory phase of the AHS, which evaluated 38,274 pesticide applicators and 27,836 of their spouses. The investigators explained that “with additional PD cases identified from extended followup as well as updated exposure data,” they were able to “examine[] associations between individual pesticides and incident PD that occurred over 20 years of follow-up among private pesticide applicators and their spouses.” (Doc. 4558-9 at 3). Investigators primarily relied on self-administered questionnaires and computer-assisted telephone interviews to enroll and follow up with study participants.

29 Srishti Shrestha et al., Pesticide use and incident Parkinson's disease in a cohort of farmers and their spouses, Env't Rsch. 191 (2020) (Doc. 4558-9).

Shrestha found a hazard ratio (“HR”)³⁰ of 1.09, with a confidence interval of .84 -1.41, for study participants who had reported paraquat use at enrollment.³¹ Id. at 6. Although this HR reflected only a moderate and statistically insignificant association between paraquat use and Parkinson's disease, Shrestha observed significant heterogeneity in their results after controlling for prior head injuries. This subgroup analysis found an HR for paraquat of 3.2, with a confidence interval of 1.38 - 7.45, among participants with a history of head injury. Id. at 7. Participants who did not have a history of head injury, on the other hand, had no increased risk of developing Parkinson's disease (HR of 1.00, with a confidence interval of .71 - 1.41). Id. This led the investigators to infer a “higher PD risk for use of . . . herbicides (paraquat and pendimethalin) among those who reported head injury.” Id.

30 A hazard ratio, like an odds ratio or relative risk, is a “measure of association used in epidemiology.” In re Lipitor (Atorvastatin Calcium) Marketing, Sales Practices and Prod. Liab. Litig., No. 2:14-cv-01879, 2016 WL 8739553, at *11 (D.S.C. Dec. 29, 2016) (citation omitted). Dr. Wells explains a hazard ratio as a measure of the “instantaneous risk of the outcome occurring in the exposed group relative to the unexposed group.” (Doc. 4355-3 at 4).

31 Based on the large number of participants in the AHS and the extended follow-up period, Shrestha reported data for remaining study participants who had not been excluded from the study over the years due to a lack of responsiveness to follow-up inquiries or missing and inconsistent information concerning their Parkinson's diagnosis. (Doc. 4558-9 at 2).

Shrestha is also notable because it reports data from the AHS and thus includes the population that was studied in Tanner (2011). Shrestha's lower reported risk estimate compared to Tanner's prompted the authors to explore the “[l]imited reproducibility” of Tanner's results. Id. at 8. The authors hypothesized that the inconsistent results may be attributable to “differences in study design, exposure data, and criteria for inclusion in analyses.” Id. Whatever the cause of the lack of reproducibility between Tanner and Shrestha, Shrestha concluded that there was “limited evidence for independent associations of incident PD with these pesticides [including paraquat].” Id.

II. Review by the Environmental Protection Agency (“EPA” or “Agency”)

The Federal Insecticide, Fungicide, and Rodenticide Act (“FIFRA”) delegates regulatory and enforcement authority over the “use, . . . sale and labeling, of pesticides” to the EPA. Ruckelshaus v. Monsanto Co., 467 U.S. 986, 991-92 (1984); see also 7 U.S.C. § 136w. Covered pesticides under FIFRA, including paraquat, must be registered with the EPA, 7 U.S.C. § 136a(a); 40 C.F.R. § 152.175, and such registrations are subject to periodic registration reviews. 7 U.S.C. § 136a(g)(1)(A); see also Hardeman v. Monsanto Co., 997 F.3d 941, 950 (9th Cir. 2021) (discussing regulatory framework under FIFRA). In 2019, paraquat underwent such a registration review. To that end, the EPA conducted a systematic review of the available literature to assess the potential relationship between paraquat and Parkinson's disease.³² (Doc. 4558-12). This review “evaluat[ed] the significance and environmental relevance of the postulated association between paraquat exposure and PD.” Id. at 5.

32 See Austin Wray & Aaron Niman, Memorandum, Paraquat Dichloride: Systematic review of the literature to evaluate the relationship between paraquat dichloride exposure and Parkinson's disease (June 26, 2019) (Doc. 4558-12).

The EPA extensively reviewed data from epidemiological, animal, and in vitro studies.³³ The review of the epidemiological literature distinguished between studies of occupational and non-occupational populations because “exposure pathways . . . vary in terms of magnitude, frequency, and duration, with occupational study populations being more likely to experience exposure as a result of direct use of paraquat.” Id. at 90. Several studies mentioned above, including Tanner (2011), van der Mark (2014), Liou (1997), Firestone (2010), Dhillon (2008), and Hertzman (1994), were evaluated to examine a potential association and causal relationship between occupational paraquat exposure and Parkinson's disease.³⁴

33 For purposes of this order, the Court will only address the EPA's examination of the epidemiological literature as animal and in vitro studies are beyond the scope of Dr. Wells' engagement.

34 Shrestha (2020) was not published until 2020 and was therefore not part of the EPA's review. Kuopio (1999) and Rugbjerg (2011) likewise were not considered.

Only Tanner was considered to be of “high quality” under the EPA's evaluation criteria based on its design as a nested case-control study and its detailed diagnostic criteria. Tanner's limitations, according to the EPA, included the “relatively small number of paraquat exposed PD cases” (23 in total) and the potential for recall bias. Id. at 19-20. Van der Mark was classified as “moderate quality,” with a strength being its recruitment from hospital neurology departments. Van der Mark's low participation rate and reliance on a crop exposure matrix were considered qualitative weaknesses. Id. at 28. Liou was also considered to be of “moderate quality.” Liou's hospital-based case-control design was a strength of the study, whereas its reliance on general questionnaires to assess pesticide use “may have introduced recall bias if cases and controls recall their past pesticide use differently.” Id. at 28-29. Firestone, Dhillon, and Hertzman were judged to be of “low quality” given the small numbers of exposed cases, which provided “insufficient information on the association between paraquat exposure and PD.” Id. at 29-31. Based on its review of these and several other epidemiological studies, the EPA concluded that “there is limited, but insufficient epidemiologic evidence at this time to conclude that there is a clear associative or causal relationship between occupational paraquat exposure and PD.” Id. at 35.

In July 2021, the EPA issued its Interim Registration Review Decision (“Interim Decision”) reapproving paraquat's registration. See Plaintiffs' Consolidated Motion to Exclude Testimony from Defendants' Experts, Exh. 9 (Doc. 4364-10 at 42-43). The Agency concluded that reapproval was appropriate because, based on its systematic review, “the weight of evidence was insufficient to link paraquat exposure from pesticidal use of U.S. registered products to Parkinson's disease in humans.” Id. at 18. The Agency also found that paraquat provided “high benefits” for the cultivation of various crops, which outweighed potential risks. Id. at 43.

Notwithstanding these findings, the EPA determined that certain mitigation measures and label changes were necessary to bring paraquat into compliance with FIFRA. Id. Mitigation measures included the prohibition of certain application tools, such as handguns and backpack sprayers; increases in restricted entry intervals, meaning that workers are not permitted to re-enter paraquat treated areas for a certain period of time; and requirements that respirators or enclosed cabs be used to protect workers from inhaling paraquat during application. Id. at 32-36.

In September 2021, several interest groups led by the California Rural Legal Assistance Foundation (“CRLAF Petitioners”) filed a petition contesting the Interim Decision in the United States Court of Appeals for the Ninth Circuit. See generally, Cal. Rural Legal Assistance Found. v. EPA, No. 21-71287 (9th Cir. 2021) (hereinafter “CRLAF v. EPA”). The CRLAF Petitioners claimed that the Interim Decision “understated the extent of paraquat's adverse effects, and . . . failed to lawfully address the serious risks it did identify.” CRLAF v. EPA (Doc. 27-1 at 10). The CRLAF Petitioners thus asked the Ninth Circuit to remand the Interim Decision to the EPA to conduct, what they considered to be a more comprehensive review of the science surrounding paraquat and its relationship to Parkinson's disease. Id. at 11. In November 2022, the EPA moved to hold the case in abeyance as it “consider[ed] the substantive issues raised” by the petition. CRLAF v. EPA (Doc. 56-1 at 2). On January 30, 2024, the EPA published its preliminary findings in response to the petition, addressing “concerns raised about [its] assessment of whether paraquat poses a risk of Parkinson's Disease.” (Doc. 5121-1 at 4). After evaluating paraquat's purported health risks and weighing them against its utility in various farming operations, the EPA affirmed its conclusion that the risks were “outweighed by the benefits of the use of paraquat.” Id. at 21.

Discussion

Before an expert is permitted to testify, it is the Court's duty to ensure that she “employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Kumho Tire, 526 U.S. at 152. In the Seventh Circuit, this means that experts are required to “follow scientific approaches normal to their disciplines.” Zenith Elec. Corp. v. WH-TV Broad. Corp., 395 F.3d 416, 419 (7thCir. 2005). Here, Dr. Wells, an accomplished biostatistician and epidemiologist, offers an opinion that occupational exposure to paraquat can cause Parkinson's disease. Although Dr. Wells presents certain data points that support this conclusion, his proffered opinion required several methodological contortions and outright violations of the scientific standards he professed to apply. These methodological deficiencies, in turn, suggest that Dr. Wells failed to apply the same level of intellectual rigor to his work in the four trial selection cases that would be required of him and his peers in a nonlitigation setting. Accordingly, the Court is unable to conclude, based on a preponderance of the evidence, that Dr. Wells' proffered opinions are sufficiently reliable to be presented to a jury. The following discussion explains why.

I. Dr. Wells' Expert Opinions and Methodologies

In his own words, Dr. Wells offers an opinion that “the available epidemiological evidence supports a causal relationship between occupational paraquat exposure and PD.” (Doc. 4355-2 at 26). In support of this opinion, Dr. Wells submitted two expert reports and extensive deposition testimony explaining the methodologies he applied to reach his conclusions. These reports and depositions are summarized below insofar as they offer probative evidence of the methodological soundness of Dr. Wells' opinions.

A. Dr. Wells' First Expert Report

Dr. Wells submitted his first expert report on October 13, 2022. (Doc. 4355-2). In it, he surveyed the epidemiological literature on the relationship between paraquat and Parkinson's disease and discussed the merits and demerits of various studies and systematic reviews. The first step in Dr. Wells' review was to identify the five most recent systematic reviews that explored the paraquat-Parkinson's disease relationship (Breckenridge (2016),³⁵ Vaccari (2017)³⁶ and (2019),³⁷ Tangamornsuksan (2018),³⁸ and Wray and Niman (2019)³⁹). After briefly discussing each of these systematic reviews and the studies within them, Dr. Wells conducted his own meta-analysis to identify what he considered to be an appropriate pooled risk estimate.

35 Charles B. Breckenridge et al., Association between Parkinson's Disease and Cigarette Smoking, Rural Living, Well-Water Consumption, Farming and Pesticide Use: Systematic Review and Meta-Analysis, PLoS ONE, Vol. 11, No. 4 (2016) (Doc. 4355-33).

36 Carolina Vaccari et al., Paraquat e doenca de Parkinson: revisao sistematica e metanalise de estudos observacionais, Dissertacao apresentada a Faculdade de Medicina, Universidade Estadual Paulista (2017) (unpublished).

37 Carolina Vaccari et al., Paraquat and Parkinson's disease: a systematic review and meta-analysis of observational studies, 22 Journal of Toxicology and Environmental Health 172 (2019) (Doc. 4355-38).

38 Wimonchat Tangamornsuksan et al., Paraquat exposure and Parkinson's disease: A systematic review and metaanalysis, Archives of Environmental & Occupational Health (2018) (Doc. 4355-37).

39 Austin Wray & Aaron Niman, Memorandum, Paraquat Dichloride: Systematic review of the literature to evaluate the relationship between paraquat dichloride exposure and Parkinson's disease (June 26, 2019) (Doc. 4558-12).

Dr. Wells' meta-analysis included seven “[o]ccupational case-control studies . . . for participants with in-person evaluations, including a standardized medical and neurological history and examination by movement disorder specialists.” (Doc. 4355-2 at 19). The seven included studies represented a subset of the available epidemiological literature, meaning that Dr. Wells excluded a significant amount of relevant information from his meta-analysis. With respect to van der Mark, for instance, Dr. Wells explained that he excluded it because it “was not restricted to agricultural workers,” its exposure and diagnostic criteria were, in his view, unreliable, and because it had a low participation rate. Id. at 16. Dr. Wells also excluded other potentially eligible case-control studies from his meta-analysis because they did not examine paraquat exposure with sufficient specificity, relied on parkinsonism as opposed to Parkinson's disease to identify cases, or because their case identification procedures were insufficiently reliable. Id. at 16-17. Although Dr. Wells offered these explanations in an ad hoc manner, his first report does not outline specific criteria that were systematically applied to all relevant studies to determine whether they would help inform his analysis of a potential association.

After aggregating the seven studies that made the cut for his meta-analysis, Dr. Wells determined that they yielded a pooled odds ratio of 2.88 (later revised to 2.84), indicating a “near tripling of PD occurrence in participants occupationally exposed to paraquat.” Id. at 18. Dr. Wells presented his fixed-effect meta-analysis in the following forest plot:

(Image Omitted) Id. at 19. Although seven case-control studies were included, Liou and Tanner made up over 90% of the weight of the resulting pooled odds ratio, meaning that the remaining five studies (Hertzman; Kuopio; Dhillon; Firestone; and Rugbjerg) were mere fringe contributors. Moreover, Liou's odds ratio of 3.22, with a confidence interval of 2.41 - 4.31, contributed nearly 74% of the weight to Dr. Wells' pooled odds ratio.

After establishing a positive association, Dr. Wells conducted a Bradford Hill analysis to determine whether occupational exposure to paraquat was causally related to Parkinson's disease. Dr. Wells discussed six of the nine Bradford Hill factors (strength of association, consistency, temporality, dose-response, experimental evidence regarding the cessation of exposure, and specificity) in his first report, and assumed that the other three factors (biological plausibility, coherence, and analogy) were satisfied based on the report of Dr. Vanessa Fitsanakis, Plaintiffs' toxicology expert. Id. at 26. He appears to have found that the six factors he evaluated were all satisfied, and on that basis, concluded that “drawing general causal inferences related to occupational paraquat exposure and PD is merited.” Id. Dr. Wells also assumed that the four trial selection plaintiffs met the exposure and diagnostic criteria of the seven studies in his meta-analysis, leading him to conclude that “[t]he elevated odds ratio of 2.8[] and the [Bradford] Hill criteria apply to these individuals.” Id. at 27.

B. Dr. Wells' First Expert Deposition

At his first deposition, Dr. Wells was confronted about the methodological rigor of his analyses and opinions. Beginning with the scope of his assignment, Dr. Wells testified that he was hired to examine epidemiological studies that investigated the relationship between occupational paraquat exposure and Parkinson's disease. Dr. Wells eschewed a clear definition of “occupational” exposure, and instead, relied on each individual study's definition of the concept to determine whether it was sufficiently relevant to his assignment. When pressed, Dr. Wells testified that studies of occupational paraquat exposure, to him, were “primarily focused on . . . people's work and, you know, exposure because of their workplace.” (Doc. 4364-20 at 6). Under this definition, however, Dr. Wells was forced to concede that Liou, the study that carries nearly 74% of the weight of his meta-analysis, “did not meet [his] own stated criteria for occupational exposure” because it “included residential exposure as well.” Id. at 31-32. He justified his decision to nevertheless include Liou in his meta-analysis because it had “good recruitment, good diagnoses, and a-seemed to follow the patients well.” Id. at 32.

Dr. Wells' initial report stated that his meta-analysis was limited to studies in which subjects had “in-person” evaluations by a movement disorder specialist. At his deposition, however, Dr. Wells deviated from this criterion: he testified that he did not require an in-person evaluation by a movement disorder specialist because one of the studies he included identified cases through a review of medical records (Firestone 2010). Id. at 27. Ultimately, Dr. Wells explained that he selected studies for his meta-analysis based on a “holistic assessment of whether or not that study was reliable enough for inclusion.” Id. at 29. The following exchange attempted to elucidate his approach:

QUESTION: So[,] you didn't have specific eligibility criteria, a set of rules that a study had to meet in order to be included in your meta-analysis?

DR. WELLS: I, as I said, there's - I looked at all the features of the study.

QUESTION: Okay. Can you give me a set of rules that a study would have needed to meet in order to be included in your meta-analysis?

DR. WELLS: So[,] you have to look at the, you know, what the case - the assessment of the disease. You have to look at the assessment of exposure. You have to look at, you know, were the cases actually cases? Were the controls selected from the same population? Were they - did they actually have PD? Was it designed well?

So[,] there's all these features that go into trying to understand whether it's a good epidemiological study.

QUESTION: So[,] I understand those are the factors that you looked at, but you cannot give me a list of criteria that each study would have had to meet in order to be included in your meta-analysis. Instead you took a holistic view.

DR. WELLS: I took a holistic view. And then you look at each of those factors and decide whether it's a good - you know, that satisfies good epidemiological practice or not.

Id. Dr. Wells never reduced his “holistic” review process to writing and as a result, appeared to concede that his process was not objectively replicable. For instance, in response to a question about the weight Dr. Wells assigned to participation rates in the relevant studies, he simply stated: “It's just part of the-my assessment. I looked at all these criteria, as I said, and I made a decision whether it should be in-should be included or not included .... [T]here's no algorithm to replicate.” (Doc. 4364-20 at 66) (emphasis added). If another researcher wanted to replicate his inclusion/exclusion decisions, Dr. Wells stated that he “think[s] they would come to the same place. If we-if we talked about what are good-quality studies and what are the issues that-where-what studies don't give as reliable evidence, they would probably come to the same place.” (Doc. 436420 at 30).

C. Dr. Wells' Rebuttal Expert Report

On April 19, 2023, Dr. Wells submitted his rebuttal report in response to the proffered opinions of Dr. Dominik Alexander, Defendants' epidemiology expert. (Doc. 4355-3). The rebuttal report reflected a methodological sea change in Dr. Wells' expert analysis as it provided much more granular and even previously undisclosed explanations of his study selection methodology. The study selection process for his meta-analysis, as the rebuttal report now explained, proceeded in two steps: (1) a determination of whether a study was eligible for inclusion in the meta-analysis based on five “inclusion/exclusion” criteria; and (2) a qualitative evaluation of eligible studies based on five quality criteria. To be eligible, a study had to (i) evaluate “occupational” exposure; (ii) specifically address paraquat exposures; (iii) identify Parkinson's disease as the outcome of interest; (iv) utilize a case-control design; and (v) report “sufficient data.” (Doc. 4355-3 at 12-16). Eligible studies were then qualitatively judged based on (i) their diagnostic criteria for the identification of cases; (ii) their exposure assessment methodologies; (iii) the composition of control groups; (iv) participation rates; and (v) their control of confounding variables. Id. at 17-22, 48. Although the rebuttal report outlined a much more comprehensible selection process, certain eligibility and quality factors were either modified or newly added altogether.

With respect to his first eligibility criterion-studies investigating “occupational” exposure-Dr. Wells seemingly abandoned his focus on workplace-related studies. The rebuttal report claimed that Dr. Wells “analyze[d] the epidemiological evidence of an association between PD and occupational or direct exposure to paraquat in the course of work, agricultural, or residential applications-in other words, individuals who used or had potential dermal contact with paraquat-as opposed to indirect community exposure through, e.g., drift.” Id. at 11 (emphases added). “Occupational exposure” was redefined as being “related to the use of paraquat, or contact with paraquat where there is the risk of dermal exposure.” Id. at 12. Of course, this revised objective no longer distinguished between studies that evaluated workplace-related paraquat exposure and residential exposure studies. Instead, studies that investigated paraquat exposure in the form of “potential dermal contact” now qualified as “occupational” exposure studies.

The rebuttal report also explained that Dr. Wells judged the quality of the eligible studies according to five quality factors. In this step of the selection process, each eligible study was assigned a “higher quality” or “lower quality” rating for each quality factor. Id. at 48. Here again, Dr. Wells' methodology appeared to shift from the “holistic” selection process he had described at his deposition. Most notably, studies in which more than 60% of cases and controls agreed to participate received a “higher quality” rating for participation rate, whereas studies that did not meet this threshold were assigned a “lower quality” rating. Although Dr. Wells previously noted van der Mark's low participation rate, the 60% threshold appeared in his rebuttal report for the first time. Dr. Wells even told defense counsel at his first deposition, “I don't have a number” when asked whether there was a “clear requirement regarding participation rates for inclusion in [his] analysis.” (Doc. 4364-20 at 65).

The quality factor of a study's exposure assessment underwent a similar metamorphosis from the general to the specific. Dr. Wells testified at his first deposition that he evaluated a study's exposure assessment to determine whether it “was done well” and “reliable.” Id. at 19. The rebuttal report, however, stated that to qualify for a “higher quality” rating on this criterion, a study had to conduct “personal interview[s] using detailed exposure questionnaires.” (Doc. 4355-3 at 48).

The rebuttal report also explained how, after applying his two-step methodology, Dr. Wells arrived at the collection of seven studies that made up his meta-analysis. According to the rebuttal report, Dr. Wells began with 36 epidemiological studies that were relevant to the association between paraquat and Parkinson's disease. Id. at 12, 45-46. He applied the five eligibility criteria to this universe of studies, which reduced the number of eligible studies to eight. Then, Dr. Wells applied the five quality factors to the eight remaining studies, where only van der Mark (2014), the study that found “no evidence” of an association between paraquat and Parkinson's disease, was disqualified. Once again, however, this process yielded conspicuously different results than Dr. Wells' first report had offered. The first report provided a list of 11 case-control studies that were ostensibly “[i]ncluded in [Dr. Wells'] [a]nalyses.” (Doc. 4355-2 at 37-40). Dr. Wells also provided an addendum to his first report containing a list of 111 informational sources that made up his reliance material. A list of the 36 studies that Dr. Wells claimed to have systematically reviewed is nowhere to be found in the first report. Thus, at a minimum, the rebuttal report exposes the lack of a systematic selection of studies in Dr. Wells' first report.

D. Dr. Wells' Second Expert Deposition

At his second deposition, Dr. Wells attempted to reconcile the methodological inconsistencies between his first report and deposition, and his rebuttal report. But beginning with his search for relevant literature, Dr. Wells was unable to articulate (or at least recall) a search strategy that led to his identification of 36 studies that were systematically reviewed according to the eligibility and quality criteria he laid out in his rebuttal report. On this point, Dr. Wells offered the following testimony:

QUESTION: And how would I follow your methodology to determine which ones end up in this list of 36 and which ones do not end up in this list of 36?

DR. WELLS: Yeah. I can't recall what the threshold was to-

QUESTION: Was there a threshold?

DR. WELLS: . . . I'm not sure at this point-or I can't remember at this point how this 36 came to be the ones that I looked at versus the ones that aren't on that list. I mean, would-I'd like to know what am I missing? I mean, am I missing anything? ...

QUESTION: So as you sit here today, you cannot describe for me the criteria that a study needed to fill to meet-to get on your list of 36?

DR. WELLS: Yeah, I can't remember the details.

QUESTION: So[,] you can't remember the details. Do you remember the criteria that a study would need to meet in order to make it on this list of 36?

DR. WELLS: I can't remember what it was to get to those 36.

(Doc. 4561-10 at 26-27). Although he could not recall his search strategy that yielded the initial set of 36 studies, Dr. Wells insisted that his identification of relevant studies was “obvious,” and that it was therefore unnecessary to explain it in his original report. Id. at 15. Moreover, regarding his two-step selection process to identify studies for inclusion in his meta-analysis, Dr. Wells testified that it was his standard practice to apply it in every meta-analysis he had previously done. Curiously, however, he was unable to point to any prior publication that would validate this claim. Id. at 17. Moreover, notwithstanding his claimed standard practice, Dr. Wells admitted that he came up with the five quality factors by which he evaluated the eight eligible case-control studies after he read them. Id. at 46.

Dr. Wells also attempted to clarify his understanding of “occupational” paraquat exposures. He testified that his assignment was to assess the risk of developing Parkinson's disease from “direct contact” with paraquat. (Doc. 4561-10 at 37). Direct contact, he testified, had nothing to do with “occupational” exposures. Rather, the focus was “direct contact with you, not occupational, not drift or anything like that” and it did not include “inhalation exposure.” Id. “Direct contact,” according to Dr. Wells, simply meant “you're using it” and “it's on you,” regardless of whether the exposure occurred at home or in the workplace. Id. at 38. Under this broader definition, Dr. Wells gave himself more flexibility to justify his inclusion and exclusion decisions, particularly with respect to Liou (1997), which examined occupational and residential exposures.

Dr. Wells also offered a new limitation on the scope of his expert opinion in his second deposition. He testified that in order for his calculated odds ratio of 2.8 to apply, a person had to be exposed to paraquat for 25 days or more. Id. at 39. In other words, his opinion that occupational paraquat exposure was capable of causing Parkinson's disease required a minimum exposure period of 25 days. Dr. Wells based this limitation on a sensitivity analysis in Tanner (2011), which applied the 25-day threshold to control for co-exposures to other pesticides. Although Tanner offered this sensitivity analysis without reporting an odds ratio, none of the other studies in Dr. Wells' meta-analysis based their reported odds ratios on a similar temporal limitation.⁴⁰ Dr. Wells justified his own 25-day restriction on the basis that “it's harder to attain,” and therefore more reliable than an ever/never exposure classification. Id. at 40.

40 Liou (1997) defined a positive exposure as one that occurred “for at least 1 year before the onset of PD.” (Doc. 4355-20 at 4).

II. Legal Analysis

A. Qualifications

Defendants first attack Dr. Wells on the basis that he is not qualified to offer an opinion about the causal relationship between occupational paraquat exposure and Parkinson's disease. This argument purports to show that although Dr. Wells possesses impressive credentials as a statistician, his qualifications are limited to just that type of work-calculating a summary risk estimate from a pre-selected universe of epidemiological studies. Thus, Defendants contend that Dr. Wells has no business (i) conducting a weight of the evidence review applying the Bradford Hill factors; and (ii) opining on the relative quality of epidemiological studies. These arguments miss the mark.

Whether an expert is qualified to testify under Rule 702 does not depend on her ability to check certain boxes on a list of required credentials. Gayton v. McCoy, 593 F.3d 610, 617 (7th Cir. 2010). Rather, an expert's qualification is based on her “full range of practical experience as well as academic or technical training.” Smith v. Ford Motor Co., 215 F.3d 713, 718 (7th Cir. 2000). At the same time, it is not enough for an expert to waltz into court with an impressive resume. Id. The Court must examine the expert's full range of experience against the specific opinions she intends to offer. Gayton, 593 F.3d at 617. Thus, while Dr. Wells' experience and credentials matter broadly, the inquiry must be tailored to “each of the conclusions he draws individually to see if he has the adequate education, skill, and training to reach them.” Id. Here, Defendants contest Dr. Wells' qualifications to (i) offer an opinion that paraquat is causally related to Parkinson's disease; and (ii) judge the relative quality of epidemiological studies. The Court will evaluate Dr. Wells' qualifications to offer each of these opinions separately. See id.

At the outset, it is worth noting that Dr. Wells testified at the Daubert hearing that he is an epidemiologist, not just a statistician. (Doc. 4793 at 62). He is exceptionally well-credentialed in both fields. He received his Ph.D. in Mathematics from the University of California in 1987 and became an Assistant Professor at Cornell University that same year. He currently serves as a Professor of Clinical Epidemiology and Health Services Research at Weill Medical School and as the Chair of the Department of Statistics and Data Science at Cornell. He has published 250 scholarly articles on statistics, human health, and other topics and has received research grants from numerous governmental and non-governmental institutions, including the U.S. Department of Agriculture, the U.S. Army, and the National Institutes of Health. (Doc. 4355-2 at 41-59). Dr. Wells' resume plainly demonstrates an impressive career as an academic, researcher, and prolific publisher in the fields of biostatistics and epidemiology.

This is also not Dr. Wells' first rodeo as an expert witness. The Parties cited several cases in which Dr. Wells offered opinions as a general causation expert, although they disagree as to the scope of those opinions and whether they establish Dr. Wells' qualifications to opine on general causation in this MDL. Indeed, Defendants' entire argument that Dr. Wells lacks the qualifications to offer a general causation opinion is based on prior cases in which they claim Dr. Wells' role was “limited to statistical issues.” (Doc. 4355 at 24). Thus, so the argument appears to go, because other courts have limited Dr. Wells' testimony to statistical matters, so too should this Court.

Defendants' argument appears to disregard the prior cases in which Dr. Wells did offer an opinion on general causation based on the Bradford Hill framework. In Monroe v. Zimmer U.S., Inc., Dr. Wells examined “whether a causal relationship existed between the use of intra-articular pain pumps and the development of chondrolysis.” 766 F.Supp.2d 1012, 1021 (E.D. Cal. 2011). This analysis required him to apply four of the nine Bradford Hill criteria (strength of association, consistency, temporality, and alternative explanations) to the epidemiological literature. Id. at 1025. Dr. Wells' review of the relevant epidemiological studies and his application of the four Bradford Hill considerations led him to conclude that intra-articular pain pump use was a “primary causative factor” of chondrolysis. Id. at 1023. The court found that Dr. Wells was qualified under Rule 702 to offer this opinion because it was appropriately limited to the Bradford Hill factors that were within his area of expertise. Id. at 1024.

Here, Dr. Wells' general causation opinion is similarly based on a Bradford Hill analysis of the relevant epidemiological evidence. In addition to the Bradford Hill factors of strength of association, consistency, and temporality (which he also considered in Monroe), Dr. Wells discusses experimental evidence related to the cessation of exposure, evidence of a dose-response relationship, and specificity. The Court is not persuaded that his discussion of these considerations pushes his testimony beyond the limits of his qualifications as a biostatistician and epidemiologist. Dr. Wells' Bradford Hill analysis is appropriately limited to the epidemiological studies he reviewed. He does not purport to venture into the realm of toxicology or other disciplines that are beyond his area of expertise. Indeed, he expressly disclaims his ability to do so by deferring discussion of the remaining three Bradford Hill factors (biological plausibility, coherence, and analogy) to Plaintiffs' toxicology expert, Dr. Vanessa Fitsanakis. The Court is therefore satisfied that, as in Monroe, Dr. Wells is “capable of understanding the data reported in studies conducted by medical professionals” and that his causation opinion is based on his understanding of statistics and epidemiology. Monroe, 766 F.Supp.2d at 1024; see also Woodard v. Stryker Corp., No. 11-CV-36-F, 2012 WL 3475079, at *2 (D. Wyo. Jul 16, 2012) (“Most [c]ourts have allowed [Dr. Wells] to testify.”).

Defendants' argument that Dr. Wells is unqualified to judge the relative quality of epidemiological studies fares no better. Here again, Defendants hang their hat on Dr. Wells' stellar credentials as a statistician to argue that he lacks the “scientific judgment” of an epidemiologist to evaluate the strengths and weaknesses of epidemiological studies. (Doc. 4355 at 26). But as noted, Dr. Wells is not just a statistician. He is also an epidemiologist and a Professor of Clinical Epidemiology at Weill Medical School. Indeed, it strikes the Court as illogical to say that a professor of epidemiology is unqualified to analyze the quality of an epidemiological study when that task lies at the very heart of his profession.

Defendants cite the Welding Fume litigation in the Northern District of Ohio in support of their position that Dr. Wells is unqualified to evaluate the merits of the epidemiological literature. See In re Welding Fume Prod. Liab. Litig., No. 1:03-CV-17000, 2010 WL 7699456, at *39-40 (N.D. Ohio June 4, 2010). That case, however, appears to have reached the opposite conclusion that Defendants press here. The defendants in Welding Fume moved to exclude Dr. Wells on the basis that he was “simply a statistician” and therefore unable to judge the merits of the relevant epidemiological studies. Id. at *39. The court “easily concluded” that this characterization of Dr. Wells' qualifications was “not well-taken.” Id. The court found that Dr. Wells' “understanding of statistical principles and methodology relevant to the design and interpretation of epidemiological studies” was “thorough and clear.” Id. “Calling Dr. Wells ‘simply a statistician'” was therefore “both unfair and uninformed.” Id.

This Court reaches the same conclusion. Dr. Wells is not just a statistician; he is a professor of epidemiology with an impressive record of scholarly publications on epidemiological issues.⁴¹As such, he is well-equipped to judge the relative quality of the epidemiological studies at issue in this MDL. To the extent that his evaluation of the quality of the relevant epidemiological studies required him to solicit input from other experts in the case, he did so. For instance, Dr. Anthony Lang, Plaintiffs' neurology expert, validated Dr. Wells' scientific judgments regarding the diagnostic criteria employed in the epidemiological studies at issue here. (Doc. 4561 at 50). Dr. Wells' reliance on other experts to complement his analysis does not impugn his own qualifications to judge the quality of the epidemiological studies in this case; it enhances them. See Jones v. PepsiCo., Inc., 185 F.Supp.3d 437, 446 (S.D.N.Y. 2016).

41 Although the court in Welding Fume noted that Dr. Wells was “not an epidemiologist,” it found him to be qualified under Rule 702 to opine on the strengths and weaknesses of relevant epidemiological studies. In re Welding Fume, 2010 WL 7699456, at *39. The Court is somewhat puzzled by this finding in Welding Fume, considering Dr. Wells' testimony that he is an epidemiologist and a professor of clinical epidemiology. In any event, this finding in Welding Fume does not alter the Court's view of Dr. Wells' qualifications in this case.

Simply put, as a biostatistician and epidemiologist, Dr. Wells is well qualified to offer a general causation opinion based on a Bradford Hill analysis and to evaluate the relative quality of epidemiological studies relevant to the causation question at issue. Moreover, his opinions are properly contained within the scope of his qualifications, which bolsters this Court's belief that Dr. Wells did not stray beyond the limits of his scientific acumen. Thus, consistent with other courts that have considered Dr. Wells' qualifications, the Court finds that Dr. Wells is qualified under Rule 702 to offer the opinions proffered in this MDL.

B. Reliability

An epidemiological causation assessment ordinarily proceeds in two steps: (i) a determination of whether a disease is associated with exposure to a particular agent; and (ii) if a positive association is found, a Bradford Hill analysis to determine whether the association is the result of a cause-and-effect relationship. RMSE at 566, 597. At a high level, Dr. Wells followed these steps as well. First, he surveyed the epidemiological literature and established a positive association by conducting a meta-analysis of seven case-control studies that examined a possible association between paraquat exposure and Parkinson's disease. Second, he conducted a Bradford Hill/weight of the evidence analysis to determine whether the totality of the evidence supported a causal relationship between occupational paraquat exposure and Parkinson's disease. Dr. Wells then offers the opinion that the elevated odds ratio from his meta-analysis applies to the four trial selection plaintiffs, Mr. Richter, Mr. Burgener, Mr. Fuller, and Mr. Coward. The Court will discuss the reliability of each of these conclusions separately. See Gayton, 593 F.3d at 617.

1. The Scope of Dr. Wells' General Causation Opinion

Dr. Wells offers an expert opinion that “occupational” paraquat exposure is causally related to Parkinson's disease. Any exposure that does not qualify as “occupational” is therefore not within the scope of his opinion. So, what is “occupational” exposure? The record reveals a strikingly amorphous definition of this term. Indeed, Dr. Wells redefined “occupational” exposure no less than three times, creating more questions than answers about the types of paraquat exposures that, according to him, can cause Parkinson's disease.

Beginning with Dr. Wells' first report, he appears to define “occupational” exposure by distinguishing it from “community” exposure. Yet his first report does not define “community” exposure, leaving one to speculate as to how, how long, in what quantities, in what frequency, or in what setting an exposure must occur to cause Parkinson's disease. (Doc. 4355-2 at 7). At his first deposition, Dr. Wells testified that occupational exposure meant that it was “primarily focused on . . . people's work and . . . exposure because of their workplace.” (Doc. 4364-20 at 6). Then in his rebuttal report, Dr. Wells changed course, stating that “occupational” exposure involved “the use of paraquat, or contact with paraquat where there is the risk of dermal exposure.” (Doc. 4355-3 at 12). And finally, at his second deposition, Dr. Wells testified that “occupational” exposure meant “direct contact” with a person, “not occupational, not drift or anything like that.” (Doc. 4561-10 at 37) (emphasis added).⁴²

42 Dr. Wells' rebuttal report and his second deposition also appear to invoke a “direct use” paradigm to define “occupational” exposure, consistent with EPA's definition of occupational exposures. (Docs. 4355-3 at 12; 4561-10 at 37). The EPA had found, in its systematic review, that “occupational study populations [are] more likely to experience exposure as a result of direct use of paraquat” (Doc. 4558-12 at 90). But if that was the focus of Dr. Wells' analyses and opinions, then it is still unclear why his rebuttal report requires either “use . . . or contact” disjunctively to generate an “occupational” exposure. (Doc. 43553 at 12) (emphasis added).

Dr. Wells' definition of “occupational” exposure evolved from being related to a person's workplace, to focusing on the “risk of dermal exposure,” to “direct contact.” Even after his first deposition, where this issue was examined in some depth, Dr. Wells was unable to settle on a clear definition of “occupational” exposure. Although his rebuttal report abandoned the focus on workplace exposures, it stated that paraquat exposure that involves the “risk of dermal exposure” is capable of causing Parkinson's disease. Dr. Wells does not define or quantify the “risk” of dermal exposure that is necessary before it can cause Parkinson's disease. Moreover, focusing on the “risk of dermal exposure” as the critical exposure metric would require one to account for the use of personal protective equipment (“PPE”) because, as Dr. Wells acknowledged, “[i]f you have PPE, then it's probably going to-that will shield you.” (Doc. 4561-10 at 38). The rebuttal report is conspicuously silent on this issue.

More fundamentally, however, the “risk” of dermal exposure is not the same as actual dermal exposure. Perhaps for this reason, Dr. Wells' reliance on a “risk” paradigm proved to be ephemeral. At his second deposition, Dr. Wells seemingly abandoned his focus on the “risk of dermal exposure” in favor of “direct contact.” Direct contact, according to Dr. Wells, simply meant “you're using it,” and “it's on you,” without any mention of “risk.” Indeed, when pressed about the role of dermal exposures, Dr. Wells offered the following testimony:

QUESTION: So[,] it doesn't actually matter if [paraquat] gets on your skin?

DR. WELLS: The endpoints are - in the studies are contact, direct use or use. And so[,] they may control for PPE, but, you know, I'm looking at the endpoints of the study that say use or this direct contact.

Id. Then, in a final definitional twist during his second deposition, Dr. Wells added a temporal limitation to his opinion, which stipulated that only exposure durations of 25 lifetime days or more increase one's risk of developing Parkinson's disease.⁴³ Id. at 39-41. This nebulous definition of the type of exposure that, according to him, is causally related to Parkinson's disease leaves it to the court-and if he were to testify, the jury-to figure out the precise contours of his opinion. See United States v. Frazier, 387 F.3d 1244, 1265 (11th Cir. 2004) (en banc) (excluding “unclear, imprecise and ill-defined” expert opinion because it was impossible to verify its reliability); cf. Reed v. Binder, 165 F.R.D. 424, 430 (D.N.J. 1996) (“Nothing causes greater prejudice than to have to guess how and why an adversarial expert reached his or her conclusion.”).

43 The 25-day threshold was first raised by Plaintiffs' counsel during Dr. Wells' first deposition:

QUESTION: So[,] if you['re] an eligible participant in any of your seven studies and you used paraquat for one day, the 2.8 applies to you in the same way that it would apply to someone who used paraquat for 20 years?

DR. WELLS: They're - that's the - that's the rule that's included in the study. QUESTION: So yes.

DR. WELLS: Yes.

PLAINTIFFS' COUNSEL: Just let me clarify. It's not a one-day. It's 25 days

DR. WELLS: It's 25 days from Tanner.

...

QUESTION: Do all the seven studies in your meta-analysis have a 25-day minimum? Dr. Wells: No.

QUESTION: So - [] what you said before is correct, that even if you are exposed for one day, if you use paraquat for one day and you're an eligible participant in the study, the 2.8[] elevated odds ratio would apply to that person.

DR. WELLS: They're - they're the participants in the study, that's all I can say.

(Doc. 4364-20 at 18) (emphasis added). Neither Dr. Wells' first, nor his rebuttal report mention the 25-day exposure threshold. The fact that it appeared, for the first time, through Plaintiffs' counsel's interposition does not support its reliability under Rule 702. See Southwire Co. v. J.P. Morgan Chase & Co., 528 F.Supp.2d 908, 933 (W.D. Wis. 2007) (expert must “reach[] his own conclusions” and not simply “parrot the arguments of counsel”).

The Eleventh Circuit's decision in United States v. Frazier offers helpful guidance on this point. 387 F.3d at 1265. There, a criminal defendant in a rape and kidnapping case offered an expert opinion from a forensic investigator that a transfer of hair and bodily fluids “would be expected” in the case. Id. Because such forensic evidence was not recovered, the defendant intended to rely on the expert's opinion to undermine the credibility of the accusation against him. Id. at 1252. The Eleventh Circuit, sitting en banc, affirmed the district court's exclusion of the expert's proffered opinion that a transfer of hair and bodily fluids “would be expected” because “the very meaning of [this] basic opinion is uncertain.” Id. at 1265. The expert failed to explain whether his “expectation” opinion reflected a state of affairs that was more likely than not, substantially more likely than not, or a virtual certainty. Id. The meaning of the opinion, according to the Eleventh Circuit, was “impossible to discern” and thus inadmissible under Rule 702. Id. at 1265-66.

The same is true of Dr. Wells' meandering definition of “occupational” exposure to paraquat. It is unclear from the record whether Dr. Wells' general causation opinion requires workplace exposure to paraquat, the “risk of dermal contact,” “direct exposure,” “direct use,” or “direct contact.” The meaning of his opinion is therefore “impossible to discern,” as it was in Frazier. Id. at 1265. This is the first strike against the reliability of Dr. Wells' expert opinions under Rule 702. See Clark v. River Metals Recycling, LLC, 929 F.3d 434, 438 (7th Cir. 2019) (“unclear” methodology warrants exclusion of expert testimony).

2. Dr. Wells' Meta-Analysis

a. The Rules and Requirements of a Reliable Meta-Analysis

The scientific community has developed certain guidelines that govern the use of meta-analysis as a scientific technique to promote transparency and reliability. The Reference Manual on Scientific Evidence, the Cochrane Handbook for Systematic Reviews of Interventions (hereinafter “Cochrane Handbook”), and a plethora of other authoritative sources on the rules of meta-analysis make it clear that objective and scientifically valid study selection criteria should be clearly stated in advance to ensure the objectivity of the analysis. The reason for this requirement is simple-the selection of studies to include in a systematic review and meta-analysis can be dispositive of its result. In re Zimmer Nexgen Knee Implant Prod. Liab. Litig., No. 11 C 5468, 2015 WL 5050214, at *10 (N.D. Ill. Aug. 25, 2015). Thus, if relevant studies are excluded from the analysis, those decisions must be based on objective criteria that can withstand scientific scrutiny.

One of the leading treatises on meta-analysis, indeed one that both sides cited in their briefing, explains why a clear search methodology and inclusion/exclusion criteria are essential to a reliable meta-analysis:

For systematic reviews, a clear set of rules is used to search for studies, and then to determine which studies will be included in or excluded from the analysis. Since there is an element of subjectivity in setting these criteria, as

well as in the conclusions drawn from the meta-analysis, we cannot say that the systematic review is entirely objective. However, because all of the decisions are specified clearly, the mechanisms are transparent.

Michael Borenstein et al., Introduction to Meta-Analysis xxiii (2009). The Cochrane Handbook (to which Dr. Wells professes his adherence) similarly stresses the importance of establishing and documenting the methodology of a systematic review “in advance” to guard against selection bias. Cochrane Handbook for Systematic Reviews of Interventions Pt. 2, Ch.1, Sec. 1-5 (Julian Higgins & James Thomas eds., 2022). This basic requirement is even more important when a researcher's objectivity might be clouded by “prior knowledge” of the evidence or the desire to achieve a certain result, as is the case for many litigation experts. Id. Presumably with these foundational principles in mind, the EPA defined its systematic review of the evidence of a potential relationship between paraquat exposure and Parkinson's disease, as “a scientific investigation that focuses on a specific question and uses explicit, pre-specified scientific methods to identify, select, assess, and summarize findings of similar but separate studies.” (Doc. 4558-12 at 9) (emphasis added). Thus, considering the compelling effect that a meta-analysis can have on its audience, courts must carefully examine its methodological rigor to ensure that it reflects a reliable view of the science, as opposed to an outcome-driven presentation concocted to impress a jury. See Deutsch v. Novartis Pharmaceuticals Corp., 768 F.Supp.2d 420, 457-58 (E.D.N.Y. 2011) (“[T]here is a strong risk of prejudice if a Court permits testimony based on an unreliable meta-analysis because of the propensity for juries to latch on to the single number.”).

b. Search for Relevant Epidemiological Studies

Dr. Wells' violations of the rules of meta-analysis are evident from the very beginning of his process. One of the initial steps in a meta-analysis involves the search for relevant studies that are then further analyzed for potential inclusion in the analysis. Dr. Wells' first report is entirely devoid of a search narrative that would allow other researchers to validate his process. In his first report, Dr. Wells provided a list of 11 casecontrol studies that were supposedly “included” in his analysis. But that list did not include Shrestha and other studies that Dr. Wells admitted were “important” to his research question. (Doc. 4561-10 at 28). At most, his first report explains that he reviewed the five most recent systematic reviews on the paraquat-Parkinson's disease relationship to identify relevant studies. It was not until Dr. Wells submitted his rebuttal report that he identified 36 studies that he claims he considered for inclusion in his meta-analysis. (Doc. 4355-3 at 45-46). And even after he identified these 36 studies, he failed to explain how he identified them in his research. Dr. Wells admitted that he “can't remember the details” about how he searched for and identified the 36 studies that made up the initial universe of studies that were then systematically reviewed. (Doc. 4561-10 at 27). As a result, any attempt to replicate his search for relevant literature would require a degree of clairvoyance that this Court does not possess.

To illustrate this point, the rebuttal report cites two studies by Tomenson et al., one from 2011 and one from 2021, among the 36 studies that Dr. Wells initially considered. (Doc. 4355-3 at 46). In the first report, however, Tomenson et al. (2011) is mentioned only once, without comment or narrative, as a study that was part of a meta- analysis by Breckenridge et al. (2016). (Doc. 4355-2 at 8). Tomenson et al. (2021), on the other hand, is only mentioned in Dr. Wells' “Materials Considered” addendum to his first report, where it appears among 110 other citations, also without comment or narrative. Id. at 35.

The search for relevant literature should be clearly documented and be “as comprehensive as possible,” to ensure that the relevant studies are identified and to allow other researchers to validate the process. Bero, supra, at 577. Dr. Wells' failure to document his search for relevant studies makes it impossible to replicate or even critique. See United States v. Hebshie, 754 F.Supp.2d 89, 125 (D. Mass. 2010) (“Documentation is necessary to test a hypothesis; in fact, reproducibility is the sine qua non of “science.”). And while this omission is not independently fatal to the reliability of Dr. Wells' metaanalysis, it is evidence of a more systemic failure to adhere to the rules of his chosen methodology.

c. Eligibility Criteria for Dr. Wells' Meta-Analysis

The next methodological red flag in Dr. Wells' meta-analysis is that until he submitted his rebuttal report, he failed to clearly articulate the inclusion/exclusion criteria that purportedly governed a study's eligibility for his analysis. Indeed, Dr. Wells testified at his first deposition that he reviewed the relevant studies “holistically” to determine “whether or not [they were] reliable enough for inclusion.” (Doc. 4364-20 at 29). This “holistic” approach was neither reduced to writing, nor did it offer any discernible objective criteria that would allow others to replicate Dr. Wells' eligibility determinations.

Then came the rebuttal report. The rebuttal report stated that a study had to meet the following five criteria to be eligible for Dr. Wells' meta-analysis: (i) studies of paraquat exposure; (ii) studies of Parkinson's disease; (iii) studies of occupational exposure; (iv) studies with a case-control design; and (v) studies with sufficient data to determine an odds ratio. (Doc. 4355-3 at 12-16). These criteria were far from clearly articulated in Dr. Wells' first report where he addressed some of them in an ad hoc manner as he discussed certain relevant studies. This omission critically undermines the reliability of his meta-analysis because “[p]redefined, unambiguous eligibility criteria are a fundamental prerequisite for a systematic review.” Cochrane Handbook, supra, Pt. 2, Ch. 3, Sec. 3-2. Indeed, Dr. Wells' failure to define his eligibility criteria in advance suggests that he selected the studies he wanted to include in his meta-analysis and then crafted his inclusion/exclusion criteria to justify his decisions. This type of post hoc methodology is the very antithesis of a systematic review, which relies on predefined eligibility criteria to ensure transparency and scientific objectivity. Cf. In re Lipitor (Atorvastatin Calcium) Mktg., Sales Pracs. & Prod. Liab. Litig., 892 F.3d 624, 634 (4th Cir. 2018) (“Result-driven analysis, or cherry-picking, undermines principles of the scientific method and is a quintessential example of applying methodologies (valid or otherwise) in an unreliable fashion.”).

Nowhere is Dr. Wells' failure to predefine his eligibility criteria more problematic than in his attempt to square the inclusion of Liou (1997) in his meta-analysis with his requirement that studies address “occupational” exposure. The Court previously addressed how Dr. Wells' dynamic definition of “occupational” exposures obfuscates the scope and meaning of his ultimate opinion on general causation. But his failure to clearly define this eligibility criterion also undermined the methodological soundness of his meta-analysis because he was forced to concede that the study that almost singlehandedly generated his elevated odds ratio of 2.8, “did not meet [his] own stated criteria for occupational exposure” because it “included residential exposure as well.” (Doc. 4364-20 at 31-32). Of course, Dr. Wells later revised his definition of “occupational” exposure to cover exposures involving “potential dermal contact” and “direct contact,” which allowed him to include Liou in his meta-analysis. Logic dictates, however, that without Liou, Dr. Wells' elevated odds ratio of 2.8 would look very different because its 74% of weight would likely be redistributed among the six remaining studies, five of which revealed a statistically insignificant association between paraquat exposure and Parkinson's disease.⁴⁴ So, while this change may have helped Dr. Wells conclude that occupational paraquat exposure was associated with a “near tripling of PD occurrence,” it violated the basic rules of meta-analysis, which mandate that researchers “develop a protocol for the review before commencement and adhere to the protocol regardless of the results of the review.” Bero, supra, at 575.

44 Dr. Wells' first report states that he applied a fixed-effect weighting model to generate his odds ratio of 2.88. (Doc. 4355-2 at 19). His rebuttal report then corrected an error in his meta-analysis and applied a random effects model to identify an odds ratio of 2.84. (Doc. 4355-3 at 11, 43). Notably, Dr. Wells' utilization of a random effects model did not result in a “more balanced” allocation of weights, as Liou (1997) remained the primary driver of the resulting odds ratio with 73.73% of the weight. (Doc. 4355-3 at 42). This drew the Court's attention to Dr. Wells' weighting methodology. The Court explored this issue at the Daubert hearing, where Dr. Wells testified that a “statistical procedure” weighted studies based on their confidence intervals. (Doc. 4793 at 68). A study with a narrower confidence interval received more weight than a study with a wider confidence interval because narrower confidence intervals reflect more precise information. Although this approach seems to be well within the scope of expert judgment, especially for a biostatistician as accomplished as Dr. Wells, this explanation was insufficient. If confidence intervals exclusively determined weight assignments, Firestone (2010) would have received more weight than Hertzman (1994) because Firestone's confidence interval ranged from .14 to 5.43, whereas Hertzman's ranged from .33 to 7.04. Yet, Dr. Wells assigned only 1.86% of the weight of the meta-analysis to Firestone and 2.66% to Hertzman. (Doc. 4355-3 at 42). Although Dr. Wells, and not this Court, possesses the statistical expertise to properly weight studies in a meta-analysis, there was an opacity to the way in which weight was assigned that only enhances this Court's concerns about the reliability of the analysis, particularly when one considers the dominant effect of one study (Liou (1997)) on the overall risk estimate. A meta-analysis ordinarily applies either a fixed-effect or a random effects model to “weight” the studies within it. Borenstein et al., supra, at 61. Weighting is a critical step in a meta-analysis because more heavily weighted studies have a correspondingly larger effect on the ultimate result. Under a fixed-effect model, the researcher assumes that all studies within the meta-analysis represent an identical true effect size, meaning that any variation in the effect size between studies is attributable to random error. Id. at 78. The fixed-effect model assigns weight to studies based on their respective population sizes with larger studies receiving more weight than smaller ones. Id. at 78-79. A random effects model, on the other hand, assumes that the true effect size varies from one study to the next so that “the studies in [the meta-analysis] represent a random sample of effect sizes.” Id. at 77. Under a random effects model, “the goal is not to estimate one true effect but to estimate the mean of a distribution of effects.” Id. at 79. Critically, a researcher applying a random effects model “do[es] not want th[e] overall estimate to be overly influenced by any one [study].” Id. Study weights are consequently “more balanced” in a random effects model than in a fixed-effect model. Id.

Dr. Wells also testified that if another researcher wanted to replicate his eligibility decisions, he “think[s] they would come to the same place. If we-if we talked about what are good-quality studies and what are the issues that-where-what studies don't give as reliable evidence, they would probably come to the same place.” (Doc. 4364-20 at 30). But Defendants' epidemiology expert, Dr. Alexander, stated in his report that “it was not possible for [him] to follow a clear set of inclusion/exclusion criteria to assess whether Dr. Wells followed his own purported rules.” (Doc. 4355-6 at 87) (emphasis added). Which of these two experts is correct is ultimately beside the point; what matters is that Dr. Wells first report and deposition offer nothing beyond an unwritten and “holistic” study selection methodology to support his meta-analysis. Thus, his belief that other researchers would “probably come to the same place” is nothing but a subjective assumption and “a good illustration of why mere expertise and subjective understanding are not reliable scientific evidence.” Brown v. Burlington N. Santa Fe Ry. Co., 765 F.3d 765, 776 (7th Cir. 2014).

The Seventh Circuit recognizes that expert opinions that cannot be objectively replicated are subject to exclusion. See Timm v. Goodyear Dunlop Tires N. Am., Ltd., 932 F.3d 986, 994 (7th Cir. 2019) (exclusion warranted where expert “knew of no way others could objectively replicate his approach.”); Zenith, 395 F.3d at 419 (“Someone else using the same data and methods must be able to replicate the result.”). Here, the lack of replicability is not just a problem in the abstract. It is a foundational deficiency in the application of Dr. Wells' chosen methodology. Clearly defined, objective eligibility criteria are precisely what lend scientific authority to a meta-analysis. A researcher who offers these criteria consistent with the standards outlined above invites her peers to evaluate and critique them; she allows other researchers to validate her process, identify gaps in her analysis and offer potential refinements that may advance the scientific project. See Borenstein et al., supra, at xxiii (meta-analysis “provides a transparent, objective, and replicable framework” to advance scientific inquiry). Dr. Wells, on the other hand, admits that “there's no algorithm to replicate” with respect to his inclusion/exclusion criteria. (Doc. 4364-20 at 66). This admission undermines the reliability of his meta-analysis because it suggests that he failed to “employ[] in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Kumho Tire, 526 U.S. at 152.

Finally, Dr. Wells' reliance on an unwritten, “holistic” methodology presents an ideal example of “because I said so” expertise that is impermissible under Rule 702. See FED. R. EVID. 702 advisory committee's note to 2000 amendments (the gatekeeping function “requires more than simply ‘taking the expert's word for it.'”). Dr. Wells insisted that he “ha[s] the credentials to do this” and that he “had a process that [he] followed.” (Doc. 4364-20 at 30). But these assurances, without more, do not show that Dr. Wells faithfully applied the necessary steps of his chosen methodology as Daubert requires. Zenith, 395 F.3d at 418. If anything, it shows the opposite. With an unwritten, “holistic” approach, Dr. Wells was free to select the studies he wanted to meta-analyze, and then justify his selections based on the results they provided. This, in turn, undermines the reliability of his meta-analysis because it allowed him to generate a pre-determined result. Dr. Wells himself does not appear to dispute this reality. In 2018, he was deposed in connection with the Testosterone Replacement Therapy Litigation in the Northern District of Illinois and offered the following testimony regarding the methodological rigor of certain meta-analyses that were relevant to the case:

QUESTION: You would certainly agree that the inclusion-exclusion criteria should be based upon objective criteria and not simply because you were trying to get to a particular result?

DR. WELLS: No, you shouldn't load the-sort of cook the books.

QUESTION: You should have prespecified objective criteria in advance, correct?

DR. WELLS: Yes.

In re Testosterone Replacement Therapy Prod. Liab. Litig., Nos. 1:14-cv-1748, 15-cv-4292, 15-cv-426, 2018 WL 7350886 (N.D. Ill. Apr. 2, 2018). It should go without saying that this Court is similarly concerned about submitting expert opinions to a jury where the expert's methodology may have allowed him to “cook the books.”

d. Inconsistent and Post Hoc Application of Quality Factors

Dr. Wells' utilization of five quality factors to determine which eligible studies to include in his meta-analysis suffers from a similar lack of systematicity. Dr. Wells' rebuttal report explained that out of the 36 studies he initially considered for his metaanalysis, eight case-control studies were eligible for inclusion: Liou (1997), Tanner (2011), Hertzman (1994), Kuopio (1999), Firestone (2010), Rugbjerg (2011), Dhillon (2008), and van der Mark (2014). (Doc. 4355-3 at 12, 45-46, 48). These studies were qualitatively judged based on their (i) diagnostic criteria for the identification of cases; (ii) exposure assessment methodologies; (iii) composition of control groups; (iv) participation rates; and (v) control of confounding variables. Id. at 17-22.

Only one of the eight eligible studies failed Dr. Wells' qualitative evaluation and was excluded from the meta-analysis: van der Mark (2014). Van der Mark, of course, is notable for its statistically insignificant odds ratio of 1.27 and conclusion that there was “no evidence for an association” between paraquat exposure and Parkinson's disease. (Doc. 4355-20 at 7). Dr. Wells classified van der Mark as “lower quality” on each of his quality metrics. (Doc. 4355-3 at 48). This qualitative indictment of van der Mark does not withstand scrutiny when viewed in the context of Dr. Wells' broader qualitative evaluation of the eight eligible studies.

Dr. Wells relied on an evolving set of quality criteria to determine which studies ultimately warranted inclusion in his meta-analysis. This alone undermines the methodological soundness of his qualitative evaluation of the literature. But that is not the only problem with this step in Dr. Wells' meta-analysis. In addition to modifying certain quality criteria, Dr. Wells inconsistently applied them in a thinly veiled attempt to ensure the inclusion of the studies that made it into his meta-analysis and to justify the exclusion of van der Mark.

The first example of a methodological flip-flop is evident in the quality factor concerning diagnostic criteria. Dr. Wells' first report states that a study had to conduct “in-person evaluations, including a standardized medical and neurological history and examination by movement disorder specialists” to ensure diagnostic accuracy in cases and controls. (Doc. 4355-2 at 18). Van der Mark did not meet this requirement because it relied on a medical records review to identify cases and controls. (Doc. 4355-20 at 3). Dr. Wells therefore justified his exclusion of van der Mark in part on the basis that “[c]ase identification did not require a PD diagnostic exam given by a movement disorder specialist, in contrast to the other case-control studies in the meta-analysis that required such a PD diagnostic exam by a movement disorder specialist.” (Doc. 4355-2 at 16). In his rebuttal report, Dr. Wells amended this criterion by grading studies that relied on an “inperson exam or medical records review with acceptable diagnostic criteria” as “higher quality.” (Doc. 4355-3 at 48) (emphasis added). This modification allowed Dr. Wells to judge Firestone (2010) as “higher quality” because it relied on a medical records review and not an in-person exam to identify cases and controls. (Doc. 4355-25 at 3). The key distinction between Firestone and van der Mark, according to Dr. Wells, was the fact that van der Mark did not provide “the diagnostic criteria that the medical records were reviewed on.” (Doc. 4561-10 at 58). Although this redefinition of a critical quality metric raises concerns about the reliability of Dr. Wells' assessment of the literature, it was not the worst violation of the scientific method at this step of his analysis.

Dr. Wells' definition and evaluation of participation rates in the eligible studies offers an even more blatant example of methodological shapeshifting. Dr. Wells has consistently criticized van der Mark for its low participation rate. This criticism is not unfounded, as the authors of van der Mark themselves acknowledged “the relatively low participation rate” in their study. (Doc. 4355-20 at 8). But Dr. Wells' qualitative critique of van der Mark's participation rate was nothing more than a one-sentence observation in his first report and was entirely uncoupled from any objective threshold that would have allowed one to systematically judge the study based on this quality criterion. (Doc. 4355-2 at 17). At his first deposition, Dr. Wells confirmed as much. He testified that he did not “have a number” when asked if there was a “clear requirement regarding participation rates for inclusion in [his] analysis.” (Doc. 4364-20 at 65). His rebuttal report then flipped the script. There, studies that achieved a participation rate above 60% were considered “higher quality,” whereas studies that fell below this threshold were “lower quality.” (Doc. 4355-3 at 48). Because van der Mark does not meet this threshold, Dr. Wells' “lower quality” rating for its participation rate, at first blush, appears reasonable.

But not all studies that ultimately made it into Dr. Wells' meta-analysis reported their participation rates. Tanner (2011), the second most-heavily weighted study in Dr. Wells' meta-analysis, simply stated that participation was “good,” and Liou (1997) reported no participation data at all (Docs. 4355-16 at 7; 4355-22). Dr. Wells nevertheless assigned both studies a “higher quality” rating for their participation rates, notwithstanding the vagueness of the information reported in Tanner and the complete absence of relevant information in Liou. (Doc. 4355-3 at 48).

Dr. Wells' explanation for this analytical inconsistency was even more puzzling. He testified at his second deposition that he assumed Liou had “a hundred percent participation rate” because the number of people that were enlisted in the study matched the number of people that completed it. (Doc. 4561-10 at 55). By that logic, however, van der Mark would have had a participation rate of a hundred percent as well because it reported the enlistment of 444 cases and 876 controls, all of whom are accounted for in the data table that reports paraquat exposure and Parkinson's disease status. (Doc. 4355-20 at 3, 7). But van der Mark did not define its participation rate based on the number of enlisted participants compared to the number of participants whose results were reported in the study. Rather, van der Mark reported that only 45% of cases and 35% of controls agreed to participate in the study. Id. at 3. Moreover, participants who declined to participate or did not respond to the researchers' outreach efforts were deemed not to have participated. Id. Dr. Wells ran with this information, not the rate of enlisted participants compared to participants whose results were reported.

It is obvious, upon closer inspection, that Dr. Wells applied one definition of a study's participation rate to Liou and another definition to van der Mark. Dr. Wells' uncritical assumption that Liou had “a hundred percent participation rate” is particularly troubling because the study offered no information about its participation rate. This should have at least given Dr. Wells pause before assuming that it had perfect participation. This is especially true where epidemiologists have observed a “reluctance” by study authors to report participation rates because of “the epidemiologic tendency to chide low participation rates as a sign of study inferiority.” Sandro Galea & Melissa Tracy, Participation Rates in Epidemiologic Studies, 17 Annals of Epidemiology 643, 644 (2007). While this Court is in no position to impute such “reluctance” to the authors of Liou, the lack of available information about its participation rate can hardly be used as evidence of perfect participation.

This Court is ultimately concerned with the methodological soundness of Dr. Wells' analyses, not the correctness of the data he analyzed or the conclusions he drew. Manpower, Inc. v. Ins. Co. of Pa., 732 F.3d 796, 806 (7th Cir. 2013). So, while it is emphatically not for the Court to lecture Dr. Wells on the nuances of epidemiological study participation rates, his inconsistent evaluation of the participation rates in Liou and van der Mark compromises the reliability of his analysis. Bearing in mind its critical gatekeeping duty, the Court is troubled by the blatancy with which Dr. Wells graded favorable studies as “higher quality” using one evaluation method and his concomitant imposition of a more stringent standard on an unfavorable study to grade it as “lower quality.” See In re Zoloft (Sertraline Hydrochloride) Prod. Liab. Litig., 858 F.3d 787, 795-97 (3d Cir. 2017) (“[I]f an expert applies certain techniques to a subset of the body of evidence and other techniques to another subset without explanation, this raises an inference of unreliable application of methodology.”); In re Bextra and Celebrex Mktg. Sales Pracs. & Prod. Liab. Litig., 524 F.Supp.2d 1166, 1179 (N.D. Cal. 2007) (excluding proffered expert's causation testimony, in part, due to inconsistent evaluation of favorable and unfavorable data).

e. Omission of Adjusted Data

Another methodological issue in Dr. Wells' meta-analysis concerns his apparent failure to follow even his own articulated reliability standards. In his first report, Dr. Wells cited the Cochrane Handbook for the following proposition:

When extracting data for a meta-analysis from non-randomized studies, adjusted effect estimates may be available (e.g., adjusted odds ratios from logistic regression analyses).... [A]djusted association estimates are generally preferable to analyses because they usually reduce the impact of confounding. If both unadjusted and adjusted intervention effects are reported in a study, the adjusted results are preferred. It is straightforward to extract the reported adjusted effect estimate and its standard error for a meta-analysis if a single adjusted estimate for a particular outcome in a primary non-randomized study [is available]. If multiple adjusted estimates of the associations are reported, the one that is judged to minimize the risk of bias due to confounding should be chosen.

(Doc. 4355-2 at 5-6). Dr. Wells appears to have violated this guideline for the most important study in his meta-analysis, Liou (1997).

Liou presented its findings of a paraquat-Parkinson's disease association in two forms: (i) a dichotomous variable analysis; and (ii) a conditional logistic regression analysis that controlled for critical confounders, including duration of rural residence, duration of farming, and exposures to other herbicides and pesticides.⁴⁵(Doc. 4355-22 at 5-6). The dichotomous variable analysis resulted in an odds ratio of 3.22, with a confidence interval of 2.41 - 4.31. Id. at 5. The logistic regression analysis in Liou resulted in an adjusted odds ratio of 6.44, with a significantly wider confidence interval of 2.41 -17.2, for participants who reported 20 years or more of paraquat use. Id. at 6. Participants with 1 to 19 years of paraquat use generated a substantially lower odds ratio of .96, with a confidence interval of .24 - 3.83, in the regression analysis. Id.

45 The conditional logistic regression analysis also examined a potential dose-response relationship by testing whether longer exposure periods resulted in an increased risk of developing Parkinson's disease. The investigators separated participants who reported paraquat exposure periods of 1-19 years from those who reported exposures of 20 years or more. (Doc. 4355-22 at 6).

Dr. Wells selected Liou's odds ratio from its dichotomous variable analysis for his meta-analysis, not the adjusted data provided in the logistic regression analysis that controlled for critical confounding variables. (Docs. 4355-2 at 19; 4355-3 at 43). More importantly, he did so without explaining his reasoning for this decision, which would appear to violate the rule that “adjusted odds ratios from logistic regression analyses . . . are preferred.” (Doc. 4355-2 at 5). When an expert presents a methodological guideline for her analysis and then proceeds to violate it, a court is well within the bounds of its gatekeeping role to be skeptical of the expert's opinion. Brown, 765 F.3d at 776. Because Liou was singularly important to Dr. Wells' odds ratio of 2.8, his reliance on minimally adjusted data, when more fully adjusted data was available, gives the Court pause.

In a similar case involving the herbicide glyphosate (marketed as “Roundup”), the Northern District of California observed that “exclusive consideration of numbers unadjusted for other pesticides, when adjusted numbers are available, would be disqualifying” under Rule 702. In re Roundup Prod. Liab. Litig., 390 F.Supp.3d 1102, 1140 (N.D. Cal. 2018). Moreover, “[f]ailing to take account of likely confounders by presenting and relying upon only unadjusted (or minimally adjusted) estimates is a serious methodological concern” that “calls [an expert's] objectivity and credibility into question.” Id. Dr. Wells appears to have done exactly that. The Court is consequently left with the impression that Dr. Wells selected Liou's minimally adjusted odds ratio over its multivariate adjusted odds ratios for his meta-analysis to avoid the wide statistical variability of the adjusted data.⁴⁶ Cf. People Who Care v. Rockford Bd. of Educ., Sch. Dist. No. 205, 111 F.3d 528, 537-38 (7th Cir. 1997) (“[A] statistical study that fails to correct for salient explanatory variables . . . has no value as causal explanation and is therefore inadmissible in federal court.”). This type of selection bias is plainly impermissible under Rule 702. See Allgood v. Gen. Motors Corp., No. 102CV1077DFHTAB, 2006 WL 2669337, at *11 (S.D. Ind. Sept. 18, 2006) (Hamilton, J.) (selection bias undermines reliability under Rule 702).

46 At the Daubert hearing, Dr. Wells offered the following explanation of the importance of confidence intervals to his weighting methodology: “if you . . . put in the confidence interval, it's very wide, those studies won't get as much weight. But if you put a study in the, you know -- put a study in and the comments are very tight, usually those get a lot more -- will get more weight.” (Doc. 4793 at 68). One potential inference that flows from this testimony is that Dr. Wells eschewed the adjusted data from Liou's regression analysis because its confidence intervals were significantly wider than the confidence interval reported in the dichotomous variable analysis. Of course, Dr. Wells may have had a valid reason for relying on data from Liou's dichotomous variable analysis, but if he did, that reason was not articulated.

f. Plaintiffs' Arguments in Defense of Dr. Wells' Meta-Analysis

Plaintiffs valiantly defend Dr. Wells' meta-analysis, arguing that his selection of studies was based on his informed scientific judgment, which should not be substantively second-guessed at the Daubert stage.⁴⁷ (Doc. 4561 at 35-56). To that end, Plaintiffs devote a substantial portion of their briefing to Dr. Wells' decision to include Tanner (2011) in his meta-analysis instead of Shrestha (2020), a decision that Defendants vigorously contest. Recall that Tanner and Shrestha are both part of the AHS and therefore studied overlapping populations. Tanner reported a significantly higher odds ratio of 2.5 than Shrestha's hazard ratio of 1.09. Moreover, Shrestha noted the “[l]imited reproducibility” between its findings and Tanner's. (Doc. 4558-9 at 8). It is not surprising that the Parties have diametrically opposing views about which of these two studies reports more reliable data. And Plaintiffs, for their part, correctly point out that “Rule 702 d[oes] not require, or even permit, the district court to choose between those two studies at the gatekeeping stage.” Schultz v. Akzo Nobel Paints, LLC, 721 F.3d 426, 433 (7th Cir. 2013). But this argument misses the point.

47 Plaintiffs also repeatedly assert that Dr. Wells' methodology and opinions are reliable because Dr. Alexander followed a similar process in forming his opinions. This argument is unpersuasive. A flawed methodology does not somehow become Daubert-compliant by mirroring an opposing expert's process in the same litigation. See Colony Ins. Co. v. Coca Cola Co., 239 F.R.D. 666, 675 (N.D.Ga. 2007) (“[I]t is no defense against a Daubert motion to argue that your expert uses the same methodology as the opposing party's expert.”). Daubert requires each expert's proffered opinion to be judged on its own merits. The Court would be disregarding its critical gatekeeping responsibilities if it accepted the premise that the similarity of Dr. Wells' and Dr. Alexander's methodological approaches, alone, establishes the reliability of Dr. Wells'

The Court does not question the scientific merit of Dr. Wells' decision to limit his meta-analysis to case-control studies, which resulted in Shrestha's exclusion as a prospective cohort study. Nor does the Court fault Dr. Wells for determining that van der Mark was ultimately of “lower quality” and thus not suited for his meta-analysis. The Court's conclusion that Dr. Wells' meta-analysis is not sufficiently reliable under Rule 702 is based on Dr. Wells' failure to reliably apply his chosen methodology. Daubert instructs courts to consider “the existence and maintenance of standards controlling the technique's operation” as a factor that bears on the reliability of an expert's proffered opinions. Accordingly, the reliability of Dr. Wells' methodology and opinions must be assessed based on his work and not that of another expert. See Brown v. Burlington N. Santa Fe Ry. Co., 765 F.3d 765, 775 (7th Cir. 2014) (“[P]ointing out deficiencies in the defendant's expert testimony cannot help [plaintiff], who bears the burden of . . . demonstrating the reliability of his own expert.”). opinion. Daubert, 509 U.S. at 594. The rules and requirements of meta-analysis are a prime example of such standards because they are designed to enhance the reliability of the analysis by avoiding results-driven data aggregations. Dr. Wells' failure to clearly predefine his eligibility criteria, his subsequent redefinition of eligibility criteria, his varying definitions of quality criteria, and his inconsistent application of quality criteria, which conveniently imposed a more onerous standard on a less favorable study, are just some examples of his violations of these standards.

Nothing in this order should be construed as the Court's independent, substantive evaluation of the epidemiological literature. In Manpower, the Seventh Circuit warned that a district court “usurps the role of the jury, and therefore abuses its discretion, if it unduly scrutinizes the quality of the expert's data and conclusions rather than the reliability of the methodology the expert employed.” 732 F.3d at 806. In practice, this means that “the selection of data inputs to employ in a model is a question separate from the reliability of the methodology reflected in the model itself.”⁴⁸ Id. at 807. This order hopefully makes clear that the Court takes no position on the relative merit of the various epidemiological studies at issue in Dr. Wells' analysis. Indeed, the Court does not find Dr. Wells' meta-analysis unreliable because it excluded van der Mark (2014), Shrestha (2020), or any other relevant study for that matter. Rather, Dr. Wells' meta-analysis does not pass muster under Rule 702 because its methodology was unclear, inconsistently applied, not replicable, and at times transparently reverse-engineered. See In re Mirena IUS Levonorgestrel-Related Prod. Liab. Litig. (No. II), 341 F.Supp.3d 213, 241 (S.D.N.Y. 2018) (“Opinions that assume a conclusion and reverse-engineer a theory to fit that conclusion are . . . inadmissible.”) (internal quotation marks and citation omitted); In re Zoloft (Sertraline Hydrochloride) Prod. Liab. Litig., No. 12-md-2342, 2015 WL 7776911, at *16 (E.D. Pa. Dec. 2, 2015) (excluding expert's opinion where he “failed to consistently apply the scientific methods he articulat[ed], . . . deviated from or downplayed certain well-established principles of his field, and . . . inconsistently applied methods and standards to the data so as to support his a priori opinion.”).

48 Because Dr. Wells' selection of studies for his meta-analysis was a key issue in the court's assessment of the reliability of his opinions, the Court requested additional briefing, asking the Parties to address how Manpower “applies to this Court's assessment of the reliability of Dr. Martin Wells' proffered testimony.” (Doc. 4756). Both sides submitted able briefing in response to this request.

In conclusion, there is no doubt that systematic review and meta-analysis are well-accepted methodological tools of the scientific community. But Rule 702 requires more than the label of a reliable methodology. Robinson v. Davol, Inc., 913 F.3d 690, 696 (7th Cir. 2019); Brown, 765 F.3d at 773. The expert must also “reliabl[y] appl[y]” her chosen methodology on her way to reaching a conclusion. FED. R. EVID. 702(d); Brown, 765 F.3d at 772 (after identifying reliable methodology, expert must “faithfully apply [it] to the facts at hand.”). That is where Dr. Wells' meta-analysis fails to meet the threshold articulated in Daubert. Thus, any testimony based on Dr. Wells' meta-analysis and the resulting odds ratio of 2.8 reflecting a “near tripling” of the occurrence of Parkinson's disease in people who are occupationally exposed to paraquat is inadmissible under Rule 702.

3. Dr. Wells' Weight of the Evidence /Bradford Hill Analysis

a. General Observations

After he generated his odds ratio of 2.8, Dr. Wells conducted a “weight of the evidence”⁴⁹ review utilizing the Bradford Hill framework to determine whether the association was attributable to a cause-and-effect relationship between occupational paraquat exposure and Parkinson's disease. This analysis involved the “combination of two methods”-weight of the evidence review and application of the Bradford Hill factors. In re Zoloft, 858 F.3d at 795. Although this approach is generally reliable, there is “very little” circuit-level authority guiding its application in toxic tort cases. Id. at 796 & n.49. Neither the Parties, nor this Court, have identified binding authority from the Seventh Circuit that would govern the assessment of the reliability of Dr. Wells' application of these methodologies. However, the First and Third Circuits, along with some others, have weighed in on the subject, and their views appear pertinent to the issue at hand.⁵⁰

49 Dr. Wells and the Parties appear to use the terms “weight of the evidence review” and “totality of the evidence review” interchangeably. (Doc. 4355-3 at 33, 34 (Wells Rebuttal Report); Doc. 4561 at 8, 68 (Plaintiffs' Response to Defendants' Motion to Exclude Dr. Wells); Doc. 4798 at 14, 15 (Defendants' Closing Brief)). See also Gilbert v. Lands' End, Inc., Nos. 19-cv-823-jdp, 19-cv-1066-jdp, 2022 WL 2643514, at *9 (W.D. Wis. July 8, 2021) (recognizing that courts use the labels “totality of the evidence” and “weight of the evidence” interchangeably when referring to the same methodology). The preponderance of authorities cited by the Parties refers to this methodology as “weight of the evidence review.” See e.g., In re Zoloft, 858 F.3d at 795; In re Mirena IUS Levonorgestrel-Related Prod. Liab. Litig. (No. II), 341 F.Supp.3d 213, 247 (S.D.N.Y. 2018). The Court will stay consistent with this designation and refer to Dr. Wells' methodology as “weight of the evidence” review.

50 See e.g., In re Lipitor (Atorvastatin Calcium) Mktg., Sales Pracs., & Prod. Liab. Litig., 892 F.3d 624, 642 (4th Cir. 2018) (excluding expert's Bradford Hill analysis because his reliance on statistically weak evidence was not “accepted in his field”); Hollander v. Sandoz Pharmaceuticals Corp., 289 F.3d 1193, 1204 n.7 (10th Cir. 2002) (briefly mentioning Bradford Hill criteria before analyzing different categories of evidence); Daniels-Feasel v. Forest Pharmaceuticals, Inc., No. 22-146, 2023 WL 4837521, at *3 (2d Cir. July 28, 2023) (selection bias warranted exclusion of expert's Bradford Hill analysis).

The First Circuit explained in Milward v. Acuity Specialty Products Group, Inc. that “[t]he hallmark of the weight of the evidence approach is reasoning to the best explanation for all of the available evidence.” 639 F.3d 11, 23 (1st Cir. 2011). Experts in toxic tort actions commonly employ this methodology to answer complex epidemiological causation questions. In re Zoloft, 858 F.3d at 795-97. Because of its widespread adoption in the scientific community and in other litigations, the general reliability of this approach is not in dispute. Id. at 795, 796-97. However, the method gives researchers significant flexibility to decide how to analyze the evidence and weight each Bradford Hill factor in relation to the others. Id. at 796. An expert could “theoretically assign the most weight to only a few factors, or draw conclusions about one factor based on a particular combination of evidence.” Id. So, while the methodology offers the benefit of flexibility, it is vulnerable to results-driven analysis, which, of course, raises significant reliability concerns. In re Mirena, 341 F.Supp.3d at 247. In Mirena, the Southern District of New York offered the following commentary:

[I]t is imperative that experts who apply multi-criteria methodologies such as Bradford Hill or the “weight of the evidence” rigorously explain how they have weighted the criteria. Otherwise, such methodologies are virtually standardless and their applications to a particular problem can prove unacceptably manipulable. Rather than advancing the search for truth, these flexible methodologies may serve as vehicles to support a desired conclusion.

Id. Thus, “[t]o ensure that the Bradford Hill/weight of the evidence criteria is truly a methodology, rather than a mere conclusion-oriented selection process[,] there must be a scientific method of weighting that is used and explained.” In re Zoloft, 858 F.3d at 796 (cleaned up). With these considerations in mind, an expert who relies on a weight of the evidence review based on Bradford Hill framework must, at a minimum, “explain . . . how conclusions are drawn for each Bradford Hill criterion and . . . how the criteria are weighed relative to one another.” Id.

A district court, for its part, should avoid treating each subsidiary conclusion that the expert draws “atomistically.” Milward, 639 F.3d at 23. This means that district courts should not impose their own view of the science to question the validity of the expert's substantive conclusions, Id. at 23-25, a proposition that the Seventh Circuit embraces as well. Manpower, 732 F.3d at 806. In Milward, an expert offered a general causation opinion that workplace exposure to benzene-containing products was capable of causing Acute Promyelocytic Leukemia. 639 F.3d at 19. The district court excluded the expert's proffered opinion because it found that some of his subsidiary conclusions regarding mechanistic and epidemiological evidence had insufficient scientific support. Id at 20-23. The First Circuit reversed, finding that the district court had improperly usurped the role of the jury by independently judging the weight of the evidence supporting the expert's conclusions. Id. at 22-23. Moreover, the district court had improperly concluded that “because no one line of evidence supported a reliable inference of causation,” the inference of causation as a whole was unreliable. Id. at 23. Instead, it was appropriate to treat “each body of evidence . . . as grounds for the subsidiary conclusion that it would, if combined with other evidence support a causal inference.” Id.

Consistent with this logic and the plethora of authorities already discussed, the Court's evaluation of Dr. Wells' weight of the evidence analysis is focused on methodological soundness, not scientific accuracy.

b. The Reliability of Dr. Wells' Bradford Hill Analysis

Dr. Wells' weight of the evidence/Bradford Hill analysis is a textbook example of the type of standardless presentation of evidence that courts have cautioned against. The most obvious methodological defect is the absence of any discernible weighting methodology. Neither Dr. Wells' first report nor his rebuttal report offer any explanation of the relative weight or importance assigned to each of the six Bradford Hill factors he analyzed. This omission is a clear violation of the requirement that experts utilizing the weight of the evidence methodology explain “how the criteria are weighed relative to one another.” In re Zoloft, 858 F.3d at 796. Dr. Wells, at best, cites favorable authority for each Bradford Hill factor before apparently concluding that it is satisfied for purposes of his analysis. But “[b]y leaving obscure the weight that he attaches to each . . . Bradford Hill factor[] and the relationship among them, [Dr. Wells'] approach effectively disables a finder of fact from critically evaluating his work.” In re Mirena, 341 F.Supp.3d at 248. Indeed, the lack of any relative weight assignments means that Dr. Wells' general causation opinion is virtually non-falsifiable, one of the most basic requirements of the scientific method. See Zenith, 395 F.3d at 419 (“[C]onclusions that are not falsifiable aren't worth much to either science or the judiciary.”). This is so because Dr. Wells has not told us which factors drive his general causation opinion (i.e., which ones he considers most probative of a causal relationship); they could all be equally important, or some could be more important than others. If a jury were to disagree with his conclusion on one factor, he would be able to declare by unilateral fiat that his ultimate opinion on general causation remains intact because that factor was not important. In re Mirena, 341 F.Supp.3d at 248-49. His general causation opinion is therefore immunized from critical scrutiny through its malleability. In re Zoloft, 858 F.3d at 796.

The second systemic methodological deficiency is that Dr. Wells leaves it to the Court (and by extension, the jury) to identify evidence in his two reports that supports his conclusions. Dr. Wells' Bradford Hill analysis spans only five pages of his first report. (Doc. 4355-2 at 21-26). Plaintiffs insist, however, that “the totality of Dr. Wells' two reports should be incorporated into his Bradford Hill analysis.” (Doc. 4561 at 35). Although the Court accepts this proposition, it is not as straight-forward as it sounds. Take for example, the Bradford Hill consideration of specificity. This consideration states that an association is more likely to be causal if the exposure is associated with only a single or a small number of diseases. RMSE at 605-06. Dr. Wells' first report devotes one paragraph to this consideration. This paragraph attempts to analogize the lack of specificity in health outcomes that are associated with paraquat exposure (he notes that paraquat exposure is also associated with kidney and lung disease) to the lack of specificity in health outcomes associated with smoking (i.e., emphysema and cardiovascular disease, in addition to lung cancer). (Doc. 4355-2 at 25-26). Dr. Wells appears to suggest-although he does not explicitly say so-that while paraquat, like smoking, is associated with a range of negative health outcomes, it can still cause Parkinson's disease because “there is little doubt in the causal nature of smoking and lung cancer.” Id. at 26. Apparently, the reader is invited to infer from this analogy that the lack of specificity does not undermine causation because smoking is causally related to lung cancer even though it is also associated with other negative health outcomes. It is entirely unclear, however, whether Dr. Wells even concludes that this factor is met based on the evidence he presents. And further, as far as the Court can tell, the remainder of Dr. Wells' first report and his rebuttal report fail to offer any additional evidence that would inform the discussion of this Bradford Hill consideration. So, even though the Court is willing to indulge Plaintiffs' invitation to consider the totality of Dr. Wells' two reports, this expanded review creates just as many questions as it answers because it (i) does not clarify whether Dr. Wells considers this factor to be satisfied; and (ii) leaves it to the Court to scour the reports and judge an evidentiary item's relevance to a particular Bradford Hill factor. This approach “makes it all too easy for [Dr. Wells] to manipulate the Bradford Hill factors to support a desired conclusion of causation, and far too hard for an ensuing expert to replicate and rigorously test [his] analytic approach.” In re Mirena, 341 F.Supp.3d at 268.

Against the backdrop of Dr. Wells' departure from the most basic methodological requirements of a weight of the evidence review, it is not surprising that his analysis reveals extensive selection bias.⁵¹Dr. Wells appears to have fallen prey to the temptations of selection bias in his discussion of several Bradford Hill factors, most notably those concerning a dose-response relationship and strength of association. Because “reliance on an anemic and one-sided set of facts casts significant doubt on the soundness of [an expert's] opinion,” Smith v. Ill. Dept. of Transp., 936 F.3d 554, 558-59 (7th Cir. 2019), Dr. Wells' outcome-driven Bradford Hill analysis compels the exclusion of his general causation opinion. The following discussion explains how Dr. Wells' disregard for the basic methodological requirements of a weight of the evidence review resulted in unreliable analyses of a dose-response relationship and strength of the association. See Daniels-Feasel v. Forest Pharmaceuticals, Inc., No. 22-146, 2023 WL 4837521, at *3 (2d Cir. July 28, 2023) (affirming exclusion where expert “cherry-picked only favorable studies to support his causal conclusion and did not rigorously explain the weight he attached to each Bradford Hill factor”).

51 The Court has already explained how Dr. Wells' selection of studies for his meta-analysis appeared to be impermissibly results-driven based on his redefinition of key concepts, his claim to have relied on a two-step selection methodology after testifying that his review was “holistic,” and his imposition of inconsistent selection criteria that favored studies showing strong associations over studies that did not. These concerns are present at this step of the Court's reliability analysis as well, considering Plaintiffs' and Dr. Wells' insistence that his Bradford Hill analysis incorporates his first and rebuttal reports in full.

i. Dose-Response Relationship

The consideration of a dose-response relationship analyzes whether increased exposures are associated with increased risk of an adverse outcome. If they are, it tends to support the existence of a causal relationship. Dr. Wells begins his discussion of this Bradford Hill factor by citing Liou (1997)'s adjusted odds ratio of 6.44, with a confidence interval of 2.41 - 17.2, for study participants who were exposed to paraquat for 20 years or more. He then compares this result to the odds ratio of .96, with a confidence interval of .24 - 3.83, generated by study participants who reported exposures of 1 to 19 years to show that longer exposure periods are associated with a higher risk of Parkinson's disease. (Doc. 4355-2 at 24). Dr. Wells also cites Tanner (2011);⁵² a case-control study by Goldman et al. (2012)⁵³(examining how genetic factors affect Parkinson's disease risk in individuals exposed to paraquat); a study by Brouwer et al. (2017)⁵⁴ (examining community exposure to paraquat (i.e., environmental drift and volatilization)); and a meta-analysis by Breckenridge (2016) in support of this proposition. (Doc. 4355-2 at 24-25). Finally, Dr. Wells cites a study by Furlong et al. (2015)⁵⁵to show that protective glove use modified associations between paraquat exposure and Parkinson's disease.

52 In two sentences of his first report, Dr. Wells cites an exposure duration analysis in Tanner to support his conclusion of a dose-response relationship. (Doc. 4355-2 at 24). Tanner compared study participants who had been exposed to paraquat for more than eight lifetime days to participants who had been exposed for eight lifetime days or fewer (the investigators had identified eight lifetime days as the median exposure duration among participants who reported paraquat exposure). (Doc. 4355-16 at 4, 12). Tanner found that participants with more than eight lifetime days of exposure were at a higher risk of developing Parkinson's disease than those with eight or fewer lifetime days of exposure (odds ratio of 3.6, with a confidence interval of 1.6 - 8.1, for participants with more than eight lifetime days of exposure; odds ratio of 2.4, with a confidence interval of 1.0 - 5.5, for participants with eight or fewer lifetime days of exposure). Id. at 12. According to the EPA, this analysis “does not constitute a formal analysis of the dose-response relationship between paraquat exposure and PD” because it was unclear whether the threshold of eight lifetime days of exposure was even “biologically meaningful.” (Doc. 4558-12 at 34).

53 Samuel M. Goldman et al., Genetic modification of the association of paraquat and Parkinson's disease, 27 Movement Disorders 1652 (2012). Goldman was part of the AHS and its study population partially overlapped with Tanner's. (Docs. 4355-2 at 25; 4355-3 at 45).

54 Maartje Brouwer et al., Environmental exposure to pesticides and the risk of Parkinson's disease in the Netherlands, 107 Environment International 110 (2017) (Doc. 4654-5).

55 Melissa Furlong et al., Protective glove use and hygiene habits modify the associations of specific pesticides with Parkinson's disease, 75 Environment International 144 (2015).

Specifically, Furlong found that when protective gloves were worn during 50% of paraquat uses or fewer, the odds ratio was 3.9, with a confidence interval of 1.5 - 10.2. Id. at 25. When protective gloves were worn during more than 50% of paraquat uses, the odds ratio plummeted to 1.6, with a confidence interval of .6 - 4.2. Id. These data points would appear to support an inference that a dose-response relationship exists between paraquat exposure and Parkinson's disease.

But the goal of a weight of the evidence/Bradford Hill analysis is not simply to find support for an outcome while disregarding conflicting evidence. In re Zoloft, 858 F.3d at 796. An expert must grapple with all relevant evidence and explain why her conclusion is scientifically justified after considering favorable and unfavorable data. Cates v. Whirlpool Corp., No. 15-CV-5980, 2017 WL 1862640, at *15 (N.D. Ill. May 9, 2017). Dr. Wells' analysis of a potential dose-response relationship falls short of this standard because it failed to account for at least two critically important studies. Shrestha (2020) and van der Mark (2014) also examined a potential dose-response relationship between paraquat exposure and Parkinson's disease. Shrestha calculated “intensity-weighted lifetime days” of paraquat exposure and found that study subjects in the highest exposure category generated the lowest association between paraquat exposure and Parkinson's disease (odds ratio of .74, with a confidence interval of .39 - 1.39) compared to study subjects with fewer days of exposure and lower intensities. (Doc. 4558-9 at 3, 27-28). Van der Mark reported similar findings-subjects who fell into higher exposure categories generated an odds ratio of 1.01, with a confidence interval of .48 - 2.12, whereas subjects in the lower exposure category had a higher odds ratio of 1.42 with a confidence interval of .71 - 2.85. (Doc. 4355-20 at 7). These findings suggest an inverse dose-response relationship, which is, of course, inconsistent with Dr. Wells' conclusion on this Bradford Hill factor. At his first deposition, Dr. Wells appeared to admit that he relied on supportive evidence to examine dose-response relationships, while disregarding evidence to the contrary:

QUESTION: So[,] you bent the rules and considered Brouwer in your Bradford Hill analysis but you wouldn't bend the rules and consider the dose-response calculations in van der Mark?

DR. WELLS: That's not part of the meta-analysis. It's just looking at general -- what are general results in this area of exposure to paraquat and PD.

QUESTION: So[,] aren't the dose-response results from van der Mark general results in the area of exposure to paraquat and Parkinson's disease?

DR. WELLS: So they're results but, yeah, they're null results; whereas, Brouwer had positive results.

(Doc. 4364-20 at 66) (emphasis added). The italicized portion of this exchange suggests that Dr. Wells was plainly interested in finding “positive results” that supported a doseresponse relationship. Indeed, he searched the epidemiological literature for data that supported his conclusion, while considering inconsistent evidence to be meaningless. His rebuttal report then doubles down on this proposition, with the conclusory declaration that “[n]on-significant statistical results would not impact the analysis and do not demonstrate a lack of a dose response.” (Doc. 4355-3 at 34). This is the very definition of unscientific cherry-picking. “[U]s[ing] only certain data from studies showing a positive dose-response relationship and omit[ting] data from studies showing either no doseresponse relationship or a negative dose-response relationship indicates a results-driven methodology.” In re Zantac (Ranitidine) Prod. Liab. Litig., 644 F.Supp.3d 1075, 1270 (S.D. Fla. 2022). Zantac's admonition is appropriate here-Dr. Wells' singular focus on “positive results” undermines the reliability of his dose-response conclusion.

The same is true of Dr. Wells' reliance on Furlong (2015) to argue that protective glove use is associated with a decreased risk of Parkinson's disease in exposed individuals and therefore supportive of a dose-response relationship. Shrestha examined protective glove use as well and was unable to replicate the modified association found in Furlong. Indeed, Shrestha cited Furlong to observe that “[w]earing chemical-resistant gloves was previously shown to modify PD associations with some pesticides.” (Doc. 4558-9 at 5). But while Shrestha found that some pesticides were associated with elevated Parkinson's disease risks among individuals who did not use chemical-resistant gloves, it made no such finding with respect to paraquat. Id. at 7. Dr. Wells was confronted with this inconsistency at his first deposition, as shown in the following exchange:

QUESTION: But you rely on [Furlong] for dose response?

DR. WELLS: But it's a nice study of gloves and PPE.

QUESTION: You know that Shrestha reports on glove use as well, correct?

DR. WELLS: Yes.

QUESTION: And Shrestha does not find an association with glove use and paraquat exposure?

DR. WELLS: Correct.

QUESTION: But you did not rely on that in your dose-response conclusion?

DR. WELLS: No, I used Furlong.

(Doc. 4364-20 at 78). Dr. Wells' testimony suggests he did not even attempt to square Furlong with Shrestha. He simply cited Furlong in support of his subsidiary conclusion and moved on. See Milward v. Rust-Oleum Corp., 820 F.3d 469, 475 (1st Cir. 2016) (expert's “complete unwillingness to engage with the conflicting studies . . . made it impossible for the district court to ensure that her opinion was actually based on scientifically reliable evidence.”).

This is not to say that Dr. Wells failed to consider Shrestha or van der Mark entirely. His reports make it clear that he did (at least in the sense that he addressed why he considered them to be of lower epidemiological quality). Thus, if he believed that Shrestha and van der Mark's data on dose-response was entitled to no weight at all because he considered them to be low quality studies, he should have said so and explained why the studies he relied on were of sufficiently high quality to displace their results entirely. The fact that Shrestha's and van der Mark's dose-response data did not affect Dr. Wells' conclusion is a reasonable position for someone with his skill and experience to take. But it was incumbent upon him to explain why. See In re Zoloft, 858 F.3d at 796 (“[A]n expert must explain . . . how conclusions are drawn for each Bradford Hill criterion.”).

To reemphasize a point made previously-the Court does not seek to substitute its view of the science for Dr. Wells'. The Court takes no position as to the scientific merit of Dr. Wells' conclusion that a dose-response relationship exists between paraquat exposure and Parkinson's disease. Nor does the Court claim to know whether the evidence Dr. Wells cites in favor of a dose-response relationship carries more scientific weight than the evidence that refutes it. Rather, the Court is concerned with Dr. Wells' methodological rigor in forming his opinions. And on this point, his failure to explain how the calculations of Shrestha and van der Mark factor into his opinion that a doseresponse relationship exists is a serious methodological flaw. See In re Zoloft (Sertraline Hydrochloride) Prod. Liab. Litig., 176 F.Supp.3d 483, 492-93 (E.D. Pa. 2016) (“[T]he Court holds that when epidemiological studies are equivocal or inconsistent with a causation opinion, experts asserting causation opinions must thoroughly analyze the strengths and weaknesses of the epidemiological research and explain why that body of research does not contradict or undermine their opinion.”).

ii. Strength of Association

Dr. Wells' examination of the strength of the association is perhaps even more onesided because it is ostensibly based entirely on the results of his meta-analysis. The consideration of the strength of the association is the “cornerstone for causal inferences” because “[t]he higher the relative risk, the stronger the association and the lower the chance that the effect is spurious.” RMSE at 602. According to Dr. Wells, an odds ratio above 2.0 “represent[s] a strong effect.” (Doc. 4355-2 at 22). To make this showing, Dr. Wells relies on the results of his meta-analysis.⁵⁶ (Doc. 4355-2 at 11-14, 22). But as noted in the discussion of Dr. Wells' meta-analysis, the odds ratio of 2.8 is the product of numerous methodological defects that strongly undermine its reliability. Accordingly, Dr. Wells' reliance on this odds ratio to support his conclusion of a strong association between occupational paraquat exposure and Parkinson's disease is problematic in and of itself. Indeed, even his reliance on his meta-analysis appears to overlook the fact that five of the seven studies in it yielded a statistically insignificant result. Although he notes that “[f]ailure to demonstrate statistical significance in a single study does not preclude the possibility of a meaningful exposure-response relationship,” (Doc. 4355-2 at 22) (emphasis added), the lack of statistical significance is not only evident in one study; it is prevalent throughout the epidemiological literature. Dr. Wells' failure to engage this critical data point is further evidence of his selective reliance on favorable evidence. See In re Acetaminophen - ASD-ADHD Prod. Liab. Litig., __ F.Supp.3d __, 2023 WL 8711617, at *28 (S.D.N.Y. 2023) (several studies showing odds ratios between 1.0 and 2.0 and lack of statistical significance undermines expert's conclusion regarding strength of an association); cf. In re Zoloft (Sertraline Hydrochloride) Prod. Liab. Litig., 26 F.Supp.3d 449, 457 (E.D. Pa. 2014) (criticizing expert's “fail[ure] to demonstrate that her reliance on non-statistically significant findings is accepted within her scientific community”). But beyond his attempt to satisfy this Bradford Hill factor with the fruit of his flawed metaanalysis, Dr. Wells' examination of the strength of the association is methodologically unsound because of the evidence he admittedly disregards.

56 Dr. Wells also briefly mentions two other meta-analyses he conducted, which included occupational casecontrol studies that were cited in the EPA's systematic review. (Doc. 4558-12). The first of these two metaanalyses included the occupational case-control studies that the EPA had considered, regardless of quality assessment. This random effects meta-analysis resulted in an odds ratio of 1.81, with a confidence interval of 1.08 - 3.01. The second of these two meta-analyses included only the occupational case-control studies that the EPA had judged to be of high or moderate quality, resulting in an odds ratio of 2.19, with a confidence interval of 1.21 - 3.97. (Doc. 4355-2 at 11-14).

At his first deposition, Dr. Wells admitted that he did not rely on studies that did not support a strong association between paraquat and Parkinson's disease in his examination of this Bradford Hill factor. (Doc. 4364-20 at 76). When he was asked about the scope of his review of the evidence concerning the strength of the association, Dr. Wells offered the following testimony:

QUESTION: The first [Bradford Hill factor] you discuss in the report is strength, correct?

DR. WELLS: Yes.

QUESTION: And you rely on the results of your meta-analysis, correct?

DR. WELLS: Yes.

QUESTION: Your opinion on strength does not rely on Shrestha or Pouchieu, correct?

DR. WELLS: No.

(Doc. 4364-20 at 76). This omission strongly indicates selection bias because Dr. Wells admitted that Shrestha and Pouchieu et al. (2018)⁵⁷ are among the “relevant collection of studies” for anyone addressing the strength of the association Id.

57 Camille Pouchieu et al., Pesticide use in agriculture and Parkinson's disease in the AGRICAN cohort study, International Journal of Epidemiology, Vol. 47, No. 1, 299 (2018) (Doc. 4355-19).

Pouchieu was a cross-sectional study that examined associations between Parkinson's disease and pesticide use, including paraquat. (Doc. 4355-19 at 2). The investigators presented their findings with and without adjustments for co-exposures to other active ingredients. Pouchieu found an unadjusted odds ratio of 1.43, with a confidence interval of 1.17 - 1.75, indicating a statistically significant risk increase in participants with paraquat exposure. Id. at 9. Critically, however, after adjusting for coexposures, the odds ratio fell to 1.01, with a confidence interval of .41 - 2.49, leading the authors to conclude that the adjustment caused the association to “disappear[].” Id. at 7, 9. Moreover, Shrestha is important because it was unable to “reproduc[e]” the positive association identified in Tanner (2011), based on its hazard ratio of 1.09, with a confidence interval of .84 - 1.41. (Doc. 4558-9 at 8).

Dr. Wells claims that he “considered” Shrestha and Pouchieu for his strength of the association analysis but “do[es] not believe that either warrants a reduction of the OR calculated in [his] meta-analysis.” (Doc. 4355-3 at 37). It is true that Dr. Wells discussed both studies throughout his two reports, but he appears to have done so only by highlighting what he perceives to be their methodological flaws, which warranted their exclusion from his meta-analysis. Dr. Wells himself acknowledges, however, that “[a]ssessing the strength of association in causal inference requires examination of the weight of evidence across the relevant collection studies.” (Doc. 4355-2 at 22) (emphasis added). And Shrestha and Pouchieu are, according to Dr. Wells, part of the “relevant collection of studies.” (Doc. 4364-20 at 76). Thus, by Dr. Wells' own admission, Shrestha and Pouchieu should have been weighted against any favorable evidence as part of a reliable weight of the evidence review. See In re Zantac, 644 F.Supp.3d at 1253 (in weight of the evidence analysis “weights must be assigned to data according to a scientific approach”). But that did not happen. The Court's best guess, based on an indulgent review of both reports, is that Pouchieu and Shrestha received no weight in Dr. Wells' analysis of the strength of the association based on his view that they were of low epidemiological quality. If that is so, then Dr. Wells' subsidiary conclusion about the strength of the association begins and ends with his meta-analysis. This is a strikingly narrow view of what Dr. Wells admits is a “rich” area of study. (Doc. 4561-10 at 14). This targeted view of the evidence also exposes Dr. Wells' analysis of this Bradford Hill factor as an “anemic and one-sided” assessment, which is impermissible under Rule 702. Smith, 936 F.3d at 558; see also Gen. Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997) (“A court may conclude that there is simply too great an analytical gap between the data and the opinion proffered.”).⁵⁸

58 Dose-response and strength of association are not the only Bradford Hill considerations that appear to have been corrupted by selection bias in Dr. Wells' weight of the evidence analysis. The consideration of temporality is another example of Dr. Wells' outcome-driven approach. Temporality is the only mandatory Bradford Hill factor because causation cannot exist if the disease predates the exposure. The only epidemiological study design that effectively controls for temporality is the prospective cohort design because it follows its subjects for extended periods of time to monitor the onset and development of the disease. Dr. Wells acknowledges as much-prospective cohort studies, as he explains, are “guaranteed to get the temporality correctly.” (Doc. 4364-20 at 23). Dr. Wells' discussion of temporality, however, completely ignores Shrestha (2020), a prospective cohort study that, by virtue of its design, would have controlled for temporality more effectively than any of the studies in his meta-analysis. Indeed, Shrestha evaluated more paraquat-exposed cases than all seven case-control studies in Dr. Wells' meta-analysis combined (87 total cases in Shrestha compared to 72 combined cases in the meta-analysis). (Docs. 4558-9 at 6; 4355-14 at 15-16). Although Dr. Wells explained why he considered Tanner (2011) to be a higher quality study than Shrestha for purposes of his meta-analysis, the omission of Shrestha from his discussion of temporality further indicates a results-driven methodology. See In re Zantac (Ranitidine) Prod. Liab. Litig., 644 F.Supp.3d 1075, 1239 (S.D. Fla. 2022) (not enough for an expert to point out what she sees as weaknesses in studies that refute her conclusion; expert must also “explain why that body of research does not contradict or undermine her opinion”) (quotation marks and alterations omitted).

At its core, Dr. Wells' weight of the evidence/Bradford Hill analysis is a selective presentation of supportive evidence that fails to meaningfully account for data points that refute his conclusions. His failure to weight the Bradford Hill factors against each other, his reliance on the Court to match evidence to each factor from other places in his reports, and his failure to meaningfully analyze evidence that refutes his conclusions requires the exclusion of his opinion that occupational paraquat exposure is causally related to Parkinson's disease under Rule 702.

c. Isolation from the Scientific Community

This Court has focused, as it must, on the methodological soundness of Dr. Wells' analyses in support of his conclusions, not the conclusions themselves. But this analytical focus does not mean that Dr. Wells' conclusions are entirely insulated from scrutiny. The Supreme Court recognized in Joiner that “conclusions and methodology are not entirely distinct from one another.” 522 U.S. at 146. This proposition accommodates a simple reality of expert testimony: the line between methodology and conclusion is “not always an easy [one] to draw.” Manpower, 732 F.3d at 806. As such, the Court offers its thoughts on an issue that was hotly contested in the Parties' briefing and at the Daubert hearing: whether Dr. Wells is “alone” in the scientific community with his opinion that occupational exposure to paraquat can cause Parkinson's disease.

Daubert expressly addressed the importance of independent validation of an expert's opinion when it observed that “[w]idespread acceptance can be an important factor in ruling particular evidence admissible.” Daubert, 509 U.S. at 594. The Advisory Committee on the Federal Rules of Evidence therefore cautions that “when an expert purports to apply principles and methods in accordance with professional standards, and yet reaches a conclusion that other experts in the field would not reach, the trial court may fairly suspect that the principles and methods have not been faithfully applied.” FED R. EVID. 702 advisory committee's note to 2000 amendments. With these principles in mind, the Court asked Plaintiffs' counsel at the Daubert hearing whether any peer-reviewed publication had found a causal relationship between paraquat exposure and Parkinson's disease. Plaintiffs' counsel responded by citing an opinion article by Dr. Earl Ray Dorsey and Dr. Amit Ray titled “Paraquat, Parkinson's Disease, and Agnotology” in Movement Disorders, the official journal of the International Parkinson and Movement Disorder Society.⁵⁹ (Doc. 4795 at 171). This article accuses Defendants of obfuscating the science surrounding their product and engaging in “attacks on science, attacks on scientists, and attacks on the health of the public.” The article then cites a newspaper article for the proposition that Parkinson's researchers “know what one cause of Parkinson's disease is-paraquat.” Dorsey & Ray, supra. And based on this conclusion, the article urges regulators to “ban[]” paraquat. Id. As this quoted language suggests, the article is styled as an advocacy piece, not a scientific analysis of the causal relationship between paraquat exposure and Parkinson's disease.

59 See E. Ray Dorsey & Amit Ray, Paraquat, Parkinson's Disease, and Agnotology, 38 Movement Disorders 949 (2023).

Considering the article's brevity and lack of scientific analysis, the Court was and still is skeptical of its ability to independently validate Dr. Wells' general causation opinion. But to further engage on this issue, the Court asked Plaintiffs' counsel whether any scientist outside of this litigation had ever conducted a Bradford Hill analysis and concluded that paraquat exposure was causally related to Parkinson's disease. On this point, Plaintiffs' counsel conceded that he was not aware of such an analysis. (Doc. 4795 at 173-74). Plaintiffs' counsel also argued that Dr. Wells and Defendants' expert, Dr. Alexander, are “the only people in the world that have ever done what we have done in this case, and it's the truth. This is litigation. But there's not a soul in the world that has looked at occupational exposure to paraquat and its relationship to Parkinson's disease like what has been done in your courtroom.” Id. at 174. This argument is somewhat perplexing as it appears to support the proposition that Dr. Wells is alone in the scientific community, notwithstanding decades of scientific inquiry into this exact issue.

It is beyond dispute, in the Seventh Circuit, that “[l]aw lags science; it does not lead it.” Rosen v. Ciba-Geigy Corp., 78 F.3d 316, 319 (7th Cir. 1996). Dr. Wells, for his part, admitted that he is not aware of any peer-reviewed literature that establishes a cause of Parkinson's disease other than genetic mutation. (Doc. 4364-20 at 11). Dr. Wells also admitted that none of the published systematic reviews he relied on to collect relevant studies for his meta-analysis found a causal relationship between paraquat and Parkinson's disease. (Doc. 4561-10 at 75). It is indeed telling that a recent review of reviews offered the following observation about the putative link between paraquat and Parkinson's disease:

No author of any published review stated that it has been established that exposure to paraquat causes Parkinson's disease, regardless of methods used and independent of funding source. A consensus exists in the scientific community that the available evidence does not warrant a claim that paraquat causes Parkinson's disease.

Douglas L. Weed, Does paraquat cause Parkinson's disease? A review of reviews, 86 Neurotoxicology 180, 180 (2021) (Doc. 4355 at 2). Dr. Wells' opinion that “drawing general causal inferences related to occupational paraquat exposure and PD is merited,” (Doc. 4355-2 at 26), places him in direct conflict with this observed “consensus.” This type of isolation from the scientific community triggers significant reliability concerns because “courts may only admit the state of science as it is.” Rider v. Sandoz Pharmaceutical Corp., 295 F.3d 1194, 1202 (11th Cir. 2002) (emphasis added).

The point here is not that Dr. Wells is substantively wrong. Again, the Court is neither qualified nor permitted to make that assessment. Smith, 215 F.3d at 719. The point is that, at least to date, Dr. Wells' causation theory has not been adopted or independently validated in any peer-reviewed scientific analysis outside of this litigation. The Seventh Circuit and courts around the country view such scientific isolation as an evidentiary red flag. See Am. Honda Motor Co. v. Allen, 600 F.3d 813, 818 (7th Cir. 2010) (per curiam) (“Despite its publication, there is no indication that [the expert's] wobble decay standard has been generally accepted by anyone other than [the expert himself],” the author of the publication); Chapman v. Maytag Corp., 297 F.3d 682, 688 (7th Cir. 2002) (expert's theory did not satisfy Daubert because it was “novel and unsupported by any article, text, study, scientific literature or scientific data produced by others in his field”); accord Tamraz v. Lincoln Elec. Co., 620 F.3d 665, 677-78 (6th Cir. 2010); Lust by and through Lust v. Merrell Dow Pharmaceuticals, Inc., 89 F.3d 594, 598 (9th Cir. 1996); In re Zantac, 644 F.Supp.3d at 1234. So too does this Court. This is not to say that the lack of peer-reviewed support for Dr. Wells' general causation opinion is alone dispositive of its reliability under Rule 702; but it does tend to undermine reliability, rather than support it. Timm, 932 F.3d at 995; see also Kirk v. Clark Equip. Co., 991 F.3d 865, 877 (7th Cir. 2021) (lack of peer-reviewed support for causation opinion combined with other methodological deficiencies weighs against admission).

4. Dr. Wells' Plaintiff-specific Opinions

Dr. Wells' third and final proffered opinion concerns the individual Plaintiffs in the four trial selection cases, Mr. Burgener, Mr. Coward, Mr. Fuller, and Mr. Richter. According to Dr. Wells, “these individuals fit the inclusion criteria in the referenced seven studies in my meta-analysis.” (Doc. 4355-2 at 26). As a result, “[t]he elevated odds ratio of 2.8[] and the [Bradford] Hill criteria apply to these individuals.” Id. at 27.

This opinion will be excluded because it is not severable from Dr. Wells' metaanalysis and his weight of the evidence/Bradford Hill analysis. Plaintiffs conceded this point at the Daubert hearing by noting that Dr. Wells' meta-analysis is “critical for specific causation.” (Doc. 4795 at 170). Without testimony concerning (i) a positive association between occupational paraquat exposure and Parkinson's disease, and (ii) a causal relationship between occupational paraquat exposure and Parkinson's disease, there is no testimony for Dr. Wells to give as it pertains to the four trial selection Plaintiffs. Accordingly, Dr. Wells' Plaintiff-specific opinions are excluded under Rule 702.

Conclusion

For these reasons, the Court concludes that Dr. Wells' proffered opinions are not admissible under Federal Rule of Evidence 702 and Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). Defendants' Motion to Exclude Dr. Wells' Expert Testimony (Doc. 4355) is GRANTED, and Dr. Wells' testimony will be EXCLUDED at trial. The Court will issue a separate order addressing the status of the trial selection cases in light of this ruling.

Finally, on March 24, 2023, Defendants filed a Motion to Strike Dr. Wells' Rebuttal Report. (Doc. 3855). On March 28, 2023, Plaintiffs filed an Emergency Motion to deem Defendants' Motion to Strike as a Daubert Challenge. (Doc. 3881). The Court denied Defendants' Motion to Strike on April 12, 2023 (Doc. 4044), and now DENIES Plaintiffs' Motion (Doc. 3881) as moot considering this ruling.

IT IS SO ORDERED.