Summary
concluding that a neurologist's testimony was not the product of reliable methodology in part because his opinion was incapable of being tested
Summary of this case from Hallmark v. EldridgeOpinion
Nos. CV-S-92-887-ECR, CV-S-93-0475-ECR, and CV-S-93-0476-ECR.
March 29, 1996.
Paul L. Larsen, Las Vegas, NV for Brian Valentine.
Larry C. Johns, Johns Johns, Las Vegas, NV, for Gary Butenschoen and Vicki Morris and co-counsel for Brian Valentine.
Kevin R. Stolworthy of Jones, Jones, Close Brown, Las Vegas, NV, J. Randall Jones of Harrison, Kemp Jones, Las Vegas, NV, for defendant.
Donald C. Smith, State of Nevada Dept. of Business Industry, Division of Industrial Relations, Las Vegas, NV, for movant Jerry Sieren.
ORDER
About an hour after midnight on May 6, 1991, a pipe connected to a storage tank at Defendant Pioneer Chlor Alkali Company's chemical manufacturing facility at Henderson, Nevada ruptured, allowing the contents of the tank to escape into the atmosphere. The tank had been used to store liquid chlorine, and the rupture of the pipe created a cloud of chlorine vapor which hung in the air and began to invade the homes and bodies of the town's inhabitants.
This case arises out of that accidental release of chlorine into the air in Henderson. The plaintiffs are three Henderson residents who claim that inhaling chlorine has damaged not only their lungs but their brains and nervous systems as well. Plaintiffs and their counsel have enlisted the help of several doctors, whom they have proposed to call to give sworn testimony as to the observed effects of chlorine gas exposure on their neurological and emotional health. Defendant Pioneer Chlor Alkali Company has moved this court for an order barring Plaintiffs' experts from offering their medical opinions as expert scientific evidence. Defendant's Motions in Limine (Doc. # 125 in Case No. 92-0887; Doc. # 122 in Case No. 93-0475).
Although these three plaintiffs, Brian Valentine, Gary Butenschoen and Vicki Morris originally filed separate lawsuits against defendant Pioneer Chlor Alkali, their claims were consolidated by Order of United States Magistrate Judge Johnston dated January 3, 1996 (Doc. # 141 in Case No. 92-887).
I. Daubert and the Law of Scientific Evidence
Lawyers, judges and juries are not usually trained in science or medicine, hence the need for the testimony of true experts in cases such as this where the plaintiffs claim a causal relation between the act or omission of the defendant and the subsequent appearance of disease in the plaintiffs. The drafters of the Federal Rules of Evidence hoped to give the federal courts some guidance in deciding when such expert testimony is appropriate, and in deciding what types of testimony legitimately qualify as science. Fed.R.Evid. 702.
A. Daubert: The Supreme Court Opinion
In 1993 the United States Supreme Court interpreted Evidence Rule 702 as superseding the traditional rule, announced in Frye v. United States, 293 F. 1013 (D.C. Cir. 1923); see also United States v. Solomon, 753 F.2d 1522, 1526 (9th Cir. 1985), which required the proponent of expert scientific testimony to show that the conclusions advanced by the proposed witness were generally accepted in the relevant scientific community. Frye was a relatively straightforward test. Litigants bolstered their proposed expert testimony with meta-evidence on the probity of the proposed testimony to scientific facts, and the court exercised its judgment with the aid of scientific consensus. Frye was the law for most of the twentieth century. In 1993 the Supreme Court wiped the slate clean.
The scientific facts to which an expert testifies need no longer achieve general acceptance in the scientific community to be admissible in federal court; rather, the proponent need only demonstrate that the investigatory methods employed by the proposed expert witness consist with generally accepted methods of scientific investigation. Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993) ( "Daubert").
Where the opponent of the proposed expert scientific testimony moves in limine against the admission of that science in evidence, the district court must determine whether the proposed testimony reflects "scientific knowledge," i.e. whether the proposed expert has arrived at her findings via the "scientific method." Put in shorthand form, the inquiry on a motion in limine is whether the expert has done "good science." Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1315 (9th Cir. 1995), on remand from 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993) ( "Daubert II"), cert. denied, ___ U.S. ___, 116 S.Ct. 189, 133 L.Ed.2d 126 (1995).
Plainly, the qualification of the proposed witness as an expert in the relevant field by no means disposes of the question whether the findings to which she will testify constitute "scientific knowledge." Daubert II, 43 F.3d at 1319. But neither does Daubert require that the accuracy of the proposed scientific evidence be established with absolute certainty. Daubert II, 43 F.3d at 1316. The key inquiry, in fact, does not concern the substantive "truth" of the proposed evidence at all, but rather focuses on the methodological basis of that evidence. Daubert II, 43 F.3d at 1316.
Thus, the essence of Daubert is the requirement that the district court be satisfied with the process employed by the proposed expert in obtaining her results. That said, Daubert offers comparatively little guidance with respect to the precise standards the district judge must apply to the proposed evidence before ruling on its admissibility.
B. Daubert II: The Ninth Circuit Opinion
The Ninth Circuit's decision on remand in Daubert itself began its attempt to apply the decision of the Supreme Court with the observation that the expert's self-serving assurance to the court of the integrity of her methodology would not satisfy Fed.R.Evid. 702. Daubert II, 43 F.3d at 1316. The Court of Appeals recognized that under the new rule, the proponent of expert scientific evidence must show that its expert witness's findings are grounded in sound science, and further recognized that that showing necessarily entails some independent indication that the expert's methodology was scientifically sound. Id. at 1316.
Judge Kozinski's opinion in Daubert II enumerated several factors to be used in determining the admissibility of expert scientific evidence: (1) whether the methods employed by the proposed witness have achieved "general acceptance" in the scientific community; (2) the extent to which the proposed scientific findings have been subjected to "peer review" and/or been published in a scientific journal; (3) the degree to which those findings are capable of empirical verification; and (4) whether the findings' margin of error lies within acceptable limits. The Daubert II court emphasized that this list of factors is by no means exhaustive, but only illustrative. 43 F.3d at 1316-17.
The Court of Appeals deemed "most persuasive and "very significant" the extent to which the scientific findings proposed as evidence derive from research conducted prior to and independent of the litigation. 43 F.3d at 1317. None of the proposed expert witnesses in this case will testify to scientific research they conducted prior to this lawsuit; indeed, there was before this case no scientific evidence, published or otherwise, that inhaling chlorine causes neurological damage.
One of plaintiffs' proposed expert witnesses, Dr. Kaye H. Kilburn, has suggested that the total absence of any previous scientific investigation into the neuropathological effects of chlorine inhalation results from the exclusive fascination of the scientific community on the effects of such exposure on the pulmonary system. Kaye H. Kilburn, Evidence that Inhaled Chlorine is Neurotoxic and Causes Airways Obstructions, 4 Int'l J. Occupational Med. Tox. 257, 275 (1995).
Absent proof that the proposed evidence results from prelitigation research, the evidence's proponent "must come forward with other objective, verifiable evidence that the testimony is based on `scientifically valid principles.'" Daubert II, 43 F.3d at 1317-18.
One indication that the proposed evidence is the result of good science is proof that the research and analysis supporting the expert witness's conclusions have been subjected to normal scientific scrutiny through the process of "peer review" and through publication in a generally-recognized scientific journal that conditions publication on a bona fide process of peer review. 43 F.3d at 1318 n. 6. The fact that the proposed evidence has withstood such scrutiny is a "significant indication that it is taken seriously by other scientists." Daubert II, 43 F.3d at 1318 (citing Daubert 509 U.S. at 593, 113 S.Ct. at 2797 (declaring scrutiny of the proposed evidence by the scientific community to be a component of good science)).
In this case, as in Daubert II, the scientific conclusion advanced by plaintiff's expert witnesses, viz. exposure to chlorine gas results in various neuropathological conditions, remains unsupported by any physician other than the doctors plaintiff expects to call at trial. Daubert II, 43 F.3d at 1314. On the other hand, unlike the proposed evidence in Daubert II, there is in the case sub judice at least one published article arguably supporting the conclusions reached by the plaintiff's proposed expert witnesses. In Daubert II, the only "independent" support for the proposed evidence was testimony in other lawsuit by the same witnesses. 43 F.3d at 1319. In Daubert II none of the plaintiff's experts had published their findings in a scientific journal or solicited formal review by their colleagues. Id. at 1318. In fact, the Daubert II experts offered in support of their research only their professional credentials, their conclusions, and self-serving assurances of the reliability of their findings. Id. at 1319.
The author of that article is one of plaintiffs' proposed expert witnesses. Or, to put it another way, one of the physicians who examined the plaintiffs in this case published the results of his examination of the plaintiffs in a medical journal. Kaye H. Kilburn, Evidence that Inhaled Chlorine is Neurotoxic and Causes Airways Obstructions, 4 Int'l J. Occupational Med. Tox. 257 (1995). Defendant Pioneer, however, implies that this particular journal does not deserve the respect usually accorded professional scientific publications. An affidavit submitted by Pioneer's own expert, Barry Gordon. M.D. indicates that the journal which published Dr. Kilburn's article does not appear among the more than three thousand medical periodicals listed in the Index Medicus of the National Library of Medicine and is not found on the shelves of the library at Johns Hopkins University School of Medicine. Nor was Dr. Gordon able to locate the International Journal of Occupational Medicine in Current Comments, which he describes as "another standard source for titles and abstracts in science." Gordon Affidavit, appended as Ex. 1 to Reply in Support of Defendant's Motion in Limine (Doc. # 138 in Case No. 92-887). While not dispositive of the question of admissibility of Dr. Kilburn's expert testimony, the apparent obscurity of the journal relied on by plaintiffs to legitimate Dr. Kilburn's scientific conclusions does weigh in the court's evidentiary calculus. Dr. Kilburn's supplemental affidavit submitted as evidence that his article was in fact indexed in Current Comments is based on hearsay, and therefore must be discounted. Kilburn Affidavit (Doc. # 153 in Case No. 92-887).
The Ninth Circuit saw the fact that the proposed expert witnesses had been "unable" or "unwilling" to publish their research as undermining the plaintiff's claim that that research was grounded in proper scientific methodology. Id. (quoting Daubert 509 U.S. at 591, 592, 113 S.Ct. at 2796, 2796).
Even proof that the proposed scientific findings predate the litigation, or that those findings have undergone some form of "peer review" is not conclusive; at most such proof constitutes a "prima facie case" for admissibility. The party opposing admission may still offer proof that the proposed scientific findings lack a sound methodological basis. Daubert II, 43 F.3d at 1318-19 n. 10.
II. Plaintiffs' Scientific Evidence
The plaintiffs in the present case claim that their exposure to chlorine gas as a result of an accidental discharge of the gas from the defendant's facility at Henderson, Nevada in May 1991 has caused them to develop encephalopathy (brain damage), polyneuropathy (multiple nerve cell damage), and "organic depression." In support of that claim, plaintiffs propose to offer testimony by four expert witnesses, Gunnar Heuser, M.D., William Spindell, Ph.D., Kaye H. Kilburn, M.D., and Alan Hirsch, M.D. Defendant Pioneer Chlor Company has moved to exclude the testimony of all four experts. Defendant's Motions in Limine (Doc. # 125 in Case No. 92-0887; Doc. # 122 in Case No. 93-0475).
The question of admissibility of expert testimony by each of these witnesses must be addressed separately.
A. Dr. Heuser
Dr. Heuser, who has not submitted affidavits describing the basis for his proposed testimony, was deposed by defense counsel. In his deposition testimony Dr. Heuser stated that he had examined plaintiffs Butenschoen and Valentine. He stated that he found what he considered "striking abnormalities" in Valentine's immune system, but when asked about the cause of those observed immune system abnormalities Dr. Heuser refused to opine that the cause was chlorine exposure, but only offered his opinion that "this is what you typically see after chemical exposure."
Dr. Heuser, a native of Germany, earned his medical degree from the University of Cologne. He holds a diploma in internal medicine from McGill University and currently holds the title of Assistant Clinical Professor of Medicine at the University of California at Los Angeles. He is the author of several dozen articles and books, mainly in the fields of neurology and endocrinology. Heuser Curriculum Vitae, appended as Heuser Ex. 1 to Plaintiff's Opposition to Motion in Limine (Doc. # 133 in Case No. 92-887).
Heuser Deposition at 56, lines 21-22, appended as Ex. A to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
When questioned about plaintiff Butenschoen, Dr. Heuser was even less confident. In support of his opinion that Butenschoen suffered neurological damage as a result of oxygen deprivation brought on by chlorine inhalation, Dr. Heuser stated that inhaling chlorine might cause oxygen deprivation and might interfere with the transfer of oxygen from the bronchial mucosa to the bloodstream. But when asked how such an effect could be tested, Dr. Heuser said, "I don't think there's a test now that you could do. . . ."
Heuser Deposition, at 62 lines 18-19, appended as Ex. 6 to Defendant's Reply in Support of Motion in Limine (Doc. # 138 in Case No. 92-887).
The proponent of expert scientific evidence shoulders the burden of proving that her evidence meets the standard of Evidence Rule 702. Dr. Heuser's opinions do not meet that standard.
B. Dr. Spindell
Dr. Spindell concluded from his examination of plaintiffs Valentine and Butenschoen that they both exhibited significant cognitive and emotional deficits. But Dr. Spindell admitted that he had made no efforts to determine the cause of those deficits, or to rule out possible etiologies other than chlorine inhalation. Dr. Spindell also acknowledged that he had not reviewed any pre-accident evaluations of plaintiffs' cognitive or emotional capacities. The fact that their measured cognitive and emotional capacities appeared subnormal is certainly no evidence that those deficits resulted from any particular trauma.
The curriculum vitae of Dr. Spindell, who is not a medical doctor, does not appear of record.
Spindell Deposition at 46 lines 9-10, appended as Ex. D to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
Id. at 46 lines 10-23.
Id. at 62 lines 22-24; 66 lines 10-20.
Dr. Spindell stated in his deposition that he was not aware of any research indicating that exposure to atmospheric chlorine could cause cognitive impairment. Thus his opinion that Valentine's abnormalities "could have occurred as a result of the toxic event" is entitled to little, if any, weight, and does not rise to the level of "scientific knowledge" within the meaning of Daubert and Evidence Rule 702. Daubert, 509 U.S. at 589-90, 113 S.Ct. at 2795 (declaring expert scientific testimony grounded only in speculation or conjecture to be inadmissible under Rule 702).
Id. at 65 lines 12-18.
Spindell Report of Neuropsychological Evaluation: Brian Valentine, at 4, appended as Ex. E to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
C. Dr. Hirsch
Dr. Hirsch took a standard medical history from plaintiff Valentine, and reviewed reports of examinations of Mr. Valentine prepared by several other doctors. He reviewed the deposition of Dr. Kilburn as to Kilburn's evaluation of plaintiffs Valentine and Butenschoen. Dr. Hirsch also conducted a physical examination of plaintiff Valentine.
Dr. Hirsch holds diplomas in psychiatry, geriatric psychiatry and neurology from the American Board of Psychiatry and Neurology. Dr. Hirsch is a member of the teaching faculties of the Mercy Hospital and Medical Center and Cook County Hospital in Chicago. He holds the title of Assistant Professor in the Departments of Neurology and Psychiatry at St. Luke's Medical Center, also in Chicago. He has authored or co-authored more than one hundred articles, mainly in the field of olfactology. Hirsch Curriculum Vitae, appended as Hirsch Ex. 1 to Plaintiff's Opposition to Motion in Limine (Doc. # 133 in Case No. 92-887).
Dr. Hirsch concluded from the results of his examination and from his review of the reports of the other doctors that Valentine was suffering from headaches as a result of his inhalation of chlorine, and he further concluded that Valentine had developed a hypersensitivity to odors, a condition known as hyperosmia. Dr. Hirsch also diagnosed plaintiff Valentine with trigeminal nerve damage. The trigeminal nerve is the fifth and largest pair of the human cranial nerves. Although Dr. Hirsch stated at his deposition that the medical literature contains extensive reports of trigeminal neuropathy resulting from exposure to trichlorethylene, he was unable to identify any studies ascribing trigeminal neuropathy to chlorine exposure.
Dorland's Illustrated Medical Dictionary 1744 (28th ed. 1994).
Hirsch Deposition at 111 lines 8-25, appended as Ex. F to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
Dr. Hirsch observed adverse neurologic effects in the five Henderson chlorine victims whom he treated, including hyperosmia and prodromata cacogeusia (precursive symptoms of a condition characterized by a bad taste in the mouth unrelated to the ingestion of any particular substance). Dr. Hirsch performed a number of standard neuropsychological tests on Brian Valentine and concluded that Valentine was suffering from chlorine-induced encephalopathy (brain damage), chlorine-induced cephalgia (headaches) and chlorine-induced polyneuropathy (multiple damage to peripheral nerves).
Dorland's Illustrated Medical Dictionary 244, 1358 (28th ed. 1994).
Dr. Hirsch was asked at his deposition to explain why he concluded that Valentine's neurological complaints, particularly trigeminal neuropathy, resulted from exposure to chlorine. The doctor did not cite any particular studies leading him to those conclusions, but instead referred generally to research into the toxic effects of exposure to solvents. Although chlorine is not a solvent, Dr. Hirsch apparently analogized to research in solvent toxicology on the ground that both solvents and chlorine cause a demyelination of the central nervous system. Demyelination is the destruction of the myelin sheath which protects the nerve cells and enables them to function. Loss of the myelin sheath impairs neurologic function, and can lead to severe pain, and, in extreme cases, to death.
Dr. Hirsch testified at his deposition that he knew of medical literature suggesting the clinical observation of chlorine-induced neuropathology. He further opined on the basis of his physical examination of plaintiff Valentine that Valentine's cephalgia was in fact a result of chlorine inhalation. But Dr. Hirsch has not published those conclusions in any recognized medical journal nor conducted any unpublished pre-litigation research into the neuropathological effects of chlorine inhalation. His testimony thus fails to conform to either of the two principal methods of satisfying Evidence Rule 702. Daubert II, 43 F.3d at 1318.
Hirsch Deposition at 71 lines 10-10, appended as Hirsch Ex. 4 to Plaintiff's Opposition to Motion in Limine (Doc. # 133 in Case No. 92-887).
Nevertheless his testimony may be admissible if he can "explain precisely how [he] reached [his] conclusions and point to some objective source — a learned treatise, the policy statement of a professional organization, a published article in a reputable scientific journal — to show that [he has] followed the scientific method, as it is practiced by (at least) a recognized minority of scientists in [his] field." Daubert II, 43 F.3d at 1319 (citing United States v. Rincon, 28 F.3d 921, 924 (9th Cir. 1994).
Dr. Hirsch has not as yet done so. Although his failure to provide such proof subjects his testimony to exclusion, the it is the judgment of the court that the narrowness of the gap between the supporting evidence he has provided and what Daubert requires weighs in favor of a ruling permitting Dr. Hirsch to attempt to bring his proposed testimony within the strictures of the evidence rules.
D. Dr. Kilburn
Dr. Kilburn is a medical doctor, board certified in internal medicine and preventive medicine and occupational health. Dr. Kilburn has concluded on the basis of various tests designed to evaluate neurological health that plaintiff Valentine suffers from damage to the trigeminal nerve as a result of exposure to chlorine gas.
Dr. Kilburn's credentials are impressive. He is Ralph Edgington Professor of Medicine at the University of Southern California School of Medicine. He has taught in the medical schools of the University of Colorado, Washington University, Duke University and the University of Missouri. He has been a medical doctor for more than forty years, has served on numerous national medical advisory committees, directed a dozen national and international medical symposia, is a member of the editorial boards of several medical journals, and has himself authored or co-authored close to two hundred papers, the majority of which have been published in the medical journals. Kilburn Ex. 1, appended to Plaintiff's Opposition to Defendant's Motion in Limine (Doc. # 133 in Case No. 92-887).
Chlorine gas is without question highly toxic, indeed it is lethal. Chlorine is the active ingredient in the "mustard gas" used with deadly effect on the battlefields of Europe beginning in 1915. Its disastrous effect on the human pulmonary system are not reasonably capable of question. But plaintiffs' experts have proposed to testify not to the effect of the gas on human lungs, but on the human brain and central nervous system. The assertions of plaintiffs' experts that exposure to chlorine damages the brain and nervous system are novel, and, as noted above, unsupported by scientific research extraneous to this litigation.
Webster's New International Dictionary 1616 (2d ed. 1957); see generally Frank B. Underhill, The Lethal War Gases, Physiology and Experimental Treatment (1920).
Dr. Kilburn does not claim to base his diagnosis upon any independent medical research. His conclusions are based almost entirely upon his examination of the plaintiffs in this case. The admissibility of his testimony to the neuropathological effects of chlorine exposure, then, depends on the validity of his own research conducted during the course of the instant litigation.
Dr. Kilburn admits that he knows of no research — other than his own, of course — on the question whether exposure to chlorine gas causes CNS damage. Transcript of Deposition of Kaye H. Kilburn M.D., at 70, appended as Ex. B to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
Plaintiffs vaunt the fact of publication in 1995 of the article by Dr. Kilburn in the International Journal of Occupational Medicine and Toxicology entitled "Evidence that Inhaled Chlorine is Neurotoxic and Causes Airways Obstruction." The article purports to find an association between the exposure of seven individuals to chlorine and their subsequent presentation with various symptomatic neuropathologies.
Association is defined as the degree of statistical dependence between two or more events or variables, and exists when those events or conditions coincide more frequently than one would expect as a result of pure chance. Association in statistical analysis must not be confused with causation in law, although a strong association commonly gives rise to an inference of causation. Events or conditions are not associated if the presence of the subject agent (the so-called dependent variable) has no effect on the incidence of a disease (the independent variable) — the null hypothesis. The strength of an association, for example, between exposure to a given toxin and the incidence of a given disease, is indicated by measuring the relative risk of disease incidence between unexposed and exposed subjects. Relative risk is the ratio of incidence in the exposed cohort to the incidence in the unexposed or control cohort. Relative risk is commonly expressed by the formul
Ie RR = ____ Ic
where Ie is the incidence of the target disease in the exposed cohort and Ic is the incidence of the target disease in the control cohort.
In order to account for the background rate of incidence in the general population, however, that background rate must first be subtracted from the aggregate rate of incidence so that the ratio of relative risk describes only the increased risk specifically attributable to the target agent, and does not include the aggregate risk from all possible factors. By discounting the incidence attributable to other factors, it is possible to arrive at the rate of the increased rate of increased risk attributable to the target agent alone. The attributable proportion of relative risk is expressed as:
Ic — Ic APR = _________ IcSee Reference Manual on Scientific Evidence 147-50, 172-73 (1994).
Taking the Daubert II court at its word, the very fact of the article's publication constitutes a significant indication that the author's colleagues have taken his work seriously. There seems, however, to be some serious debate in the scientific community itself over the significance of publication per se and the adequacy of pre-publication evaluation of scientific writings. One aspect of this debate concerns the distinction, unfortunately elided by both the U.S. Supreme Court in Daubert and by the Ninth Circuit in Daubert II (and by both parties and their expert witnesses in this case), between "editorial" peer review and "true" peer review.
Concern about the reliability of scientific publications, especially in the life sciences, has prompted broad reevaluation of pre-publication review. See, e.g., D. Rennie, Editorial Peer Review: Let us Put it on Trial, 13 Controlled Clin. Trials 443 (1992); M. Ruderfer, The Fallacy of Peer Review — Judgment Without Science and a Case History, 3 Speculations Sci. Technol. 533 (1980); M. Sun, Peer Review Comes Under Peer Review, 244 Science 910 (1989); L.K. Altman, Errors Prompt Proposals to Improve Peer Review, N.Y. Times June 6, 1989, at C3. The American Medical Association responded to those concerns when in May 1988 it sponsored the First International Congress on Peer Review in Washington, D.C. See Arnold S. Relman Marcia Angell, How Good is Peer Review?, 321 New Eng.J.Med. 827 (1989).
In the broadest sense of the phrase, "peer review" is the process by which scientific claims are evaluated by members of the relevant discipline. The "scientific method" venerated by lawyers and judges consists in great measure of the allegiance of those we call "scientists" to the principle that only those results capable of replication through independent investigation can justify acceptance as scientific "truth." That principle is Galileo's great gift to human inquiry into the nature of the universe; since the seventeenth century doctors, astronomers, chemists, physicists and biologists have been educated in the tradition which teaches us to question authoritative pronouncements about what the world is really like. As Einstein observed, "knowledge" is not truth, but only mindless agreement. Only by continuing to examine all possible explanations for a given phenomenon does one approach the truth.
Einstein's message is echoed in the writings of Yale physicist Robert K. Adair, who declared that science "advances by a winnowing and sifting process . . . through which sound hypotheses become accepted and incorrect data and ideas are forgotten." Kenneth R. Foster et al., eds., Phantom Risk: Scientific Inference and the Law 19((1993).
In this sense "peer review," or, in the nomenclature of one legal commentator, "true peer review," is the essence of science. One investigator makes her methods and findings public, so that others can attack or support her conclusion by following the same protocols while asking what besides the stated principle could account for the documented results.
Effie J. Chan, Note, The "Brave New World" of Daubert: True Peer Review, Editorial Peer Review, and Scientific Validity, 70 N.Y.U.L.Rev. 100 (1995).
By contrast, the pre-publication "peer review" of scientific writings is of much narrower scope. Obviously journal referees cannot afford to duplicate the efforts of the scientist whose manuscript they are asked to evaluate. In this context "peer review," or, to adopt Effie Chan's shorthand, "editorial peer review" consists of the reference by a journal editor of an article submitted for publication to two or more outside reviewers. The reviewers make confidential comments on the article's scientific accuracy, style, originality and importance, and make a recommendation to the journal that the article be accepted or rejected. Militating against forensic use of editorial peer review as a proxy for genuine critical examination of purported scientific evidence is the fact that the average referee spends less than two hours assessing an article submitted to a biomedical journal.
Id.
Arnold S. Relman Marcia Angell, The Journal's Peer-Review Process, 221 J.Am.Med.Ass'n 837 (1989).
Id.
S. Lock J. Smith, What Do Peer Reviewers Do?, 263 J.Am.Med.Ass'n 1341, 1341-43 (1990).
Because the scope of editorial peer review is necessarily narrower than true peer review, it is a serious error to conflate the two processes, and, by extension, to assume that because an article is accepted for publication, even in a prestigious scientific journal, that the science it contains is therefore valid. Editorial peer review "is not and cannot be an objective scientific process, nor can it be relied upon to guarantee the validity or honesty of scientific research, despite much uninformed opinion to the contrary." This is especially true at journals below the very first rank, where editors and referees may not even have checked the manuscript under submission for statistical accuracy.
Also, because the acceptance rates at medical journals is often very low, the appearance of an article in a journal may obscure the rather damaging possibility that the article may have previously been rejected by as many as ten or twenty other journals. Chan, supra note 24, at 122.
Relman Angell, How Good is Peer Review?, supra note 22.
Chan, supra note 24, at 134 n. 103.
For these reasons, this court's obligation under Daubert to insure that proposed expert scientific evidence be grounded in good science requires it to look behind the fact that Dr. Kilburn published the results of his research into the putative neuropathological effects of chlorine inhalation, and to examine for itself the method Dr. Kilburn employed in reaching his conclusions. Claar v. Burlington N.R.R. Co., 29 F.3d 499, 501 (9th Cir. 1994) (recognizing obligation of district court to scrutinize carefully the reasoning and methodology underlying proffered expert scientific evidence).
Indeed, there is before the court at present only Dr. Kilburn's word that the article was subjected to pre-publication editorial peer review. Kilburn Affidavit, at 24, appended to Plaintiff's Opposition to Motion in Limine (Doc. #133 in Case No. 92-887).
If the research and analysis described in the article comports with established scientific investigatory principles, Dr. Kilburn ought to be permitted to testify. If not, his testimony must be excluded.
Dr. Kilburn's investigation may best be described as a hybrid of traditional "cohort" and "case-control" models. He examined seven individuals who were accidentally exposed to chlorine gas and compared their responses to questionnaires and to clinical tests with the responses of a control group, recruited from another community which had not been exposed to atmospheric chlorine and selected to match the age, sex and educational attainment of the exposed subjects.
In an epidemiological cohort study, the investigator divides members of the subject population into "exposed" and "not exposed" members. She then examines the individuals in both groups for the target disease. In a case-control study, all the members of the study group have already been diagnosed with the target disease, and all the controls have been found not to suffer from the disease. The investigator then determines which members of the subject and control groups were exposed to the target agent. Reference Manual on Scientific Evidence 135-38 (1994). In Dr. Kilburn's study, the subjects were selected on the basis of both past exposure to atmospheric chlorine, and presentation with symptoms of neurological disorders. The results of those examinations were then compared with the results of the same tests performed on the "not exposed" controls.
The questionnaires were designed to elicit data regarding histories of respiratory illness, neurological disorders, head trauma, past exposure to environmental toxins, use of tobacco, alcohol, and drugs, and general medical history. The subjects were clinically tested for body balance, reaction time, blink reflex, color discrimination, visual field range, hearing acuity, extremities vibration, recall, nonverbal nonarithmetic intelligence, constructional, interpretive and integrative capacity, attention span, dexterity, coordination, decision-making, peripheral sensation and discrimination, vocabulary, mood states, spirometry and expiration.
Dr. Kilburn found that five of the seven exposed subjects suffered from impaired verbal recall, three suffered from impaired visual recall, and two suffered from cognitive defects. He found that all the exposed subjects displayed elevated adverse moods. Despite the unavailability of chlorine dosage data on the exposed subjects, Dr. Kilburn nevertheless concluded that "the severity of impairments suggests `high' exposures" to atmospheric chlorine." He wrote, "[t]he impairments are attributed to chlorine because there is no other explanation for the symptoms common" to the exposed subjects.
Dr. Kilburn's methodology appears to have ignored a number of important issues. In any epidemiological or toxicological study, the size of the sample populations studied is crucial. The entire community of Henderson, Nevada was exposed to chlorine gas. Defendant Pioneer Chlor Alkali's estimate that several thousand Henderson residents were exposed in the May 1991 accident does not appear overly generous. Dr. Kilburn acknowledged in his deposition that "in order to make real conclusions about what is the average effect of chlorine or ammonia or trichlorethylene, one really has to study a crowd of people, on a population [sic], not two or three individuals." Yet Kilburn studied only seven people, all of whom, incidentally, were, at least at one point, involved in the litigation arising out of the accident. In the words of Kenneth Foster, "[a] theory constructed by means of special pleading and on the basis of a few observations selected from a multitude is not satisfactory."
Defendant's Reply in Support of Motion in Limine at 10 n. 9 (Doc. # 138 in Case No. 92-887).
Kilburn Deposition, at 94 lines 5-9, appended as Ex. B to Defendant's Motion in Limine (Doc. # 125 in Case No. 92-887).
Foster et al., supra note 23, at 19.
The probability of selection bias is too high to be overlooked. Dr. Kilburn did not select the members of the exposed group at random; they are described in his article as "patients referred to an environmental clinic specializing in neurotoxicology." This method of selection is unacceptable because the study group has self-selected for disease: Residents of Henderson who were exposed to chlorine but escaped injury would obviously not be "referred" to a health facility. Typically the criteria employed in selecting individuals for a study group are clearly stated and well documented in order to avoid bias. Dr. Kilburn's study violates this practice.
Reference Manual on Scientific Evidence 138 (1994).
The small size of the study group increases the risk that the difference between the rate at which chlorine-exposed people develop the claimed neurological disorders and the background rate for those same disorders in the general population may be too small to be statistically significant. Nowhere in Dr. Kilburn's article does he discuss the background rate of neurological disorders. Omission of the background rate from the experimental calculus creates a classic risk of type I or alpha error: If the background rate of neurological disorders is high, and the measured degree of neurological impairment is low, only the examination of a very large sample will serve to guard against a fatal level of type I error. Otherwise the risk of false positive results renders the investigator's rejection of the null hypothesis suspect.
There is no indication that Dr. Kilburn even attempted to eliminate the possibility that the seven exposed subjects might also have been exposed to another neuropathogenic agent unrelated to atmospheric chlorine. Before leaping to the conclusion that exposure to the target agent is associated with subsequent neuropathological conditions, an investigator must try to identify factors other than exposure that might account for the elevated risk of the target disease, in this case, neurological disorders. See Reference Manual on Scientific Evidence 135-36.
Dr. Kilburn's statement that he "has seen no evidence" that plaintiffs "had any of these neurotoxic abnormalities prior to the incident" is no substitute for the requisite affirmative proof that their alleged abnormalities did not predate their exposure to chlorine. In order for Dr. Kilburn's assertion that these alleged abnormalities result from that exposure to carry any weight, it is essential that he provide evidence that the alleged conditions did not exist prior to chlorine exposure. Otherwise his conclusions regarding causation rest on the enormous assumption that the conditions arose after exposure.
The difficulty of determining pre-exposure physiological and neuropsychological parameters inherent in so-called retrospective occupational and environmental epidemiological studies is the primary reason such studies are considered less accurate than prospective or follow-up studies. Where pre-exposure medical data is collected by interview, "quality control procedures should probe the reliability of the individual and whether the information is verified by other sources." Reference Manual on Scientific Evidence 144-45 (1994).
See Peter Arlien-Soborg, Solvent Neurotoxicity (1991) (cited by both parties in this case). Arlien-Soborg, like Dr. Kilburn, employed cohort studies to test the effect of exposure to toxins. He recognized, however, that although "neurophysiological methods may be valuable in detecting group differences between exposed workers and their controls" that the changes are "mild and nonspecific and do not help identify their etiology." Id. at 16. Moreover, "neuropsychological functions are of a kind having no clear delimitation between normal and abnormal, and normal variability is considerable. . . . In other words, no individual should be considered impaired because of not suiting group norms, but only if a change from preexisting functional level can be registered." Id. at 40. The total absence from Dr. Kilburn's study of good idiographic, or individualized diagnostic methods by which to compare "individual actual performance with prior functional parameters to reveal changes of diagnostic significance," ibid., therefore casts grave doubt on his conclusions. Just as serious is the suggestion raised by Dr. Kilburn's admission in his deposition that the ostensible normative data from the control group, which, according to Dr. Kilburn was selected from a known unexposed community to match the age, sex and educational attainment of the exposed group was not actually utilized in the study despite statements in the published study to the contrary. Compare Dr. Kilburn's published claim, see Kilburn, supra note 2, at 269, with his sworn admission, see Kilburn Affidavit at 9 lines 8-11, appended as Ex. 3 to Reply in Support of Defendant's Motion in Limine at 10 (Doc. # 138 in Case No. 92-887).
Also missing from Dr. Kilburn's study is any standardized or otherwise reliable method of measuring the degree of plaintiffs' exposure. Typically when the investigation takes place long after the exposure, subjects are tested for the presence of biological markers indicating the relative extent of exposure. Often, however, no such marker is available. Dr. Kilburn acknowledges in his article that no dosage data was available on his seven patients. In addition, no attempt to isolate the potential effect of recall bias appears in Dr. Kilburn's analysis. Often exposed subjects associate irrelevant symptoms with their history of exposure to the target agent.
Kilburn, supra note 2, at 271.
In summary, Dr. Kilburn's study suffers from very serious flaws. He took no steps to eliminate selection bias in the study group, he failed to identify the background rate for the observed disorders in the Henderson community, he failed to control for potential recall bias, he simply ignored the lack of reliable dosage data, he chose a tiny sample size, and he did not attempt to eliminate so-called confounding factors which might have been responsible for the incidence of neurological disorders in the subject group. As a result, his conclusions that the plaintiffs exposure to atmospheric chlorine caused their neurological disorders cannot be said to be derived from acceptable scientific methodology.
Conclusion
For the foregoing reasons, IT IS THEREFORE ORDERED that Defendant's Motions in Limine (Doc. # 125 in Case No. 92-0887; Doc. # 122 in Case No. 93-0475) are HEREBY GRANTED to the following extent: The proposed testimony of Drs. Heuser, Spindell and Kilburn is HEREBY EXCLUDED. IT IS FURTHER ORDERED that Defendant's Motions in Limine (Doc. # 125 in Case No. 92-0887; Doc. # 122 in Case No. 93-0475) are HEREBY DENIED with respect to Dr. Hirsch.
IT IS FURTHER HEREBY ORDERED that Dr. Hirsch shall no later than 4:00 p.m. on april 8, 1996 file with the court and serve upon opposing counsel an affidavit or declaration describing in detail the medical literature to which he referred in his deposition as support for his conclusions.
IT IS FURTHER HEREBY ORDERED that Defendant shall no later than 4:00 p.m. on April 15, 1996 file with the court and serve upon opposing counsel any counter-affidavits, counter-declarations or other opposing material. There shall be no other or further filings by either side with respect to this matter.
IT IS FURTHER HEREBY ORDERED that the parties or their witnesses shall file such affidavits, declarations or other material relevant to the court's ruling on the motions in limine with the Clerk of the United States District Court in Las Vegas, Nevada.
IT IS FURTHER HEREBY ORDERED that the parties or their witnesses shall, contemporaneously with the filing of such documents with the Clerk of the court in Las Vegas, transmit copies of such documents by electronic facsimile to the chambers of the undersigned District Judge in Reno, Nevada, at (702) 784-5342.
The court does not anticipate that any extensions of time to comply with the provisions of this Order shall be granted in the absence of extraordinary circumstances.