Opinion
Case No. 3:13-cr-00764-WHO-1
08-15-2017
UNITED STATES OF AMERICA, Plaintiff, v. ALFONZO WILLIAMS, et al. Defendants.
ORDER ON ADMISSIBILITY OF DNA EVIDENCE
Re: Dkt. Nos. 638, 639, 648
INTRODUCTION
Certain forensic methods of DNA analysis are widely accepted throughout the scientific community and legal system. Others are not. Methods involving mixed samples, limited amounts of DNA, and suspect-driven statistical interpretations are undergoing scrutiny and continued refinement. Laboratories across the country continue to make changes to their protocols based on new developments. As a result, the reliability of evidence analyzed using older methods has been called into question.
In this eleven defendant RICO prosecution of alleged members of the Central Divisadero Players ("CDP"), defendants Adrian Gordon, Esau Ferdinand, and Charles Heard are charged with counts of attempted murder and murder. The government intends to introduce against them expert reports from two laboratories that conducted DNA testing of evidence taken from the crime scenes. Gordon, Ferdinand, and Heard seek to prevent the government's DNA experts from testifying because the DNA evidence lacks a reliable basis under Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993).
I conducted a two-day Daubert hearing during which I heard testimony from the labs' analysts from the Contra Costa Crime Lab for Gordon and the Serological Research Institute ("SERI") for Ferdinand and Heard, as well as the defendants' DNA expert. Both labs were accredited and many of the issues involved in analyzing the DNA evidence were similar: there were small quantities of DNA in the samples, so each had to be enhanced to increase the sensitivity of the data, and each was a complex mixture involving at least two individuals. The entire amount of the samples were consumed in the analyses, so the defendants did not have the opportunity to perform their own evaluations. Despite those similarities, there were substantial differences in the way in which the analysts reached their opinions. I conclude that the results generated by the Contra Costa Crime Lab are sufficiently reliable to be admissible because the analyst followed lab protocols and the assumptions she made can be tested in cross-examination. That is not the case for SERI's results; its methodology was unsound and its conclusions are untestable because of the way SERI performed its analysis. For the reasons stated below, Gordon's motion to exclude is DENIED, and Heard's and Ferdinand's motions are GRANTED.
BACKGROUND
There has been extensive briefing surrounding the admissibility of DNA evidence against these defendants. See Defendant Adrian Gordon's Motion/Memorandum To Exclude DNA Test Results And Request For Daubert Hearing (Dkt. No. 639) and the related Exhibits (Dkt. No. 639-1); Defendant Charles Heard's Joinder In Codefendant Adrian Gordon's Motion To Exclude DNA Test Results And Request For A Daubert Hearing (Dkt. No. 638); Defendant Esau Ferdinand's Motion to Exclude Evidence of DNA Testing Performed by Serological Research Institute and Request for Daubert Hearing and Memorandum of Points and Authorities (Dkt. Nos. 647 & 648); Defendant Charles Heard's Joinder in Codefendants' Motions (including Mr. Gordon's and Mr. Ferdinand's DNA/Daubert motions) (Dkt. No. 683); Defendant Charles Heard's Supplemental Memorandum re: DNA Daubert Hearing (Dkt. No. 1172); Defendant Esau Ferdinand's Supplemental Memo re DNA Daubert Hearing (Dkt. No. 1175); Defendant Adrian Gordon's Post-Hearing Memorandum Regarding DNA Daubert Hearing (Dkt. No. 1204); Defendants Charles Heard's and Esau Ferdinand's Post-Hearing Memorandum re: DNA Daubert Hearing (Dkt. No. 1206); United States' Supplemental Brief in Opposition to Motion to Exclude DNA Evidence (Dkt. No. 1208); Supplemental Reply Memorandum re: DNA Daubert Hearing, Submitted by Defendants Charles Heard, Adrian Gordon and Esau Ferdinand (Dkt. No. 1214); and United States' Supplemental Post-Hearing Response to Motion to Exclude DNA Evidence (Dkt. No. 1215).
FACTUAL BACKGROUND
A. Brief Synopsis of DNA Forensic Testing
This background is intended to provide context to the discussion. The information comes from exhibits attached to Gordon's original motion. See Jayne Decl. ¶¶ 2-9 (Dkt. No. 639-2); id. at Exs. A-H (Dkt. No. 639-1).
To aid in forensic identification, samples of deoxyribonucleic acid (DNA) are analyzed by focusing on variations in sequence and length at a specific locus or genetic marker. Each locus has two alleles (one inherited from each parent), and each allele, identified by a number, can be the same or different. A locus with more than two alleles present indicates a mixed sample with at least two contributors. The process of polymerase chain reaction (PCR) enables labs to take a small amount of DNA and amplify it to detectable limits. Generally, the testing is composed of five phases: extraction, quantification, amplification, electrophoresis, and interpretation. During the amplification phase, labs may utilize enhanced detection techniques to increase the sensitivity of data. These processes may lead to certain artifacts deemed "stochastic effects," such as peak height imbalance, allelic drop out, allelic drop in and high stutter. These stochastic effects result from the randomness imparted by amplifying and enhancing small amounts of DNA.
In short tandem repeat (STR) analysis, the number represents the number of repeats in a given allele.
During quantification, a lab attempts to quantify the amount of DNA contained in a sample. Low Copy Number (LCN) or Low template (LT) DNA testing refers to the testing of very small amounts of DNA. E.g., Dkt. 639-1 at 6. Although a precise definition does not exist, the scientific community has referred to DNA samples at or below 100-200 picograms of DNA, where the resulting peak height of alleles may indicate unreliable results. Id. But assigning a concentration level to the definition is problematic for complex (i.e., mixed) samples because it is unknown how much of the DNA material can be associated to any individual contributor. For this reason, the Scientific Working Group on DNA Analysis Methods ("SWGDAM") established "SWGDAM Guidelines for STR Enhanced Detection Methods," which define low template or low copy DNA analysis as "a subset of Enhanced Detection Methods that, in addition to the increased potential for stochastic effects, have an increased potential for non-reproducible alleles." See generally Dkt. No. 639-1 at 147.
A lab uses a test amplification kit to analyze DNA samples. For each kit, a lab assigns an analytical threshold, below which alleles cannot be distinguished from background noise, and a stochastic threshold, below which allelic dropout may have occurred. The potential for allelic dropout is one of the random effects inherent to disproportionate amplification. The analytical results are produced in a graph called an electropherogram, which, in a single-source sample, shows two peaks if the two alleles are different (heterozygous), or one larger peak if the two alleles are the same (homozygous). The allele repeat number and the height of the peak are listed for each peak. In PCR/STR testing, several STR loci are analyzed to create a genetic profile for any relevant persons. Analysts then perform statistical interpretations to compare genetic profiles from evidentiary samples to known or reference profiles and determine whether the latter are "included" or "excluded" as possible contributors to the former.
According to SWGDAM, the stochastic threshold is defined as "the peak height value above which it is reasonable to assume that, at a given locus, allelic drop-out of a sister allele has not occurred." SWGDAM Interpretation Guidelines for Autosomal STR Typing by Forensic DNA Testing Laboratories, 2010 at 28.
The "peaks" represent a measurement of relative fluorescence units (RFU).
In the analyses at issue in this case, the entire amount of DNA recovered at the crime scenes was consumed, so the defendants could not perform independent evaluations.
B. Contra Costa Crime Lab and Defendant Adrian Gordon
Gordon is charged with attempting to kill Victim 3 on May 20, 2011 in San Pablo, California. Second Superseding Indictment (Dkt. No. 139). Helen Kim of the Contra Costa County Crime Lab analyzed the samples that the government seeks to use in its case against Gordon. The lab is accredited by ASCLD/Lab International and conforms to the FBI's Quality Assurance Standards (QAS). 5/16/17 Hr'g Tr. at 22-26 (Dkt. No. 1185). Kim used the AMPFISTRTM Identifiler to test swabs from a firearm—Item 4-1 (swabbing from right and left side hand grip), Item 4-2 (swabbing from trigger), and Item 4-3 (swabbing from right side of top slide). The lab amplified 0.19 nanograms (ng) of Item 4-1, 1.53 ng of Item 4-2 and 0.20 ng of Item 4-3A and consumed the entire extraction for each sample. Based on the small quantities of DNA in the samples, the lab used two enhancement detection techniques to increase the sensitivity of the data—a "super juice" on each of the samples, and an increased injection time for sample 4-2. 5/16/17 Hr'g Tr. at 85 (Dkt. No. 1185).
For comparison to estimates of what constitutes LCN or LT testing, 1 nanogram = 1,000 picograms. See supra note 4.
Kim developed the following conclusions:
• Item 4-2: A degraded mixture profile consisting of at least three individuals, including at least one male was obtained from the trigger. There is a clear partial major female component (designated female A), minor component, and a trace component present. No further conclusion can be made regarding the presence of the minor and trace components due to insufficient data.10/28/11 Report of Laboratory Examination (5/16/17 Hr'g Def. Ex. B, BG077225). When asked to conduct DNA STR typing on a reference buccal sample collected from Gordon, Kim concluded,
• Item 4-3A: A degraded mixture profile consisting of at least two individuals, including at least one male was obtained from the right side of the top slide. The female A profile obtained from the trigger appears to be represented in this mixture profile. Assuming that there are two contributors and that female A is one of the contributors, a partial male profile can be deduced (designated male A). The male A profile will be entered into CODIS.
• Item 4-1: A degraded mixture profile consisting of at least three individuals, including at least one male was obtained from the hand grip. There is a major component and a trace component present. Female A and male A profiles appear to be represented in the major component. No further conclusion can be made regarding the presence of the trace component due to insufficient data.
DNA testing revealed Gordon is included as a potential contributor to the partial male profile deduced from the mixture profile (designated male A) developed on the swab from the top slide of the gun. It is estimated that one in 100 trillion (1012) African Americans, one in 15 quadrillion (1015) Caucasians and one in 2.4 quintillion Hispanics (1018) would possess profiles that would be included as potential contributors to the deduced partial male profile.11/7/11 Report of Laboratory Examination (5/16/17 Hr'g Def. Ex. J, BG077309).
CODIS is the Combined DNA Index System, a national DNA database maintained by the FBI.
C. Serological Research Institute (SERI) and Defendants Charles Heard and Esau Ferdinand
Heard is charged with the August 14, 2008 murders of Andre Helton and Isiah Turner. Second Superseding Indictment (Dkt. No. 139 at 13). Evidentiary samples were analyzed by serologists with a private laboratory, the Serological Research Institute (SERI). SERI is also accredited by ASCLD and follows the FBI's QAS for DNA testing. 5/17/17 Hr'g Tr. at 215-17 (Dkt. No. 1186). But because it is a private laboratory, it does not have access to CODIS. Id. at 219-20.
1. Interpretation of Samples Compared to Heard's Reference Sample
SERI used the Identifiler Plus test kit to analyze material from swabs taken from the left rear door of a vehicle in which the victims were found murdered on August 14, 2008. SERI amplified 0.08 ng of Item 5-1 (from the left rear door lock) and 0.25 ng of Item 6-1 (from the left rear door handle) using 29 cycles of amplification. It used a 10 second injection for both samples, and doubled the amount of DNA injected for Item 6-1. The results indicated that both samples were complex mixtures—Item 5-1 containing at least 3 individuals and Item 6-1 containing at least 4 individuals.
The following recitation is extracted from Heard's Supplemental Memorandum, which includes citations to Bates-stamped documents. See Dkt. No. 1172.
SERI used statistic interpretation called the Combined Probability of Inclusion (CPI) to analyze Items 5-1 and 6-1. For Item 5-1, the lab concluded:
Charles Heard could be a contributor to item 5-1. Approximately one person in 12,600 could also be similarly included with respect to Heard. Isiah Turner could be a minor contributor to this DNA profile. Approximately 1 person in 155 could also be a minor contributor with respect to Turner. Andre Helton and Reginald Elmore are not contributors to the DNA from item 5-1.SERI Second Analytical Report at 7. And for Item 6-1,
Charles Heard could be a contributor to item 6-1. Approximately one person in 20 could also be similarly included with respect to Heard. Isiah Turner and Reginald Elmore could be minor contributors to this genetic profile. Approximately one person in 18 could also be a minor contributor with respect to Isiah Turner and approximately one person in 2 could also be a minor contributor with respect to Reginald Elmore. Andre Helton is not a contributor to the DNA profile from item 6-1.
2. Interpretation of Samples Compared to Ferdinand's Reference Sample
Ferdinand is charged in connection with the Jelvon Helton murder on November 1, 2010. Swabs from a Cincinnati Reds baseball cap (Item 3A from the front sweatband, Item 3B from the rear sweatband, and Item 3C from the back inside dome) and an Acura automobile (Item 2-2 from the steering wheel, Item 2-3 from the front passenger handle, Item 5 from rear driver carpet stain) were tested in connection with the murder. The samples were sent to SERI for analysis. Chief Forensic Serologist Gary Harmor analyzed the swabs from the baseball cap and issued a report on January 27, 2015; Forensic Serologist Cassedy Baker analyzed the Acura samples, and issued a report on January 23, 2015.
The Analytical Reports were admitted as exhibits during the Daubert hearing.
Harmor generated the following conclusions:
• The genetic marker profile obtained from the back inside dome swabbing of the red baseball cap #1B115 (item 3C) is a mixture from at least three individuals. Jelvon Helton #1B219 could be a major contributor to this mixture. The chance someone unrelated to him could also be the major contributor is approximately one in 13 sextillion. Esau Ferdinand #1B232 could be a minor contributor to the mixture as well as approximately one in 12 persons with relationship to him.1/27/15 SERI Analytical Report at BG084405.
• The genetic marker profile obtained from the rear sweatband swabbing of the red baseball cap #1B115 (item 3B) is a mixture from at least three individuals. Jelvon Helton #1B219 could be a major contributor to this mixture as well as approximately one in 6170 persons. Esau Ferdinand #1B232 could be a minor contributor to the mixture. Approximately one in every 55 individuals could also be contributors to the mixture with relationship to Esau Ferdinand.
• The genetic marker profile obtained from the front sweatband swabbing of the red baseball cap #1B115 (item 3A) is a mixture from at least three individuals. Jelvon Helton #1B219 (item 1-1 , SERI Case No. M'9851'14) could be a possible contributor to this mixture as well as approximately one in 59,000 persons. Esau Ferdinand #1B232 (item 4A-1) and Vernon Carmichael #1B218 ( item 2A-1. SERI Case No. M'9852'14) could be minor contributors to the mixture. Approximately one in every 9 individuals could also be contributors to the mixture with relationship to Ferdinand and Carmichael.
Baker's conclusions include:
• DNA recovered from the steering wheel swab (item 2-2) is a mixture of at least four people. Esau Ferdinand and Vernon Carmichael are each included as possible contributors to the
mixture. The chance that a randomly selected person, unrelated to Esau Ferdinand and Vernon Carmichael would be similarly included as a possible contributor is one in ten. Jaquain Young is also included as a possible contributor. The chance that a randomly selected person, unrelated to Jaquain Young would be similarly included as a possible contributor is one in six hundred sixty.1/23/15 SERI Analytical Report at BG078562-BG078563.
• DNA recovered from the front passenger handle swab (item 2-3) is a mixture of at least three people. Esau Ferdinand is included as a possible contributor to the mixture. The chance that a randomly selected person, unrelated to Esau Ferdinand, would be similarly included is about one in two hundred eighty. Vernon Carmichael is also included as a possible contributor. The chance that a randomly selected person, unrelated to Vernon Carmichael, would be similarly included as a contributor is about one in eleven.
II. PROCEDURAL BACKGROUND
On December 1, 2015, Gordon, Heard, and Ferdinand filed motions challenging the admissibility of certain DNA evidence. Dkt. Nos. 638, 639, 648. On January 27, 2016, I ordered a Daubert hearing to consider the reliability of particular DNA results. Dkt. No. 836. After lengthy delays requested by counsel, I conducted a Daubert hearing on the results generated by Helen Kim with Contra Costa County Crime Lab on May 16, 2017, and a hearing on the results generated by Gary Harmor with SERI on May 17, 2017. Dkt. Nos. 1179, 1183 (Transcripts at Dkt. Nos. 1185, 1186). Defendants also presented their DNA expert, Marc Taylor.
Cassedy Baker, who analyzed some of the samples at SERI, did not testify.
LEGAL STANDARD
Federal Rule of Evidence 702 allows a qualified expert to testify "in the form of an opinion or otherwise" where:
(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;Fed. R. Evid. 702.
(b) the testimony is based on sufficient facts or data;
(c) the testimony is the product of reliable principles and methods; and
(d) the expert has reliably applied the principles and methods to the facts of the case.
To be admissible under Rule 702 expert testimony must be relevant and reliable. See Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 589 (1993). "[R]elevance means that the evidence will assist the trier of fact to understand or determine a fact in issue." Cooper v. Brown, 510 F.3d 870, 942 (9th Cir. 2007); see also Primiano v. Cook, 598 F.3d 558, 564 (9th Cir. 2010) ("The requirement that the opinion testimony assist the trier of fact goes primarily to relevance.")(internal quotation marks omitted).
Under the reliability requirement, the expert testimony must "ha[ve] a reliable basis in the knowledge and experience of the relevant discipline." Primiano, 598 F.3d at 565. To ensure reliability, the court must "assess the [expert's] reasoning or methodology, using as appropriate such criteria as testability, publication in peer reviewed literature, and general acceptance." Id. These factors are "helpful, not definitive," and a court has discretion to decide how to test reliability "based on the particular circumstances of the particular case." Id. (internal quotations marks and footnotes omitted). "When evaluating specialized or technical expert opinion testimony, the relevant reliability concerns may focus upon personal knowledge or experience." United States v. Sandoval-Mendoza, 472 F.3d 645, 655 (9th Cir. 2006).
DISCUSSION
I. STANDARDS TO APPLY IN DETERMINING ADMISSIBILITY
The Contra Costa Crime Lab and SERI reached their conclusions in October 2011 and January 2015, respectively. The scientific community's approach to forensic DNA testing has evolved, and I am now faced with determining the analyses' admissibility in 2017. Do I apply current standards, or standards in place at the time of testing, to determine the reliability of the DNA expert reports here. The government concedes that "the Court's consideration of reliability should consider the state of the evidence as it currently stands[,]" but "the fact that the state of the art in forensic testing has evolved does not render the prior testing unreliable[.]" United States' Supp. Post-Hr'g Resp. at 8 (Dkt. No. 1215). Defendants assert that "the touchstone of Rule 702 and each of the Daubert factors referenced above call for the application of current standards rather than standards that existed at some earlier time." Heard's and Ferdinand's Post-Hr'g Memo at 14 (Dkt. No. 1206).
Although, it is worth noting that this was not the government's original position. 5/17/17 Hr'g Tr. at 264:2-5 (Dkt. No. 1186)("[T]he key issue for the Daubert hearing is what was the state of the art when the tests were conducted and did - SERI's testing, was it in conformity with that state of the art."); see United States' Supp. Br. in Opp'n at 9-10 (Dkt. No. 1208)(arguing that outdated methodology is a subject for impeachment).
An old method is not unreliable simply because there has been scientific progress since it was employed, but that progress must be considered. Federal Rule of Evidence 702 "assign[s] to the trial judge the task of ensuring that an expert's testimony both rests on a reliable foundation and is relevant to the task at hand." Daubert, 509 U.S. at 597. Is a foundation reliable if the scientific community has continually chipped away at it to the point where it is barely standing? While the nature of the judicial system often leaves it lagging behind the scientific community, it would be counterproductive to manufacture a "historical" concept of reliability simply because the methodology at issue complied with scientific principles that were accepted at the time.
The Daubert court quoted the Honorable Benjamin Cardozo when comparing the characteristics of the judiciary and the scientific endeavor: "The work of a judge is in one sense enduring and in another ephemeral.... In the endless process of testing and retesting, there is a constant rejection of the dross and a constant retention of whatever is pure and sound and fine." Daubert, 509 U.S. at 597 n.13 (quoting B. Cardozo, The Nature of the Judicial Process 178, 179 (1921)).
Daubert was particularly focused on articulating the appropriate test for admissibility of novel scientific evidence. See 509 U.S. at 584-586 (looking to FRE 702 and finding that it displaced Frye's "general acceptance" test as the exclusive test for admitting expert scientific testimony in federal trials). But its tenets prove helpful here in deciding the admissibility of methodology that has fallen out of favor. When the basis of an expert's testimony is "scientific knowledge," the "adjective 'scientific' implies a grounding in the methods and procedures of science[,]" and "the word 'knowledge' connotes more than subjective belief or unsupported speculation." Id. at 590. To be "ground[ed] in ... science" certainly implies the applicability of current standards because the "[p]roposed testimony must be supported by appropriate validation—i.e., 'good grounds,' based on what is known." Id. (emphasis added). The court must have had in mind current standards of what is known, not what was known, for example, two and a half to six years ago (as in this case). Moreover, the very concept of validity in the context of science implies consideration of what is presently known, not what was known in the past. See id. 592-93 (holding that the decision on the admissibility of expert scientific testimony "entails a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid... .")(emphasis added); see also City of Pomona v. SQM N. Am. Corp., ___ F.3d ___, 2017 WL 3378770, at *5 (9th Cir. Aug. 7, 2017)(Pomona II)(finding the district court's denial of a motion to update an expert report amounted to an abuse of discretion because "the level of support for his conclusions was critical and it was important that his proffered testimony be accurate and reflect the current state of knowledge in the scientific community.").
This conclusion is reinforced when considering the factor of whether a theory has been subjected to peer review. "[S]ubmission to the scrutiny of the scientific community is a component of 'good science,' in part because it increases the likelihood that substantive flaws in methodology will be detected." 509 U.S. at 593. What is the purpose of identifying "substantive flaws in methodology" if they are to be ignored only because they were discovered after the testing at issue? See id. at 597 ("those [scientific hypotheses] that are incorrect will eventually be shown to be so, and that in itself is an advance."). Reliability is often referred to as trustworthiness, e.g., id. at 590 n.9 ("our reference here is to evidentiary reliability—that is, trustworthiness"). An opinion based on tarnished methodology can hardly be considered trustworthy. With the understanding that I must consider current standards when assessing this issue, I proceed to a discussion of the proposed DNA experts in this case.
II. HELEN KIM WITH CONTRA COSTA CRIME LAB (RE: GORDON'S DNA SAMPLES)
Gordon seeks to exclude specific DNA evidence analyzed by the Contra Costa Crime Lab. His initial challenge was based on his assertion that the lab tested LCN DNA samples, which led to unreliable results. Gordon's Mot. to Exclude DNA Results and Request for Daubert Hr'g at 4 (Dkt. No. 639). After the Daubert hearing, Gordon refined his argument. He narrowed it to the admissibility of sample 4-3, arguing that the sample must be excluded as unreliable because Kim's opinion is based on two unsubstantiated assumptions: first, that Female A was present in the sample; and second, that only two contributors were present. Gordon's Post-Hr'g Memo at 2 (Dkt. No. 1204).
The government insists that Gordon's challenges go to the weight of the evidence, and not its admissibility. U.S.'s Supp. Post-Hr'g Response at 2-6 (Dkt. No. 1215). I agree. Kim testified—and Gordon does not dispute—that her lab is accredited, her techniques were approved, accepted, and validated, and consistent with quality assurance standards promulgated by the FBI, as well as SWGDAM. 5/16/17 Hr'g Tr. at 85. She arrived at her conclusion that sample 4-3 included only two contributors based on another assumption—that Female A was a "major contributor" to the sample. This assumption was deduced from looking below the analytical threshold and identifying possible alleles consistent with Female A. She testified that these assumptions were based on reliable techniques. E.g., 5/16/17 Hr'g Tr. at 141:22-142:1 ("I don't think I could interpret this data without assuming that Female A is present. This assumption was made based on the fact that she's fully represented in this data. So if I was not able to assume Female A was present, I would have said inconclusive for this."); id. at 142:14-16 ("If there are three contributors and not the two that I made the assumption of, then I wouldn't - there would be - I wouldn't be able to interpret this."); id. at 142:19-21 ("The assumptions are made based on the data; and based on what I see in the data, an assumption is made so that I can go forward with that.").
Gordon asserts that had Kim looked below the analytical threshold at different loci, she would have had to conclude that "there [were] at least three individuals contributing to the mixture at 4-3." Gordon's Post-Hr'g Memo at 4 (Dkt. No. 1204). He argues that data in the stochastic range is inherently unreliable, and Kim should have exercised more caution in her analysis. Kim explained her methodology, specifically, her decisions to look below the analytical thresholds at certain loci, by relying on her training and experience. She points to no guidelines to support her analysis.
Gordon urges me to apply the rationale of United States v. McCluskey, 954 F. Supp. 2d 1224 (D.N.M. 2013), in which the court excluded DNA testing results derived from a low copy number (LCN) DNA sample. Id. at 1274-1288. The McCluskey court excluded the LCN test results based on several factors, including the lab's lack of certification and validation of its LCN testing. Id. at 1288; see also United States v. Morgan, 53 F. Supp. 3d 732, 736 n.2 (S.D.N.Y. 2014) (discussing McCluskey's reasoning in excluding the LCN data, and ultimately ruling LCN DNA test results admissible). The court highlighted its concern that the analyst justified her use of certain methodology based on her own experience "without citation to scientific studies, scientific literature, or any special training[.]" McCluskey, 954 F. Supp. 2d at 1286; see id. ("A court making a determination of reliability under Daubert and Rule 702 is justified in rejecting the 'ipse dixit' of an expert."). In deciding to exclude the LCN evidence, the court was careful to articulate its basis for exclusion—not merely the use of an LCN DNA sample, but rather, the lab's methodology in interpreting that sample. See id. at 1286 (describing the case as one in which "reliable methodology is applied to a different area [LCN testing] without sufficient justification").
The government insists that the testing here did not constitute LCN testing, so the reasoning of McCluskey should not apply. Sample 4-2 consisted of 1.53 nanograms of material and sample 4-3A consisted of .2 nanograms of material. Kim's analysis used two enhanced detection techniques—one included using a "superjuice" which helps overcome inhibition, and the other increased the injection time to 10 seconds, which is designed to bring up any peaks that fall below the analytical threshold. She testified that since she did not use the enhanced detection technique of increasing the amplification cycles, there was no LCN analysis here. 5/16/17 Hr'g Tr. at 85:9-18 (Dkt. No. 1185).
As the McCluskey court made clear, the label associated with the testing is not the determinative factor in deciding the admissibility of the results. 954 F. Supp. 2d at 1286-88. Rather, the critical inquiry is whether the lab utilized reliable testing methods. Id. As Gordon highlights, the Government cannot and should not rely solely on a lab's accreditation and verification protocol to argue that the lab's results are reliable. To ensure reliability, the particular testing performed by an accredited lab must still conform to the lab's established protocol.
This is one of the distinguishing flaws with SERI's testing. See discussion infra section III.
Gordon cannot point to any evidence that Kim failed to abide by established protocol. Instead, he challenges the assumptions underlying her interpretation of the data. Gordon has all the information he needs regarding Kim's analysis to cross-examine her at trial. It would be improper to exclude such evidence from the purview of the jury when the lab utilized reliable methods that meet the standards under Daubert. See Daubert, 509 U.S. at 596 ("Vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence."). III. GARY HARMOR OF SEROLOGICAL RESEARCH INSTITUTE (SERI)(RE: FERDINAND'S AND HEARD'S DNA SAMPLES)
Heard's and Ferdinand's challenge to the government's expert reports from SERI go to the "pattern and practice" of SERI. See, e.g., 5/17/17 Hr'g Tr. at 310:1-5. Although Casseday Baker analyzed some of the samples associated with Ferdinand's case, Harmor testified that the methodology and general testing procedures employed by Baker were consistent with his testimony. See 5/17/17 Hr'g Tr. at 309:10-25. For this reason, I will refer to the testing done by SERI, rather than differentiating between Harmor and Baker.
The problems with SERI's methodology and implementation of that methodology are serious. Heard and Ferdinand assert that there is no reliable scientific basis for the DNA testing conducted by SERI because it failed to apply scientific methods when interpreting the samples. See Heard's Supp. Memo. re: DNA Daubert Hr'g at 2 (Dkt. No. 1172); Ferdinand's Supp. Memo. re: DNA Daubert Hr'g at 6 (Dkt. No. 1175). They raise several reasons to exclude the testing: results generated fell within the lab's designated stochastic range, the use of enhanced detection methods leads to greater stochastic effects, and the use of the Combined Probability Index (CPI) statistical model, which involves a "suspect-centric" approach, results in interpretation bias and is inappropriate for complex samples. Id. at 3. In their post-hearing briefing, the defendants focus on "new and potentially more problematic issues" surrounding Harmor's methodology: that it is not contained in the lab's protocol, is not supported by peer-reviewed articles, and is based "wholly on Mr. Harmor's training, experience, and subjective judgment." Defs. Heard's and Ferdinand's Post-Hr'g Memo at 2 (Dkt. No. 1206). They argue that SERI's results are not reproducible and therefore are unreliable.
Defendants appear to drop their challenges based on Harmor's use of enhanced detection techniques (29 amplification cycles, a 10-second injection time, and the use of a 2 microliter sample instead of the standard 1.5 microliter sample). I nonetheless discuss them here because they are relevant to the reliability of SERI's expert opinions.
The government frames SERI's methodology in a much different light. It argues that "SERI's use of CPI was consistent with its accreditation requirements, with QAS, and with the manner in which many other labs were using it[,]" and "the fact that the scientific community tweaked the application of an accepted methodology does not render the prior use of that methodology unreliable." United States' Supp. Br. in Opp'n at 9-10 (Dkt. No. 1208); see id. ("[T]he fact that consensus within the scientific community has changed the way a particular methodology is applied—or has moved on to a more refined technique—does not mean that testimony based on that methodology is so unreliable that the trial court should prevent the jury from hearing it.").
I cannot agree with the government. The issues with SERI's expert reports are threefold. First, SERI's testing generated results below the stochastic threshold, which indicates the possibility of allelic dropout. Although Harmor did not consider alleles below the stochastic threshold in developing a profile, he did consider them in comparison to references. As discussed in the background section, the mere presence of results below the stochastic threshold indicates that some degree of randomness, and therefore questionable reliability, exists. See 5/17/17 Hr'g Tr. at 265:20-23. Second, he used two enhanced detection methods to account for the small amount of DNA available for testing. He testified that the lab protocol recommended using one or the other, but he chose to do both because he was "starting with low-template copy DNA." 5/17/17 Hr'g Tr. at 305:23-306:3; id. at 306:15-17. The enhanced detection methods were individually validated, but he "[didn't] recall" whether they were validated for use at the same time. Id. at 306:4-6. This presents a problem; SWGDAM provides that "testing of the combination should be done in a step-wise fashion by adding one Enhanced Detection Method at a time[,]" and "a full internal validation of the Enhanced Detection Method procedure must be conducted in accordance to the QAS." SWGDAM Guidelines for STR Enhanced Detection Methods 3.3 (Dkt. No. 639-1 at 156). Further, Harmor testified that the use of 29 cycles and 10 seconds increases the possibility of stochastic results. 5/17/17 Hr'g Tr. at 229:24-230:2. These issues raise substantial questions concerning SERI's methodology.
This relates to the suspect-centric CPI statistical interpretation. See discussion infra.
In addition to the known stochastic effects, it is unclear if the stochastic threshold for the Identifiler Plus system was set based on complex samples. 5/17/17 Hr'g Tr. at 304:7-305:10. Further, the enhanced detection techniques increased peak heights above the stochastic threshold, but that does not necessarily make that data more reliable because the "randomizing" effects occur during the amplification process itself. 5/17/17 Hr'g Tr. at 304:2-6; see United States v. McCluskey, 954 F. Supp. 2d 1224, 1285 (D.N.M. 2013)("[O]ne of the criticisms of LCN interpretation is that, once stochastic effects have occurred at some loci, there is no way to be certain that the stochastic effects were confined to those loci and did not occur at other loci.").
Third, SERI applied the CPI statistical model on complex mixed samples in an unreliable and untestable manner. Added to the other issues, this is an insurmountable problem. The questions stemming from the use of this method generally, and specifically by SERI, bear scrutiny.
The Probability of Inclusion (PI) for a locus is calculated as the sum of allele frequencies for all detected alleles at a given locus, and the CPI is the product of the individual locus PIs. SWGDAM Interpretation Guidelines for Autosomal STR Typing by Forensic DNA Testing Laboratories § 4C (Dkt. No. 1214-1 at 178).
Each of the three reasons could be further expounded, but I am focusing on the third reason in alignment with defendants' post-hearing briefing.
"For scientific opinion, the court must assess the reasoning or methodology, using as appropriate such criteria as testability, publication in peer reviewed literature, and general acceptance, but the inquiry is a flexible one. Primiano, 598 F.3d at 564. This assessment counsels against admitting SERI's reports. First, the methodology is untestable. Harmor admitted that his interpretations involved "very subjective" determinations completely detached from any objective criteria. 5/17/17 Hr'g Tr. at 274:3-24. In determining which loci to interpret and which to ignore, he was driven by whether the suspect's profile could be present. This necessarily biased his conclusions. See Lasalle v. McNeilus Trucking Mfg., Inc., Case No. 16-cv-0076-WHO, Dkt. No. 72 ("Rather than engage with the evidence and explain any scientific basis for dismissing alternate theories, his only methodology was to ignore anything inconsistent with that conclusion."). The subjective determinations are ingrained in the statistical interpretation that generates the conclusion, and that interpretation cannot be removed from the methodology underlying the testing.
That he may have had a reason for his decisions and they were not made on a "whim," does not mean that they were not subjective. See Hr'g Tr. at 326:15-327:15.
Moreover, SERI's methodology was not in compliance with the SWGDAM guidelines, although admittedly ambiguous at the time. Harmor conceded that his lab had to make certain adjustments to the way it performs CPI analysis based on feedback from one of the authors of the Texas audit report, discussed below, and those changes were implemented after the testing at issue here. Id. at 239:14-240:3. Moreover, SERI's methodology was not supported by its own protocol, id. at 306, had no basis in scientific publications, id. at 296, and was so subjective that it could even vary between analysts at SERI, id. at 275; id. at 296:16-19.
The most recent SWGDAM Guidelines on CPI provide,
When the interpretation makes no assumptions of the number of contributors, loci with alleles below the stochastic threshold may not be used for statistical purposes to support an inclusion. In these instances, the potential for allelic dropout raises the possibility of contributors having genotypes not encompassed by the interpreted alleles. At a locus having some alleles above the stochastic threshold and one or more alleles below that threshold, in the standard application of the CPI and CPE, no calculation would be performed at this locus. Because allelic dropout is possible at these loci, genotypes from possible contributors may not be represented in the statistical calculation.SWGDAM Interpretation Guidelines for Autosomal STR Typing by Forensic DNA Testing Laboratories § 4C.6.2.
One of the changes pertained to estimating the number of contributors to confirm whether the model fits at every marker, not just those above the stochastic threshold, and the other related to accounting for "stutter," which would affect whether a particular peak height fell above the stochastic threshold. 5/17/17 Hr'g Tr. at 240-241. Harmor stated that the changes recommended by Dr. Budowle would lead to a "more rare" result than their previous methodology, meaning the techniques used to analyze the defendants' samples here were more favorable to them (i.e., less rare). Because the defendants did not receive any discovery related to the Budowle recommendation and were unable to challenge Harmor's representation of them, see Heard's and Ferdinand's Post-Hr'g Memo at 2, 8, I do not consider the effects of those changes in my analysis.
To make matters worse, neither Harmor nor Baker took any notes tracking their interpretations of the data. Id. at 232:9-16; id. at 273:15-275:20; id. at 294:1-10; id. at 295:4-296:19. This means that there is no way for an expert reviewing SERI's statistical interpretations to understand the choices they made. See id. at 294:11-18.
"The question is whether an expert's methodology can be challenged in some objective sense, or whether it is instead simply a subjective, conclusory approach that cannot reasonably be assessed for reliability." City of Pomona v. SQM N. Am. Corp., 750 F.3d 1036, 1046 (9th Cir. 2014)(Pomona I)(quoting Fed. R. Evid. 702 Advisory Committee's Note to 2000 Amendments)(internal quotation marks omitted); id. at 1044 ("Opinion based on unsubstantiated and undocumented information is the antithesis of ... scientifically reliable expert opinion.")(quoting another source). Harmor's testimony was tantamount to an admission that "someone else using the same data and methods" might not "be able to replicate the results." See id. at 1047 (describing the "primary requirement" under Daubert's testability factor). It was not reliable.
CPI methodology has certainly been published in peer reviewed literature. But the literature raises serious concerns about the reliability of the methodology, especially in circumstances such as here, where analysts were interpreting complex mixed samples with observed stochastic effects. See, e.g., P. Gill, et al., DNA Commission of the International Society of Forensic Genetics: Recommendations on the Interpretations of Mixtures, 160 Forensic Sci. In. 90, 95, 101 (2006); B. Budowle, F. Bieber, Final Report on Review of Mixture Interpretation in Selected Casework of the DNA Section of the Forensic Science Laboratory Division, department of Forensic Sciences, District of Columbia (2015)(Dkt. No. 1214-1 at 328). Even Harmor admitted that the forensic community is moving away from the interpretation of mixtures using the CPI statistic. 5/17/17 Hr'g Tr. at 266:15-18. "[A] known technique which has been able to attract only minimal support within the community may properly be viewed with skepticism." Daubert, 509 U.S. at 594 (internal quotation marks and citation omitted). Defendants point to various scientific studies, including a 2016 audit report conducted by the Texas Forensic Science Commission, which indicate that CPI decisions driven by a known profile suffer from interpretation (also known as contextual) bias, and lead to unreliable results. See Final Audit Report for Austin Police Department Forensic Services Division DNA Section (July 8, 2016), available at https://www.dropbox.com/sh/y629croc63hz8im/AACy1KXZdUQlh4oqeoUk2Q5Ka?dl=0&previe w=APD+Audit+Final+report+071116.pdf.
The Report criticized methodology similar to that used by SERI.
Analysts conducted an initial review of evidentiary profiles before reviewing suspect or victim reference profiles. Thus, the analysts believed there was no confirmation or other contextual bias in their interpretation of mixture evidence. However, the analysts decided whether a locus would be used for statistical calculations depending upon the alleles observed in the known profile (whether suspect or victim). The analysts did not determine a priori which loci had a high probability of allele dropout. Instead, they compared the evidence and known reference profiles, and then selected the loci to be used for statistical calculations based on whether the suspect or victim had "dropped out" at a particular locus. If an allele was missing , they invoked "allele dropout" as a reason for not observing the allele and then did not use the locus for statistical purposes. The appropriate approach is to decide which locus (or loci) should be used first based on whether there may be potential allele dropout as indicated by analysis of the evidentiary sample in its entirety, not on which alleles are present or absent based on the victim or suspect reference profiles.
The defense expert noted that CPI analysis is generally accepted for mixtures where all peak heights are above the stochastic threshold, 5/17/17 Hr'g Tr. at 356, but he testified that SERI's suspect-centric technique was "not acceptable under any circumstances in [his] interpretation of what's going on in the field right now." Id. at 356; see also Defs.' Supp. Memo at 4-6 and related exhibits (citing various audit reports documenting labs' inappropriate use of the CPI statistic)(Dkt. No. 1214); SWGDAM Guideline 4.2. Defendants also point out that the government has failed to offer any evidence that SERI's method of employing suspect-centric CPI is (or was) generally accepted in the scientific community. See Heard's and Ferdinand's Post-Hr'g Memo. at 2 n.3.
The risk in using CPI analysis to compare reference samples to profiles from mixed samples is that the suspect-centric approach leads analysts to consider only data that inculpated the defendants while disregarding low-level alleles that did not match the suspect's profile. See, e.g., John M. Butler, ADVANCED TOPIC IN FORENSIC DNA TYPING: INTERPRETATION Ch. 12 (Dkt. No. 639-1 at 81)("CPI is based on evidence only. Selecting different loci for comparison purposes, something often referred to as 'suspect-driven CPI,' is inappropriate since decisions on which loci are suitable for comparison should be made prior to doing a comparison to reference samples."). This risk is exacerbated when analysts consider results with the possibility of allelic dropout. This suspect-centric interpretation may have led to an overstatement of the rarity of the mixtures and the inclusion of individuals because certain data was not considered. See 5/17/17 Hr'g Tr. at 273-75. These factors combine to seriously undermine the reliability of SERI's expert reports. See Daubert, 509 U.S. at 597 ("Conjectures that are probably wrong are of little use, however, in the project of reaching a quick, final, and binding legal judgment—often of great consequence—about a particular set of events in the past.").
The government argues that the defense expert "made clear" that "defendants' challenge to SERI's DNA testing and the opinions developed from those tests is not an attack on the methodology itself; it is an attack on how SERI employed those methods." United States' Supp. Br. in Opp'n at 9 (Dkt. No. 1208). It then argues that evaluation of the attack on what SERI did is the jury's function, and not a job for the Court as gatekeeper. Id.
In this case, of course, the defendants do attack the methodology itself; specifically, the use of suspect-centric CPI statistical analysis. See, Heard's Supp. Memo. re DNA Daubert Hr'g at 2 (Dkt. No. 1172); Ferdinand's Supplemental Memo re DNA Daubert Hr'g at 6-9 (Dkt. No. 1175). Regardless, I fail to see the practical distinction the government seeks to draw between a methodology and the application of that methodology when it comes to my role as gatekeeper. Rule 702 explicitly directs courts to consider whether "the expert has reliably applied the principles and methods to the facts of the case." Fed. R. Evid. 702(d)(emphasis added). Proper application of the methods is a necessary component of ensuring the reliability of the opinion testimony. If SERI improperly employed accepted methodology then the results would lack a sound basis. That inquiry is appropriately included within the scope of a Daubert analysis. See Daubert v. Merrell Dow Pharmaceuticals, Inc. ("Daubert II"), 43 F.3d 1311, 1316 (9th Cir. 1995)("Our task, then, is to analyze not what the experts say, but what basis they have for saying it."). The basis for an expert's opinion must necessarily entail how he employed his methodology; that consideration is critical to a determination of whether the opinion "rests on a reliable foundation." See Daubert, 509 U.S. at 597.
Consideration of a methodology's scientific validity must necessarily entail how the methodology is employed, even when the methodology itself is generally accepted. E.g., B. Budowle, Low Copy Number Typing Still Lacks Robustness and Reliability (2010)(Dkt. No. 639-1 at 8).
The government also argues that the "tweaking" of SERI's protocols does not rise to the level of "nonsense opinion" or "junk science" that should be removed from the purview of the jury. See United States' Supp. Post-Hr'g Resp. at 9. The Ninth Circuit has cautioned district courts to take a "measured approach to an expert's adherence to methodological protocol" because "there is a strong emphasis on the role of the fact finder in assessing and weighing the evidence." Pomona I, 750 F.3d at 1048. In determining where to draw the line, "expert evidence is inadmissible where the analysis is the result of a faulty methodology or theory as opposed to imperfect execution of laboratory techniques whose theoretical foundation is sufficiently accepted in the scientific community to pass muster under Daubert." Id. at 1047-48 (internal quotation marks omitted). "The rationale of this approach is that a minor flaw in an expert's reasoning or a slight modification of an otherwise reliable method does not render expert testimony inadmissible." Id. at 1048.
To be sure, other courts in this district have admitted DNA evidence analyzed by SERI. See, e.g., USA v. Hayes, No. 13-CR-00085-JD-1, 2014 WL 5470496, at *3 (N.D. Cal. Oct. 21, 2014)(finding SERI adhered to protocol in using RMP for single-source samples and CPI for mixed samples and admitting the results); United States v. Sleugh, No. 14-CR-00168-YGR-2, 2015 WL 3866270, at *2 (N.D. Cal. June 22, 2015)(admitting DNA evidence where "government offered assurances that its serologist had not observed any stochastic effects."). My decision to exclude SERI's results here does not rest on a "minor flaw" or "slight modification." SERI analysts failed to adhere to their own lab protocol or take any notes documenting their decision-making process. And they cannot point to any objective criteria guiding their methodology. Harmor repeatedly testified that his decisions were "very subjective" and based on his training and experience. "[N]othing in either Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence that is connected to existing data only by the ipse dixit of the expert. A court may conclude that there is simply too great an analytical gap between the data and the opinion proffered." Gen. Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997). Reliance on experience without support from scientific literature does not provide a sufficient foundation for scientific knowledge. See McCluskey, 954 F. Supp. 2d at 1286 ("It is not sufficient to simply point to lengthy experience in the field, without explaining how that experience supports the expert's opinion."). SERI's failures are "significant enough to render [their] entire analysis unreliable." Pomona I, 750 F.3d at 1048.
I am not excluding the reports from SERI because they contain conclusions that are wrong. See Alaska Rent-A-Car, Inc. v. Avis Budget Grp., Inc., 738 F.3d 960, 969 (9th Cir. 2013)("The district court is not tasked with deciding whether the expert is right or wrong... ."). "[T]he test under Daubert is not the correctness of the expert's conclusions but the soundness of his methodology." Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1318 (9th Cir. 1995)(Daubert II). I have no idea whether the conclusions are correct. I am excluding SERI's opinions because they are not based on sound methodology. Because of the lack of objective criteria, or even notes documenting the admittedly subjective determinations, the defendants would be unable to challenge this evidence in front of a jury. See id. ("T]he judge is supposed to screen the jury from unreliable nonsense opinions, but not exclude opinions merely because they are impeachable."). Contrary to the government's position, this is not a case of "shaky but admissible evidence" that can be resolved through "[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof." See Daubert, 509 U.S. at 596.
This result is reinforced by the fact that defendants admit that the conclusions at issue here are "not exceedingly rare." Heard's and Ferdinand's Post-Hr'g Memo at 11. In other words, they are, relatively speaking, fairly common. --------
If I were to conclude otherwise, I would still exclude it under Federal Rule of Evidence 403. See Daubert, 509 U.S. at 595 ("Expert evidence can be both powerful and quite misleading because of the difficulty in evaluating it. Because of this risk, the judge in weighing possible prejudice against probative force under Rule 403 of the present rules exercises more control over experts than over lay witnesses."). In the context of DNA evidence, the Ninth Circuit has stated:
Daubert enjoins watchful assessment of the risk that a jury would assign undue weight to DNA profiling statistics even after hearing appellant's opposing evidence, the testimony of Government
witnesses under vigorous cross-examination and the careful instructions of the district court on burdens of proof. Of particular concern is where the Government seeks to present probability testimony derived from statistical analysis, the third main phase of DNA profiling. Numerous hazards attend the courtroom presentation of statistical evidence of any sort. Accordingly, Rule 403 requires judicial vigilance against the risk that such evidence will inordinately distract the jury from or skew its perception of other, potentially exculpatory evidence lacking not so much probative force as scientific gloss.United States v. Chischilly, 30 F.3d 1144, 1156 (9th Cir. 1994), overruled on other grounds by United States v. Preston, 751 F.3d 1008 (9th Cir. 2014)(footnotes omitted). Accordingly, Rule 403, and the risks of undue prejudice stemming from the untrustworthy results generated by SERI, provides an alternate reason for excluding SERI's DNA evidence.
CONCLUSION
For the foregoing reasons, defendant Gordon's Daubert motion to exclude DNA evidence is DENIED, and defendants Heard and Ferdinand's Daubert motions to exclude DNA evidence are GRANTED.
IT IS SO ORDERED. Dated: August 15, 2017
/s/_________
William H. Orrick
United States District Judge
Id. at 15-16.