Opinion
S4 15-CR-153 (VSB)
06-05-2018
UNITED STATES OF AMERICA, v. DEAN JONES, a/k/a "Korrupt," a/k/a "Blacko," a/k/a "Christopher C. Walker," Defendant.
Appearances: Thomas McKay Christopher Jordan Clore U.S. Attorney's Office New York, New York Counsel for the Government Anthony Strazza Law Office of Anthony Strazza White Plains, New York Giovanni Rosania Law Offices of Giovanni Rosania Uniondale, New York Counsel for Defendant
OPINION & ORDER Appearances: Thomas McKay
Christopher Jordan Clore
U.S. Attorney's Office
New York, New York
Counsel for the Government Anthony Strazza
Law Office of Anthony Strazza
White Plains, New York Giovanni Rosania
Law Offices of Giovanni Rosania
Uniondale, New York
Counsel for Defendant VERNON S. BRODERICK, United States District Judge:
Defendant Dean Jones moved to exclude any evidence at trial produced by the Forensic Statistical Tool ("FST") and requested a hearing under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and its progeny. I granted Defendant's request for a Daubert hearing, and held the hearing on November 6, 8, 14-16, and 21, 2017 (the "Hearing").
I issued an order on November 30, 2017, (Doc. 396), closing the record and denying Jones's motion to exclude evidence at trial produced by the FST, (Doc. 15). In issuing the order, I reviewed the extensive record, including: (1) Defendant's Notice of Motion to exclude evidence produced by the FST and requesting a Daubert hearing, (Doc. 15), and Defendant Dean Jones Memorandum of Law in Support of His Motion to Exclude Evidence Produced by the FST and Request for a Daubert Hearing, (Doc. 16), and exhibits; (2) the Government's Opposition to Defendant's Motion to Exclude Evidence Produced by the FST and for a Daubert Hearing, (Doc 19), and exhibits; and (3) Defendant Dean Jones Reply Memorandum in Support of His Motion to Exclude Evidence Produced by the FST and/or Request for a Daubert Hearing, (Doc. 21). I also reviewed the testimony at the Hearing of Dr. Craig O'Connor, Dr. Adele Mitchell, Dr. Eli Shapiro, and Nathaniel Adams; the prior testimony of Dr. Ranajit Chakraborty in People v. Collins, 15 N.Y.S.3d 564 (Sup. Ct. 2015), which was submitted by the parties at the Hearing in lieu of live testimony; the exhibits introduced in the record at the Hearing; and the various letters, supplemental briefing, and exhibits the parties filed as of November 30, 2017. In the instant Opinion & Order, I state my findings and reasoning in reaching the determination that the methods of the FST that the Office of the Chief Medical Examiner ("OCME") employed in this matter are sufficiently reliable to satisfy the Daubert standard.
1. Background
The overview of DNA and DNA testing is based upon the testimony of Dr. O'Connor. During his testimony, Dr. O'Connor referred to various PowerPoint slides as an aide to his testimony. (GX 100.) "GX" refers to Government exhibit.
Deoxyribonucleic acid ("DNA") is located in almost every cell of the human body; it contains all of the information that determines an individual's genetic makeup. The structure of DNA is sometimes compared to a twisted ladder or a double helix. (Tr. at 16.) Each "rung" of the twisted ladder consists of DNA bases or base pairs: A, C, G, and T. (Id.) The base on one strand forms a "rung" of the ladder with a corresponding base on the other strand. (Id.) The order in which the base pairs are arranged constitutes an individual's DNA sequence; variations in that sequence are what give rise to genetic differences. (Id. at 16-17.)
"Tr." refers to the transcript of the Hearing, which was held over several days on November 6, 8, 14-16, and 21, 2017.
The letters A, C, G, and T stand for adenine, cytosine, guanine, and thymine, respectively. (Doc. 19, at 7.)
A "locus" is a location on the DNA itself, and the plural of locus is "loci." (Id. at 18.) For example, there is an eye color locus, which is a location on the DNA that corresponds to eye color. (Id.) An "allele" is a form of DNA at an individual locus. (Id.) The eye color locus, for instance, is a location that determines the eye color for an individual, and on that locus are the alleles for blue eyes or brown eyes or green eyes. (Id.) In other words, the differences in the DNA at a given locus are determined by the make-up of the alleles. (Id.)
Modern DNA testing involves four steps: (1) extraction, (2) quantitation, (3) Polymerase Chain Reaction ("PCR") amplification, and (4) analysis of the resulting DNA alleles. Extraction is the recovery of DNA from human cells. (Id. at 23-24.) Quantitation is the measurement of the amount of DNA extracted from a given sample. (Id. at 23.) If sufficient DNA is detected, amplification and analysis can be attempted. (Id.) PCR amplification produces larger amounts of DNA from small starting amounts of DNA. The process involves "making copies, multiple copies, millions of copies of the DNA at the locations that [are being tested] in order to better analyze the DNA." (Id. 23-24.)
After PCR amplification, fifteen different loci are then analyzed to give the number of short tandem repeats ("STRs") at each particular location. A STR is a segment of DNA that is repeated a number of times in tandem. (Id. at 18.) The number of repeats at a particular locus constitutes the DNA type or allele present at that location. (Id. at 18-19.) For example, if fifteen repeats of a sequence of bases were present in the STR locus, the DNA allele would be marked "15." (Id. at 19.) Because each person receives half of her DNA from one biological parent and half of her DNA from the other biological parent, each person has two DNA alleles at each locus. (Id.) Thus, the DNA type at one locus may be "5, 7," which is sometimes referred to as an "allele call," and would show the presence of five STRs from one parent at a particular locus, and seven STRs from the other parent at the same locus. (See id. at 18-19, 26-27.) The resulting series of "allele calls" at the fifteen different loci constitutes the DNA profile. (See id. at 19-20.) The DNA profile can then be compared against other profiles—including the profiles created from DNA samples collected from crime-scene evidence—to determine whether a DNA match exists.
B. The FST
While certain methods of DNA testing have historically been widely accepted by courts as reliable, the admissibility of the particular form of DNA testing at issue here, FST testing, presents an issue of first impression in this Circuit. At a high level, the FST is a software program that OCME uses to examine DNA evidence and put quantitative weight to qualitative conclusions about that DNA evidence. (Id. at 30.) To achieve this goal, the FST calculates a statistic—a likelihood ratio ("LR")—which is a ratio of two different probabilities:
As of the date of this Opinion & Order, I am not aware of any federal decision ruling on the admissibility of evidence produced by the FST under the Daubert standard. The Government points to a number of criminal cases in this Circuit where evidence produced by the FST was used at trial. (Doc. 387 (citing United States v. Wallace, 15 Cr. 794 (S.D.N.Y. 2016); United States v. Chandler, 15 Cr. 131 (E.D.N.Y. 2016); United States v. Jones, 16 Cr. 594 (S.D.N.Y. 2016); United States v. Figueroa, 15 Cr. 495 (E.D.N.Y. 2016); United States v. Legrier, 15 Cr. 206 (S.D.N.Y. 2015); United States v. Soto, 12 Cr. 556 (S.D.N.Y. 2013)).) Based upon a review of the dockets and representations by the Government, these cases did not involve a Daubert challenge to the admissibility of FST evidence.
In the numerator is the probability of a set of data conditional on one hypothesis; in the denominator is the probability of the same set of data conditional on a mutually exclusive hypothesis. For forensic DNA applications, the data are the alleles found in the evidence sample, the
hypothesis in the numerator is that of the prosecutor (Hp), and the hypothesis in the denominator is that of the defense (Hd). The LR is a measure of the support for the prosecution hypothesis relative to that of the defense. If the LR is greater than one, Hp is better supported by the data than Hd; if the LR is less than one, Hd is better supported by the data than Hp. For single source evidence profiles, the Hp is typically that a particular suspect is the source of the crime scene DNA and Hd is that an unknown, unrelated person is the source of that DNA. For two-person evidence profiles, there are more options for Hp and Hd. . . . For three-person evidence profiles, there are even more possibilities, as up to two known contributors may be included in either or both hypotheses. The number of contributors in the two hypotheses need not be the same and a known contributor that is included in either the numerator or the denominator does not need to be included in the other.(GX 3, at 2.) OCME is the only laboratory in the United States that uses the FST "for the purpose of analyzing DNA evidence and generating a result to use against a criminal defendant in a criminal case in court." (See Doc. 16, at 11.)
C. OCME
OCME was established in 1918. It is a governmental agency charged with identifying the manner and cause of death in specified cases, as well as providing state-of-the-art forensic analysis. OCME was the first governmental agency of its kind in the United States, and it established the first toxicology and serology laboratories in the nation. OCME is a department within New York City government, and it is not affiliated with any law enforcement agency. (Tr. at 11-12.)
See NYC Office of the Chief Medical Examiner, http://www1.nyc.gov/site/ocme/about/about-ocme.page (last visited May 9, 2018).
DNA analysis at OCME is conducted by the Department of Forensic Biology. (Id. at 12-13.) The OCME Forensic Biology Laboratory has been performing DNA testing in criminal cases since the early 1990s. (Id.) OCME's DNA analysis includes the examination of homicide, sexual assault, and other crime evidence for DNA extraction and typing, and that analysis may either incriminate or exclude a suspect. (Id. at 4-5, 23.) OCME performs more DNA analyses than any other public laboratory in the country, and has been called the "gold standard" of DNA laboratories.
See Press Release, New York City Office of the Mayor, Mayor Bloomberg and Chief Medical Examiner Hirsh Cut Ribbon at Largest Government DNA Laboratory in the Country (July 18, 2007), available at http://www1.nyc.gov/office-of-the-mayor/news/245-07/mayor-bloomberg-chief-medical-examiner-hirsch-cut-ribbon-largest-government-dna-laboratory#/0.
The OCME laboratory is accredited by the American Society of Crime Laboratory Directors Laboratory Accreditation Board ("ASCLD Board"). To maintain accreditation, OCME undergoes an assessment every four years, alternating internal and external audits in the intervening years. (Id. at 13.) The external audits are organized and implemented by the National Forensic Science and Technology Center ("NFSTC"), an outside agency that develops standards for the accreditation of all public laboratories in New York State. (Id. at 14-15; GX 102.)
See Forensic Biology Accreditation Certificates, NYC Office of the Chief Medical Examiner, http://www1.nyc.gov/site/ocme/services/accreditation-certificates.page (last visited May 9, 2018); (Tr. at 12-14).
D. Validation
In order to ensure that the FST was reliable, OCME subjected it to a certification and validation process. As part of this process, the New York State Commission on Forensic Science (the "NYSCFS" or the "Forensic Commission") and its DNA Subcommittee approved the use of the FST in criminal cases. The NYSCFS is charged with "increas[ing] and maintain[ing] the effectiveness, efficiency, reliability, and accuracy of forensic laboratories, including forensic DNA laboratories," and "ensur[ing] that forensic analyses, including forensic DNA testing, are performed in accordance with the highest scientific standards practicable." N.Y. Exec. Law § 995-b(2).
The DNA Subcommittee, in turn, is charged with evaluating all methodologies that New York laboratories propose to use for forensic DNA analysis. See id. § 995-b(13)(b) ("The DNA subcommittee shall assess and evaluate all DNA methodologies proposed to be used for forensic analysis . . . ."). The DNA Subcommittee consists of a group of well-known and respected scientists and experts in the field of DNA analysis who advise the Forensic Commission on matters relating to implementation of scientific controls and quality assurance procedures for the performance of forensic DNA analysis. (Tr. at 159.) Its members include distinguished experts in the fields of forensic science, population genetics, molecular biology, and laboratory standards. (Id. at 156, 159.) The Subcommittee makes recommendations to the Forensic Commission, which in turn holds ultimate authority for accrediting forensic laboratories throughout New York State. (Id. at 158-59.) The Subcommittee reviewed the FST over the course of a year and a half, with OCME making four presentations on the FST to the Subcommittee. (Id. at 159-60.) Before each of these presentations, OCME sent the Subcommittee an executive summary, and during the presentations OCME provided summary slides and voluminous reference materials. (Id. at 160; GX 8A, 8B, 8C, 8D.)
Following its presentations to the DNA Subcommittee, OCME spent several months testing and validating results generated by the FST. Among other things, OCME tested the parameters used by the FST regarding what are known as "drop-in" and "drop-out" rates, both of which are used in calculating a LR. (Tr. at 106-07.) "Drop out" describes the situation where an allele that is actually known to exist at a particular locus in a person's DNA sample is not found in the analysis. (Id. at 79-80, 106.) "Drop in" describes the situation where an allele is detected during analysis that is known not to belong to the person or persons contributing to the sample. (Id. at 106-07.)
To develop the drop-in and drop-out parameters used in the FST, OCME conducted an empirical validation study on over 2,000 DNA samples, drawn from known contributors, of varying weights and mixtures, and used that data to deduce probabilities of drop in and drop out in a given DNA sample. (Id. at 41-42; see also GX 2 ("To determine drop-out and drop-in rates, over 2000 samples representing a wide range of DNA concentrations and mixture types were evaluated . . . .").) Specifically, OCME compared the individual DNA profiles of the known contributors to the profiles generated for the mixtures of DNA derived from two, three, or four persons. (Tr. at 136-37; see also GX 2 ("These included purposefully degraded and non-degraded mixtures with known contributors, as well as mock casework samples from items handled by two, three, or four known persons.").) OCME then determined how often a piece of DNA dropped out, which revealed that the drop-out frequency correlated with, among other factors, the quantity of DNA amplified (i.e., "quant"), the number of amplification cycles, the number of contributors to the sample, and the approximate mixture ratio. (See GX 2; GX 3; Tr. at 40, 42-43, 80, 106-09.)
OCME ultimately decided to use quant as a factor in setting the drop-out rates in the FST. (Tr. at 111.) OCME considered alternatives to the use of quant, including the use of peak heights, but ultimately chose to proceed with quant, largely because peak height was less useful in predicting the probability of drop out across all of OCME's electrophoresis instruments. (Id. at 111-12.) OCME subsequently presented the results of its drop-in/drop-out data—including its use of quant as a factor to determine drop-out rates—and a plan to validate them to the DNA Subcommittee. (Id. at 158-60.)
OCME executed the validation plan it had presented to the DNA Subcommittee. It created mock DNA casework samples consisting of saliva, blood, and touched items using materials provided by known contributors. (Id. at 137.) These mock casework samples came from known contributors who represented a wide range of racial and ethnic backgrounds. (See GX 3.) OCME combined these DNA contributions into two- and three-person mixtures, ultimately using 439 samples in the validation process. (See id.; Tr. at 137-38.) OCME then used the FST to calculate a LR for each sample. (See GX 3.) The results showed that the LRs were consistent with the qualitative statements that OCME provided for DNA mixture analysis prior to the use of the FST. (See id.)
Dr. Mitchell testified that, when creating these samples, some of them did not fit the casework criteria for going forward with analysis. For example, some of the samples did not have enough loci that amplified. Ultimately, OCME worked with a total of 439 samples in validating the FST. (See Tr. at 137-39.)
OCME performed its validation of the FST based on the guidelines created by the Scientific Working Group of DNA Analysis Methods ("SWGDAM"). (Tr. at 71-72.) SWGDAM is a group that develops guidelines for a variety of forensic laboratory protocols. (Id. at 138-39.) As SWGDAM recommends, OCME used known samples in performing validation so that it could verify its results. SWGDAM also provides guidelines on how many mixtures should be used in the validation process, recommending use of at least fifty samples. (See GX 15, at 2.) In validating the FST, OCME used 439 samples. (Tr. at 139.) SWGDAM, however, does not provide any guidance on how many different contributors must be used in validation samples. (See id. at 139-40.)
OCME also tested for false positive results. (Id. at 148-49.) The two- and three-person casework samples were compared with DNA from over 1,200 individuals who had not contributed to the casework samples. (Id. at 149.) All together, approximately half a million comparisons were made between the casework samples and the DNA profiles of the non-contributors. (Id.) The tests showed the FST's false positive rate to be very low—of the more than a half million tests performed, only 163 resulted in a false positive. (Id. at 151.) Therefore, the FST's overall false positive rate was 0.03%. (Id. at 149-51.) Of the false positive results contributing to this overall rate of 0.03%, the highest number of false positive results occurred where the FST indicated limited support (a LR of between 1 and 10) for a match, when in reality there was no match. (Id. at 151.) The false positive rate for such instances was 0.01%. (Id.) Where the FST indicated moderate support (a LR of between 10 and 100), the false positive rate was 0.0025%. (Id.) Where the FST indicated strong support (a LR of between 100 and 1000), the false positive rate was 0.0009%. (Id. at 151-52.) Where the FST indicated very strong support (a LR over 1000), there was only one instance where a match was erroneously indicated—a false positive rate of 0.0002%. (Id.) Therefore, the expectation from this analysis is that most of the false positives would occur where the FST indicated a LR of between 1 and 10 for a match. (Id. at 151.)
OCME presented the FST validation results to the DNA Subcommittee, which voted unanimously to approve the use of the FST in forensic casework and issued a recommendation to that effect to the Forensic Commission. (GX 9; see also Tr. at 163.) In December 2010, the Forensic Commission accepted the DNA Subcommittee's recommendation. (GX 10; see also Tr. at 164-65.) As a result, the FST is now accredited for use in forensic DNA analysis in New York State. (GX 12; see also Tr. at 165.)
In addition to receiving the approval of the DNA Subcommittee and the Forensic Commission, OCME has presented the FST for peer review at numerous conferences and in journals. (See, e.g., GX 2; see also Tr. at 81, 90-91, 112.) In addition, the FST has been reviewed and approved by the NFSTC—the entity described above that provides external audits of DNA methods at publicly-funded laboratories. (GX 102; see also Tr. at 14-15.)
E. The Source Code
The source code underlying the FST (the "Source Code") was not publicly available at the time that Defendant moved to exclude the FST evidence in this case. On June 7, 2016, Judge Valerie E. Caproni granted the defendant's request in United States v. Johnson, No. 15-cr-565 (VEC), for a Rule 17(c) subpoena requesting that OCME produce the Source Code to the defendant in that case. (See Doc. 299-1.) Nathaniel Adams, who testified at the Hearing, was one of the individuals who reviewed the Source Code in connection with the Johnson case. (See Doc. 299-2; Tr. at 704-05.) In connection with his review, Adams signed a nondisclosure agreement as required by the protective order governing that case. (Tr. at 705.) In October 2017, Judge Caproni lifted the protective order in Johnson, and OCME subsequently made the Source Code publicly available on a source code repository. (See id. at 705-06, 778.)
F. The Robbery , Blue Latex Glove , and Indictment
On or about June 14, 2013, Jones was arrested on state charges related to a December 21, 2012 robbery ("Robbery"). On or about March 25, 2014, the Honorable Margaret Clancy, Supreme Court Justice, New York, Bronx County, issued an order compelling Jones to provide a DNA sample. In accordance with that order, law enforcement officials collected a DNA sample from Jones, which OCME subsequently compared with DNA samples from the black hat, black mask, and blue latex glove collected as evidence during the investigation of the Robbery. On or about June 10, 2014, OCME issued a Laboratory Report (the "OCME Report") indicating the results of its DNA comparisons.
As to the blue latex glove, the OCME Report indicated that "[b]ased on a comparison of the DNA profile of [Jones] to the mixture found on the sample [collected from the blue latex glove], [Jones] cannot be ruled out as a contributor." (GX 105A.) Using the FST, the criminalist next calculated the probability that Jones was a contributor to the sample collected from the blue latex glove—i.e., the LR. (Id.) The LR revealed that "[t]he DNA mixture found on the glove swabs is approximately 1340 times more probable if the sample originated from [Jones] and two unknown, unrelated persons than if it originated from three unknown, unrelated persons." (Id.) Therefore, the OCME Report concluded that there was "very strong support that [Jones] and two unknown persons contributed to the mixture, rather than three unknown, unrelated persons." (Id. (emphasis omitted).) In the instant motion, Jones seeks to exclude expert testimony related to the FST and OCME's conclusions with regard to blue latex glove, including the LR.
The OCME Report also contained results of the DNA testing related to the black hat. Because the black hat was single source DNA evidence—rather than a DNA mixture from more than one contributor—OCME did not run the FST on the hat. OCME concluded that, "[b]ased on the random match probability for unrelated individuals, Dean Jones is the source of the DNA found on the [black hat]." (GX 105A (emphasis omitted).).
On or about March 11, 2015, a grand jury returned an indictment charging Jones with one count of Hobbs Act robbery, in violation of 18 U.S.C. § 2 and § 1951(a), and one count of possessing a firearm during and in relation to that robbery, which was discharged, in violation of 18 U.S.C. § 2 and § 924(c)(1)(A)(iii). (Doc. 1.) Over several days in November 2017, I held the Hearing; at the close of evidence, I issued a short order denying Defendant's motion to exclude evidence at trial produced by the FST. (Doc. 396.) Trial commenced on December 11, 2017. On December 19, 2017, the jury found Defendant guilty on all counts.
II. Legal Standard
Federal Rule of Evidence 702 permits the admission of expert testimony as long as:
(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; (b) the testimony is based on sufficient facts or data; (c) the testimony is the product of reliable principles and methods; and (d) the expert has reliably applied the principles and methods to the facts of the case.Fed. R. Evid. 702. The Supreme Court in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), held that the trial judge is responsible for "the task of ensuring that an expert's testimony both rests on a reliable foundation and is relevant to the task at hand." Id. at 597. "In determining the admissibility of expert testimony, it is the proponent's burden under Daubert to establish admissibility, rather than the opponent's burden to establish inadmissibility." United States v. Morgan, 53 F. Supp. 3d 732, 740 (S.D.N.Y. 2014) (internal quotation marks omitted), aff'd, 675 F. App'x 53 (2d Cir. 2017) (summary order).
To assist with the task of determining the reliability of expert testimony, Daubert provides the trial judge with five non-exclusive, illustrative factors to apply to the expert's reasoning or methodology: (1) whether a theory or technique has been or can be tested; (2) "whether the theory or technique has been subjected to peer review and publication;" (3) the technique's "known or potential rate of error;" (4) "the existence and maintenance of standards controlling the technique's operation," and (5) whether the technique is generally accepted in the relevant scientific community. Daubert, 509 U.S. at 593-94. These factors are not exclusive and do not constitute a "definitive checklist or test," id. at 593; rather, the "trial judge [has] considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable," Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 152 (1999); see also Amorgianos v. Nat'l R.R. Passenger Corp., 303 F.3d 256, 266 (2d Cir. 2002) ("Although Rule 702 sets forth specific criteria for the district court's consideration, the Daubert inquiry is fluid and will necessarily vary from case to case.").
As the Second Circuit has recognized, however, the inquiry under Daubert is limited, noting that "[a] minor flaw in an expert's reasoning or a slight modification of an otherwise reliable method will not render an expert's opinion per se inadmissible." Amorgianos, 303 F.3d at 267. Rather, "[t]he judge should only exclude the evidence if the flaw is large enough that the expert lacks good grounds for his or her conclusions." Id. (citations and internal quotation marks omitted). At bottom, the Daubert analysis is intended to give the district court the discretion "needed to ensure that the courtroom door remains closed to junk science while admitting reliable expert testimony that will assist the trier of fact." Id.
III. Discussion
Having considered the extensive record in this case, and for the reasons that follow, I find that the FST testing and analysis by OCME are admissible under the standards set forth in Federal Rule of Evidence 702 and Daubert.
A. Prior Cases and the FST
As noted above, I am not aware of any federal decision ruling on the admissibility of FST evidence under the Daubert standard. Despite this issue being of first impression in federal court, the use of FST evidence in courts is by no means new or novel—state courts have repeatedly admitted FST evidence as reliable, even under the less permissive standard annunciated in Frye v. United States, 293 F. 1013 (D.C. Cir. 1923). The Government identifies more than forty state court decisions that have rejected arguments similar to those advanced by the defense here. (See Doc. 19, at 26 (collecting cases as of December 21, 2015; Doc. 292, at 1-2 (collecting cases as of August 17, 2017)). For example, in People v. Belle, No. 3955/13, 2015 WL 2131497 (Sup. Ct. Apr. 29, 2015), the New York Supreme Court ruled that FST evidence was admissible and declined to hold a Frye hearing on the issue. Id. at *1. The court concluded that:
Under Frye, the proponent of a scientific technique must establish the general acceptance of the reliability of the technique within the relevant scientific community. Frye, 293 F. at 1014. Even under Frye, however, the technique at issue did not have to be unanimously accepted by the scientific community. See, e.g., People v. Middleton, 54 N.Y.2d 42, 49 (1981) ("But the test is not whether a particular procedure is unanimously indorsed by the scientific community, but whether it is generally acceptable as reliable."). After the Frye decision, the Supreme Court in Daubert set aside Frye as a matter of federal evidentiary law. See Daubert, 509 U.S. at 586-87. All federal courts have adopted the Daubert standard. Because the Daubert rule constituted an interpretation of the Federal Rules of Evidence, it was not mandated as a matter of constitutional law. Thus, states were free to accept Daubert or continue to adhere to Frye in deciding when expert testimony should be admitted at trial. New York has continued to adhere to the Frye standard, which governed the state court decisions described herein.
[The] FST is based on long-established principles used to calculate statistical probabilities, and utilizes those principles in calculating where possible an easy to understand mathematical number measuring the likelihood of whether a particular defendant, already determined by DNA analysis to be a contributor to a DNA mixture in question, was statistically-speaking, that contributor.Id. at *5. Accordingly, the court determined that evidence and testimony regarding the FST was admissible under Frye. Id. at *4 (stating that it "join[ed] the vast majority of judges who have rejected the same request [to preclude FST evidence]" and citing cases).
The parties have identified only one state court decision that found the FST to be inadmissible: People v. Collins, 15 N.Y.S.3d 564 (Sup. Ct. 2015). In Collins, the New York Supreme Court conducted a Frye hearing, which concluded in December 2013, and on November 7, 2014, the court issued an oral ruling that the FST evidence at issue was inadmissible under Frye. The court in Collins followed up with its written decision on July 2, 2015, declining to admit the FST evidence. Since the written decision in Collins, a number of courts have explicitly rejected its reasoning and held that evidence generated by the FST is admissible under Frye. See, e.g., People v. Lopez, 23 N.Y.S.3d 820, 822, 825-26 (Sup. Ct. 2015) (rejecting Collins and ruling FST evidence admissible without holding a Frye hearing); People v. Debraux, 21 N.Y.S.3d 535, 540-45 (Sup. Ct. 2015) (same); cf. Sullivan v. William Lee, No. 10-CV-425 (CBA), 2017 WL 3634598, at *10 (E.D.N.Y. Aug. 22, 2017) (distinguishing Collins as an outlier in the DNA software program cases and denying petitioner's writ of habeas corpus pursuant to 28 U.S.C. § 2254).
Moreover, the evidence generated by the FST has also been introduced by defendants in cases where—in contrast to here—the LR generated by the FST tended to exculpate a defendant. See People v. Garcia, 963 N.Y.S.2d 517, 523 (Sup. Ct. 2013) (explaining that "[l]ikelihood ratios are expressed by OCME using the FST in terms of strength that are accepted by the scientific community as generally reliable, and actually favored the suspect in over one third of 300 separate cases resulting in 511 likelihood ratios reviewed by OCME in 2012"); Belle, 2015 WL 2131497, at *6-7 (recounting instances in which defendants have called OCME experts to testify about FST evidence where the evidence favored the defendant). Thus, not only has FST evidence been routinely introduced at criminal trials by the prosecution (i.e., where the FST's LR tends to implicate a defendant), but also by the defense (i.e., where the FST's LR tends to exculpate a defendant).
With this background in mind, I turn to the application of the Daubert factors to expert testimony on the FST in this case.
B. Daubert Factors
Of Daubert's non-exclusive factors that I may consider in determining the reliability of expert testimony on the FST, the only two factors that are in meaningful dispute between the parties are (i) the known or potential error rate of the FST and (ii) general acceptance of the FST in the scientific community. I consider each of these factors in turn.
1. Known or Potential Error Rate
As the Government concedes, there is no known or potential error rate for the FST because the number of contributors cannot be determined with certainty in casework. (See Tr. at 824; GX 202, at 21 ("Strictly the true number of contributors to a sample is never known.").) In validating the FST, however, OCME tested for false positive results. (Tr. at 148-49.) In other words, OCME conducted testing to determine the results of the FST when a non-contributor to a sample was tested using the software. (Id.) The two- and three-person casework mixtures were compared with DNA from over 1,200 individuals who had not contributed to the casework mixtures. (Id. at 149.) Approximately half a million comparisons were made between the mixture samples and the DNA profiles of the non-contributors. (Id.) The tests showed the FST's false positive rate to be very low, with only 163 tests resulting in a false positive out of the more than half a million comparisons that were performed. (Id. at 151.) This meant that—in the context of these half a million comparisons—the FST's overall false positive rate was 0.03%. (Id. at 149-51.)
With respect to this case and the blue latex glove in particular, the fact that the false positive tests were highly concentrated at LRs close to one is also informative. OCME's validation of the FST showed that as the LR increased, the false positive rate decreased dramatically. Notably, OCME calculated a LR of 1340 using the FST for the blue latex glove. (GX 105A.) In that specific likelihood range—above 1,000 and below 10,000—the false positive rate was 0.009%. (Tr. at 152.) Therefore, the false positive rate with respect to the evidence at issue in this case is even lower than the overall false positive rate of the FST.
At the Hearing, Dr. Shapiro challenged OCME's calculation of false positive results by attacking OCME's disproportionate use of certain individuals in the 439 samples. Dr. Shapiro testified that, of the contributors at OCME in the validation samples, fourteen individuals had contributed to more validation samples than others. (Id. at 573.) Rather than conducting tests himself using samples, Dr. Shapiro created a universe in which OCME's validation samples were made up of only those fourteen individuals, and using their genotypes and profiles, he calculated a combined probability of exclusion. (Id. at 573-74.) To calculate that probability of exclusion, Dr. Shapiro relied on a combined probability of inclusion ("CPI") calculation. (Id. at 660.) I find Dr. Shapiro's hypothetical study unpersuasive in this context because it fails to mimic the parameters that OCME used in validating the FST, in which OCME considered, among other factors, both allelic drop in and drop out. Dr. Shapiro, by contrast, used a CPI calculation, which excludes individuals when an allele is missing and does not consider allelic drop in and drop out. By Dr. Shapiro's own admission, CPI calculations are disfavored in the scientific community. (Id.) Moreover, SWGDAM represents that fifty mixtures are sufficient for validation, and imposes no restrictions on the proportionality of contributors in validation samples. (See GX 15.) Here, OCME far exceeded fifty mixtures, surpassing the validation guideline imposed by SWGDAM, and it did not violate any validation guideline by using certain contributors more frequently than others in creating those mixtures.
2. General Acceptance
To demonstrate that the FST is not generally accepted in the scientific community, Defendant argues primarily that OCME is the only laboratory in the world that uses the FST, and that this fact alone demonstrates that the FST by definition is not generally accepted in the scientific community. (See Doc. 16, at 15-16.) I disagree. As described above, (supra Section III.A), nearly every court to have considered the FST has found it to be a reliable tool that is generally accepted by the scientific community. Importantly, these court rulings are corroborated by the fact that the FST has been approved for use in casework by members of the relevant scientific community and subjected to peer review.
Defendant also argues that the underlying components of the FST—e.g., LR statistical analysis and Bayesian mathematics—may indeed be generally accepted, but the FST uses each of those components in a manner that renders the tool improper for use in a jury trial. (See Doc. 299, at 4-5.) Defendant, however, is unable to point to any study, article, or presentation that confirms that the FST is not a proper tool. (See Tr. at 861-62.) Putting aside Defendant's inability to point to a specific study, "a slight modification of an otherwise reliable method will not render an expert's opinion per se inadmissible." Amorgianos, 303 F.3d at 267. Each of the assumptions incorporated into the FST—including allelic drop-out and drop-in rates—has been the subject of the exhaustive testing, validation, peer-review, accreditation, auditing, and other review processes described above. Moreover, the fact that the components of the FST—e.g., LR statistical analysis and Bayesian mathematics—are generally accepted militates in favor of a finding in this particular case that the FST is generally accepted.
As described above, the FST is a software program that OCME developed to examine DNA evidence and put quantitative weight to qualitative conclusions about that DNA evidence. (See supra Section I.B.) Notably, the underlying qualitative conclusions do not require the FST program and only require PCR/STR DNA testing. "[C]ourts have been virtually unanimous in finding that the use of [PCR/STR DNA] testing is admissible and many of these courts have taken judicial notice of the general reliability of such tests." United States v. Morrow, 374 F. Supp. 2d 51, 61 (D.D.C. 2005) (collecting cases and concluding that, "as a general matter, PCR/STR DNA testing meets the strictures of Daubert and is admissible"); see also Debraux, 21 N.Y.S.3d at 541 (explaining that the New York Supreme Court had previously ruled that "the FST is generally accepted as reliable in the forensic scientific community because it rests firmly upon [PCR/STR DNA testing] and the likelihood ratio," both of which "have long been generally accepted by forensic scientists as reliable" (internal quotation marks omitted)). Therefore, the underlying qualitative conclusion in this case—that Jones "cannot be ruled out as a contributor" to the blue latex glove, (GX 150A)—would be presented to the jury even if expert testimony on the FST was excluded in this case, and the admissibility of such testimony was not challenged under Daubert or otherwise.
In any event, cross-examination at trial is the more appropriate avenue for Defendant to mount his challenges to the underlying components of the FST. See Debraux, 21 N.Y.S.3d at 543 ("[T]he manner in which OCME incorporates the probability of stochastic effects, including allelic drop-in and drop-out rates, into the FST's use of Bayesian mathematical principles to calculate likelihood ratios of the presence of an individual's DNA in a mixed sample is properly a matter for consideration by the jury, as finder of fact at trial, and is not appropriate for resolution by this court . . . ."). Thus, to the extent there is any legitimate debate regarding these underlying assumptions of the FST, "the proper forum for the debate is before a jury, and it is for the jury to decide which of two conflicting experts' testimony to credit, and how much weight to give the evidence it accepts." Morgan, 53 F. Supp. 3d at 743.
C. Quant Versus Peak Height
Throughout the Hearing, Defendant consistently challenged one component of the FST more than any other: the FST's reliance on the use of quant—i.e., the amount of DNA in a sample—rather than allelic peak height to determine drop-out rates. (Doc. 16, at 16-17; Tr. at 858.) Defendant's premise—that the FST is the only forensic tool to use quant to determine drop-out rates—is undisputed by the Government. Nonetheless, Defendant's argument that OCME's use of quant is therefore unreliable fails.
As a factual matter, Dr. Mitchell testified that OCME's choice to use quant rather than allelic peak height was intentional. Specifically, Dr. Mitchell testified that OCME chose to use quant rather than allelic peak heights because OCME is an unusually large laboratory with a variety of different electrophoresis instruments. (Tr. at 111-12.) In creating and validating the FST, OCME found that peak heights tended to vary between those instruments, and that peak height was less useful than quant in predicting the probability of drop out across all instruments. (Id.) Dr. Mitchell—an expert in the field of human genetics, molecular biology, forensic science, and the statistical analysis thereof—and her team at OCME thus considered the use of peak height and rejected it after testing.
I find that OCME's use of quant to determine drop-out rates is insufficient to render expert testimony on the FST inadmissible. Defendant is correct that a number of other genotyping programs use peak height rather than quant to determine drop-out rates, and that the FST is the only such program to use quant to determine pre-set drop-out rates. However, the Second Circuit has explicitly held that the trial court "should only exclude the evidence if the flaw is large enough that the expert lacks good grounds for his or her conclusions." Amorgianos, 303 F.3d at 267 (internal quotation marks omitted). OCME's decision to use quant as one of the applicable factors in the FST was explicitly presented to the DNA Subcommittee, (Tr. at 112), and the DNA Subcommittee nonetheless voted unanimously to approve the use of the FST for forensic casework, (GX 9). The evidence in the record, on balance, supports that OCME's use of quant rather than peak height is not a flaw, and even if it could be considered a flaw, it is not large enough to exclude expert testimony on the FST in this case.
I note that Dr. Chakraborty, whose prior testimony in Collins was submitted by the parties at the Hearing in lieu of live testimony, was a member of the DNA Subcommittee that approved the FST. He was present for three of the four presentations that OCME made to the DNA Subcommittee, and he voted to approve the FST for use in criminal casework. (Collins Tr. at 1104-05.) He now contends that the FST is not generally accepted in the scientific community and that it should not be used in criminal casework, despite having had the authority himself to object to its approval. Whatever the reasons for the change of his opinion, Dr. Chakraborty's prior testimony does not raise concerns sufficient to render expert testimony on the FST unreliable in this case. "Collins Tr." refers to the transcript from the Frye hearing conducted in Collins in December 2013.
I also find that Defendant's concerns about OCME's use of quant go to the weight of FST evidence rather than the admissibility of such evidence. Even if Dr. Shapiro is correct that OCME should not have used quant to determine drop-out rates, it is not the role of the courts to weigh the credibility of competing scientific evidence. Such determinations should be left to a jury. See Highland Capital Mgmt., L.P. v. Schneider, 551 F. Supp. 2d 173, 180 (S.D.N.Y. 2008) ("The jury can best form a judgment when presented with experts' competing theories." (quoting In re Blech Sec. Litig., No. 94 Civ. 7696(RWS), 2003 WL 1610775, at *20 (S.D.N.Y. Mar. 26, 2003))).
D. Weight Versus Admissibility
Defendant repeatedly contends that his arguments speak to admissibility rather than weight, stressing that the FST is at its core far less reliable than traditional DNA evidence. He also contends that Dr. Shapiro's analysis of the FST evidence on the blue latex glove demonstrates that the FST should not have been applied to this case, as the evidence was not suitable for comparison under OCME's protocols. Based on the entirety of the record, I find that both of these arguments speak to the weight of the evidence produced by the FST, rather than the admissibility of that evidence.
In the DNA context, federal courts have considered the very types of arguments that Defendant advances here and have ruled that they address issues of weight, rather than admissibility. See, e.g., United States v. Gipson, 383 F.3d 689, 694-95 (8th Cir. 2004) (holding that challenge to use of improper DNA kit to create profiles went to weight, not admissibility); Fidelity Nat'l Title Ins. Co. of N.Y. v. Intercounty Nat'l Title Ins. Co., No. 00 C 5658, 2002 WL 1433717, at *10 (N.D. Ill. July 2, 2002) (concluding that the defendant's argument that the laboratory improperly used the wrong threshold in measuring the fluorescence of the alleles in a DNA profile went to weight, not admissibility); cf. McCullock v. H.B. Fuller Co., 61 F.3d 1038, 1044 (2d Cir. 1995) (explaining that alleged faults in methodology "go to the weight, not the admissibility," of expert testimony). The Morgan case is particularly informative on this issue. Although the Second Circuit in Morgan concluded that low-copy number DNA evidence was supported by "significantly weaker evidence of reliability than traditional DNA analysis," it nevertheless affirmed the district court's admission of the evidence under the Daubert standard, because "vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence." Morgan, 675 F. App'x at 55-56 (quoting Daubert, 509 U.S. at 596).
As is the case here, OCME was "the only publicly funded lab in the United States that perform[ed] LCN testing" at the time of the district court's decision in Morgan. Morgan, 53 F. Supp. 3d at 735. --------
Here, the FST has been rigorously tested and subjected to peer review. OCME performed validation studies of its methods, published those studies in a peer-reviewed journal, and the DNA Subcommittee approved the FST testing for use in criminal casework. To the extent that Defendant disagrees on how the FST was applied in this particular case, he can address those concerns at trial by putting on expert testimony and cross-examining witnesses, allowing the jury to make any such determination as to the application of the FST. Therefore, considering the record as a whole, I find that expert testimony on the FST in this case rests on a reliable foundation and is relevant to the task at hand. See Daubert, 509 U.S. at 597.
IV. Conclusion
The foregoing constitutes my findings and reasoning for my Order dated November 30, 2017, (Doc. 396), denying Defendant's motion to exclude evidence produced by the FST at trial, (Doc. 15). SO ORDERED. Dated: June 5, 2018
New York, New York
/s/_________
Vernon S. Broderick
United States District Judge