Opinion
No. 1:19-cr-02032-SMJ-1, 1:19-cr-02032-SMJ-2
2021-12-17
UNITED STATES of America, Plaintiff, v. James Dean CLOUD (01), and Donovan Quinn Carter Cloud (02), Defendants.
Richard Cassidy Burson, Thomas J. Hanlon, Assistant US Attorneys, U.S. Attorney's Office, Yakima, WA, for Plaintiff. John Barto McEntire, IV, Lorinda Meier Youngcourt, Public Defenders, Federal Defenders, Spokane, WA, Jeremy B. Sporn, Public Defender, Federal Defenders, Yakima, WA, for Defendant James Dean Cloud. Richard A. Smith, Smith Law Firm, Yakima, WA, Mark A. Larranaga, Walsh and Larranaga, Seattle, WA, for Defendant Donovan Quinn Carter Cloud.
Richard Cassidy Burson, Thomas J. Hanlon, Assistant US Attorneys, U.S. Attorney's Office, Yakima, WA, for Plaintiff.
John Barto McEntire, IV, Lorinda Meier Youngcourt, Public Defenders, Federal Defenders, Spokane, WA, Jeremy B. Sporn, Public Defender, Federal Defenders, Yakima, WA, for Defendant James Dean Cloud.
Richard A. Smith, Smith Law Firm, Yakima, WA, Mark A. Larranaga, Walsh and Larranaga, Seattle, WA, for Defendant Donovan Quinn Carter Cloud.
ORDER ON DEFENDANTS’ DAUBERT MOTIONS
SALVADOR MENDOZA, JR., United States District Judge Before the Court are James Cloud's Motion to Exclude or Limit Toolmark Identification Evidence, ECF No. 325, and Donovan Cloud's Motion to Exclude or Limit Expert Testimony on Friction Ridge Analysis, ECF No. 347. After review of the file and three days of hearings on the motions, the Court grants each motion in part. More specifically, the Court finds sufficient scientific support for the firearm comparison techniques at issue in this case but will limit the expert's testimony to reflect only a level of confidence supported by the appropriate validation studies. And although the Court generally finds sufficient scientific support to allow expert testimony purporting to compare fingerprints, the Court declines to qualify the Government's witness as an expert in fingerprint analysis.
BACKGROUND
A. The Shootings and Carjacking
On June 8, 2018, seven people were shot at or near a Medicine Valley Road residence in White Swan on the Yakima Nation Reservation. ECF No. 155 at 6. Only two of those persons survived their wounds. Id. A short time later, a family's truck was car jacked from their property on Evans Road, approximately seven miles from the Medicine Valley Road residence. Id. at 7. In the course of the carjacking, a teenage member of the family had a gun placed to his head and was forced into the back of the truck, but he escaped as the truck was being driven away. Id. at 8.
B. Investigation and Forensic Examination
In the course of its investigation, law enforcement came to suspect the Clouds’ involvement, and the two are now indicted on a slew of criminal counts stemming from the shootings and carjacking. See ECF No. 285. Relevant to these motions, law enforcement recovered several pieces of forensic evidence, including a Ruger .22 rifle, seventeen .22 cartridge casings, and a latent fingerprint. Each of these items were examined by FBI Forensic Examiners working in Quantico, Virginia.
1. Ballistics Evidence
The ballistics evidence was examined by Forensic Examiner Michael Van Arsdale ("FE Van Arsdale"), who compared the recovered casings against casings that he test-fired using the recovered Ruger .22 rifle. ECF No. 325-3. FE Van Arsdale examined the casings using the AFTE theory of identification—a comparison methodology named after the Association of Firearm and Toolmark Examiners. As a result, FE Van Arsdale "identified" twelve of the recovered casings "as having been fired in the [.22 rifle]." Id.
The AFTE methodology—the most common methodology used to compare toolmark evidence—has several steps. First, the examiner compares "the class characteristics of the shell casings found at the crime scene and the firearm being tested, including the caliber of both, the general types of marks on the casings (linear or circular), any other features determined by the manufacturer, and the operability of the firearm." United States v. Adams , 444 F. Supp. 3d 1248 (D. Or. 2020) (summarizing the AFTE methodology). "Class characteristics are gross features common to most if not all bullets and cartridge cases fired from a type of firearm, such as caliber and the number of lands and grooves on a bullet ...." United States v. Tibbs , No. 2016-CF1-19431, 2019 WL 4359486 at *2 (D.C. Super. Sep. 05, 2019) (cleaned up).
Second, the examiner test fires the gun being examined, allowing the "examiner to give his opinion as to the firearm's operability and provides the exemplars for comparison – i.e. the bullets and shell casings that can be compared to the suspected matches." Adams , 444 F. Supp. 3d at 1252–53.
Third, the examiner conducts a forensic comparison between the test-fired shells and the crime scene shells, typically using a comparison microscope that allows the examiner to view each cartridge case side-by-side in one field of vision, like a split screen. See id. at 1253. The images can then be magnified to varying degrees. Id. The examiner then searches for "impressed and striated toolmarks present on two cartridge cases to determine if patterns of similarity exist." ECF No. 325-3 at 5. Importantly, an examiner must work to distinguish subclass characteristics from individual characteristics. The subclass characteristics are those markings that "appear on a smaller subset of a particular make and model of firearm, such as a group of guns produced together at a particular place and time. They are produced incidental to manufacture, sometimes as the result of being manufactured by the same irregular tool." United States v. Shipp , 422 F. Supp. 3d 762, 779–80 (E.D.N.Y. 2019). Individual characteristics, on the other hand, are "microscopic markings produced during manufacture by the random and constantly-changing imperfections of tool surfaces as well as by subsequent use or damage to the firearm." Id. These markings are purported to be sufficiently unique such that an examiner would be unlikely to confuse subclass and individual characteristics and so unique as to permit an individualized source determination to the exclusion of all other firearms. Id.
Following the evaluation, the examiner expresses one of three opinions: (1) source exclusion, (2) source identification, or (3) inconclusive. Id. at 5–6; see also ECF No. 220-5. Finally, after FE Van Arsdale reached his source identification conclusion, a qualified examiner reviewed the items, conducted their own comparison, and verified FE Van Arsdale's findings. See ECF No. 419 at 66.
A "[s]ource identification is an [e]xaminer's conclusion that two bullets did not originate from the same source," and is based on the examiner's decision that the two bullets or cartridge casings "can be differentiated by their class characteristics." ECF No. 220-5 at 4. A "[s]ource identification is an [e]xaminer's conclusion that two bullets originated from the same source." Id. An "inconclusive" determination represents the examiner's "conclusion that all observed class characteristics are in agreement but there is insufficient quality and quantity of corresponding individual characteristics such that the [e]xaminer is unable to identify or exclude the two bullets as having originated from the same source." Id. at 5.
2. Fingerprint Evidence
FBI forensic examiner Krystal N. Watts compared a recovered "latent print" (a complete or partial friction-ridge impression from an unknown subject) to Donovan's known fingerprint (fingerprints deliberately collected under a controlled setting from known subjects) and was "fully confident" "that the overall degree of correspondence and concomitant lack of discordance between the latent and known prints would not be observed if the prints had originated from different sources." ECF No. 347 at 14–15. Rather than express her identification in terms of ground truth or absolute certainty, FE Watts intends to assert there is "extremely strong support for the proposition that the two prints came from the same source and extremely weak support for the proposition that the two prints came from different sources." Id. at 15. Another examiner independently verified the comparison. ECF No. 430 at 68.
To perform this analysis, both examiners used the Analysis, Comparison, Evaluation, and Verification ("ACE-V") method, which requires an examiner to make a "series of subjective assessments." Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods , PCAST at 89, https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf (hereinafter "PCAST Report"). Using her subjective judgment, an examiner selects particular regions of a latent print to analyze. Id. Where, as here, a suspect's known print is available, the examiner "manually compares the latent print to the fingerprints of the specific person of interest" "and then comes to a subjective decision as to whether they are similar enough to declare a proposed identification." Id. Finally, the opinion is verified by another examiner. Id.
Fingerprint features are compared at three levels of detail. PCAST Report at 89 n.253 (citation omitted). Level one is referred to as "ridge flow" and "refers to classes of pattern types shared by many individuals, such as loop or whorl formations; this level is only sufficient for exclusions, not for declaring identifications." Id. At level two, " ‘[r]idge path’ refers to minutiae that can be used for declaring identifications, such as bifurcations or dots." Id. Finally, at level three, " ‘[r]idge shapes’ include the edges of ridges and location of pores." Id.
C. Noteworthy Developments in Forensic Science
For much of the past century, forensic sciences, including toolmark evaluation and fingerprint comparison, have been widely accepted in and out of court. More recently, owing in part to the emergence of DNA analysis and the resulting exonerations, reports and studies have called into question these forensic disciplines. In 2009, the National Research Council, a division of the National Academy of Science, issued a report critical of the state of the forensic sciences, including toolmark and fingerprint comparison. See Strengthening Forensic Science in the United States: A Path Forward , Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council at 142–45, 154–55 (Aug. 2009), https://www.ojp.gov/pdffiles1/nij/grants/228091.pdf (hereinafter "NRC Report").
Then, in 2016, the President's Council of Advisors on Science and Technology, or "PCAST," issued another report on the state of forensic sciences, aimed at identifying any "additional steps on the scientific side, beyond those already taken" in the wake of the 2009 National Research Council report, "that could help ensure the validity of forensic evidence used in the Nation's legal system." PCAST Report at x. The PCAST Report identified "two important gaps: (1) the need for clarity about the scientific standards for the validity and reliability of forensic methods and (2) the need to evaluate specific forensic methods to determine whether they have been scientifically established to be valid and reliable." Id.
Thus, PCAST coined "foundational validity" for the purpose of helping evaluate from a scientific perspective the requirement under Rule 702(c) that testimony be the product of reliable principles and methods. Id. at 42. According to PCAST, foundational validity requires that a forensic feature-comparison method be "a reproducible and consistent procedure for (i) identifying features in evidence samples; (ii) comparing the features in two samples; and (iii) determining, based on the similarity between the features in two sets of features, whether the samples should be declared to be likely to come from the same source." PCAST Report at 65. Moreover, there must be empirical estimates, from appropriately designed studies from multiple groups, that establish (i) the method's false positive rate ... and (ii) the method's sensitivity—that is, the probability it declares a proposed identification between samples that actually come from the same source." Id. at 64–65.
Importantly, PCAST also put forth a set of criteria for what it considers proper scientific validation studies: "(a) they should be based on sufficiently large collections of known and representative samples from relevant populations; (b) they should be conducted so that the examinees have no information about the correct answer; (c) the study design and analysis plan should be specified in advance and not modified afterwards based on the results; (d) the study should be conducted or overseen by individuals or organizations with no stake in the outcome; (e) data, software and results should be available to allow other scientists to review the conclusions; and (f) to ensure that the results are robust and reproducible, there should be multiple independent studies by separate groups reaching similar conclusions." Id. at 78–79. For subjective forensic methods, such as fingerprint and toolmark analysis, foundational validity "can be established only through empirical studies of examiner's performance to determine whether they can provide accurate answers; such studies are referred to as ‘black-box’ studies." Id. at 49. Establishing foundational validity of an objective method, on the other hand, would require measuring the accuracy, reproducibility, and consistency of each of the method's steps. Id.
Once a method has been established as foundationally valid, any expert's claims about the method's accuracy and probative value must be based on the empirical studies to be valid. Id. at 66. The expert must also, of course, be shown to be capable of reliably applying the method and to have reliably applied the methodology. Id. (describing "validity as applied").
PCAST evaluated a number of "feature-comparison" methods, including latent fingerprint and toolmark analysis, to determine whether the methods had foundational validity, to determine what was required to satisfy validity as applied, and to suggest next steps to improve the practices. Id. Regarding toolmark evidence, PCAST found a lack of foundational validity, finding only a single "black box" study capable of estimating the subjective methodology's error rate. PCAST Report at 11. Finding that a single study is insufficient for establishing foundational validity, PCAST also noted that the work was commissioned and funded by the Defense Department's Forensic Science Center, was conducted by the Ames Laboratory, a Department of Energy national laboratory affiliated with Iowa State University, and reported an estimated false-positive rate of 1 in 66, with a confidence bound indicating that rate could be as high as 1 in 46. Id.
PCAST found, based on the limited number of studies, that latent fingerprint analysis is a "foundationally valid subjective methodology." Id. at 9. Still, it pushed for the field to make the practice objective rather than subjective and noted that the false positive rate is "substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis." PCAST Report at 9–11. Others, including James Cloud's expert witness William A. Tobin, a metallurgist and former Manager of Forensic Metallurgy operations at the FBI Laboratory, are even harsher in their condemnation of forensic practices. For example, Tobin testified that key premises underlying the AFTE theory of identification—uniqueness, the premise that no other items would produce the same marks on the cartridge being evaluated; discernible uniqueness, the premise that a human can observe and discern those unique markings; and repeatability, the premise that the purportedly unique markings made by a firearm persist over time such that a recovered cartridge or bullet could be compared to a test-fired round at a later time—are all unproven assumptions or presumptions. ECF No. 419 at 217–19, 222–24, 239–42. Reflecting on this last premise, Tobin also noted what he called an "irrational dichotomy" in the theory, which is that the markings made by the manufacturing tools change over time so that consecutively manufactured firearms are distinguishable, but that the markings made on the cartridge by the firearm do not dramatically change over time. ECF No. 419 at 270–71.
LEGAL STANDARD
In Daubert v. Merrell Dow Pharmaceuticals, Inc. , 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993), the Supreme Court cast trial judges in the role of gatekeepers in determining whether to admit or exclude expert evidence in accordance with Federal Rule of Evidence 702. The Daubert Court "held that Federal Rule of Evidence 702 commands the primary focus for courts evaluating the admissibility of expert testimony." Cooper v. Brown , 510 F.3d 870, 942 (9th Cir. 2007). That Rule provides in part that "[i]f scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact issue," an expert "may testify thereto." Fed. R. Evid. 702. Before a witness may come "before the jury cloaked with the mantle of an expert[ ]" under Rule 702, the Ninth Circuit has cautioned that "care must be taken to assure that a proffered witness truly qualifies as an expert, and that such testimony meets the requirements of [that] Rule." Jinro America Inc. v. Secure Investments, Inc. , 266 F.3d 993, 1004 (9th Cir. 2001). Thus, as a threshold matter, in accordance with Rule 702 the court must determine whether the proffered witness is "qualified as an expert by knowledge, skill, experience, training, or education." Fed. R. Evid. 702 ; see also Daubert v. Merrell Dow Pharm., Inc. , 43 F.3d 1311, 1315 (9th Cir. 1995) (" Daubert II ") (whether proffered expert testimony is admissible "only arises if it is first established that the individual[ ] whose testimony is being proffered [is an] expert[ ] in a particular ... field[ ]").
Once a court makes the "preliminary" determination under Rule of Evidence 702(a) that a witness qualifies as an expert, the focus shifts to that witness’ proffered testimony. Rule 702 permits a witness to give expert testimony if "(1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case." Fed. R. Evid. 702. As interpreted by the Daubert Court, there is a two-part inquiry under Rule 702 for determining the admissibility of proffered expert opinion testimony. Expert testimony must be "not only relevant, but reliable." Daubert , 509 U.S. at 589, 113 S.Ct. 2786. This requires consideration of whether:
(1) the reasoning or methodology underlying the testimony is scientifically valid (the reliability prong); and
(2) whether that reasoning or methodology properly can be applied to the facts in issue (the relevancy prong).
Daubert , 509 U.S. at 592–93, 113 S.Ct. 2786.
1. Reliability
In determining reliability, a court may consider a number of factors including: (1) whether the theory can be and has been tested; (2) whether it has been subjected to peer review; (3) the known or potential rate of error; (4) the existence and maintenance of standards controlling the technique's operation; and (5) whether the theory or methodology employed is generally accepted in the relevant scientific community. Id. at 593–94, 113 S.Ct. 2786. But " Daubert's list of specific factors neither necessarily nor exclusively applies to all experts or in every case." Kumho Tire Co. v. Carmichael , 526 U.S. 137, 141, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999). "[F]ar from requiring trial judges to mechanically apply the Daubert factors ... judges are entitled to broad discretion when discharging their gatekeeping function." United States v. Hankey , 203 F.3d 1160, 1168 (9th Cir. 2000) (citing Kumho , 526 U.S. at 150–52, 119 S.Ct. 1167 ).
The court need not admit an expert opinion that is connected to the underlying data "only by the ipse dixit of the expert." Gen. Elec. Co. v. Joiner , 522 U.S. 136, 146, 118 S.Ct. 512, 139 L.Ed.2d 508 (1997). It may exclude such testimony if it determines "that there is simply too great an analytical gap between the data and the opinion proffered." Id. "The trial court's gate-keeping function requires more than simply taking the expert's word for it." Daubert II , 43 F.3d at 1319. In addition, "any step that renders [the expert's] analysis unreliable ... renders the expert's testimony inadmissible. This is true whether the step completely changes a reliable methodology or merely misapplies that methodology." In re Silicone Gel Breast Implants Prods. Liab. Lit. , 318 F. Supp. 2d 879, 890 (C.D. Cal. 2004). Something does not become scientific knowledge just because it is uttered by a scientist; nor can an expert's self-serving assertion that his conclusions were derived by the scientific method be deemed conclusive. Daubert II , 43 F.3d at 1315–16. "[T]he expert's bald assurance of validity is not enough. Rather, the party presenting the expert must show that the expert's findings are based on sound science, and this will require some objective, independent validation of the expert's methodology." Id. at 1316.
2. Relevance
Evidence is relevant under Daubert if the evidence will assist the trier of fact to understand or determine a fact in issue. Daubert , 509 U.S. at 591–92, 113 S.Ct. 2786. "The gatekeeping inquiry must be ‘tied to the facts’ of a particular ‘case.’ " Kumho , 526 U.S. at 150, 119 S.Ct. 1167 (quoting Daubert , 509 U.S. at 591, 113 S.Ct. 2786 ). "Encompassed in the determination of whether expert testimony is relevant is whether it is helpful to the jury, which is the ‘central concern’ of Rule 702." Mukhtar v. California State University, Hayward , 299 F.3d 1053, 1063 n.7 (9th Cir. 2002) (citation omitted). Rule 702 states that "[i]f scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue," an expert "may testify thereto."
3. Burden of Proof
"It is the proponent of the expert who has the burden of proving admissibility." Lust v. Merrell Dow Pharmaceuticals, Inc. , 89 F.3d 594, 598 (9th Cir. 1996). Admissibility of the expert's proposed testimony must be established by a preponderance of the evidence. See Daubert , 509 U.S. at 592 n.10, 113 S.Ct. 2786 (citing Bourjaily v. United States , 483 U.S. 171, 175–76, 107 S.Ct. 2775, 97 L.Ed.2d 144 (1987) ). The party presenting the expert must demonstrate that the expert's findings are based on sound principles and that they are capable of independent validation. Daubert II , 43 F.3d at 1316.
DISCUSSION
James and Donovan Cloud each bring a motion to exclude or limit forensic evidence testimony. ECF Nos. 325, 347. James Cloud challenges the toolmark comparison testimony; Donovan Cloud challenges both the qualifications of the Government's expert, Forensic Examiner Krystal N. Watts ("FE Watts"), and the reliability of the methodology she used. Id. Each Defendant also brings a "validity as applied" challenge. Id.
A. James Cloud's Motion to Exclude of Limit Toolmark Testimony
James Cloud moves to exclude or limit testimony regarding the recovered cartridges and the recovered Ruger .22 rifle. ECF No. 325 at 3. As the Government has proffered only FBI Forensic Examiner Michael Van Arsdale as an expert, the Court will focus on his findings and proposed testimony.
1. Forensic Examiner Van Arsdale is Qualified as an Expert
As a threshold matter, the Court finds that FE Van Arsdale is qualified as an expert in the forensic examination and comparison of firearms/toolmarks. Briefly, FE Van Arsdale has a Bachelor of Science from the University of Michigan, has worked in some capacity with the FBI Laboratory since 1997, and has worked as a Forensic Examiner in the FBI Laboratory since 2015. ECF No. 220-4. Moreover, FE Van Arsdale has completed extensive trainings related to his field of practice and has published three articles, all published in the Association of Firearms and Toolmark Examiners’ Journal ("AFTE Journal"). Id. As such, the Court is convinced FE Van Arsdale has the knowledge, skill, experience, training, and education to testify as an expert on firearms/toolmark analysis and comparison within the bounds of this Order.
2. The AFTE Methodology is Reliable
As explained, the Court considers a non-exhaustive list of five factors to help determine whether the reasoning or methodology underlying the proposed testimony is scientifically valid/reliable: (1) whether the methodology is testable; (2) whether it has been subjected to peer review; (3) the known or potential rate of error; (4) the existence and maintenance of standards controlling the technique's operation; and (5) whether the theory or methodology employed is generally accepted in the relevant scientific community. Daubert , 509 U.S. at 593–94, 113 S.Ct. 2786.
i. Testability
Testability comprises two parts: falsifiability and replicability. Id. at 593–94, 113 S.Ct. 2786. Falsifiability asks whether a proposition can be subjected to an observation or test that renders the proposition false. Replicability asks whether someone else could repeat the exact methodology the expert used. As detailed below, the AFTE methodology struggles with the second prong—so this factor weighs slightly against the Government.
a. Falsifiability
The proposition that a particular shell casing was fired from a particular gun is a falsifiable proposition. It is possible to observe characteristics in a test-fired casing that differ greatly from a recovered round such that one could conclude that the two casings could not have been fired in the same gun. As such, one would be able to reject the proposition that the AFTE methodology allows forensic examiners to determine with accuracy whether two cartridge casings were fired and ejected from the same firearm.
b. Replicability
The AFTE comparison methodology is only replicable to a point. The methodology includes a number of subjective steps and practices that would be familiar to anyone working within a laboratory. But "[i]n order to be replicable, a method must be objective enough that someone else not associated with the case could duplicate it and get the same results." Adams 444 F. Supp. 3d at 1260. Using the AFTE methodology, examiners use the "best known non-match" as a baseline comparator. See ECF No. 419 at 77–78. Thus, when looking at the split-screen image produced by the comparison microscope, the examiners compare the degree of correspondence on the image against the best possible non-match comparison that the examiner is aware of between consecutively manufactured firearms. But these baselines are not standardized, and examiners hold these baselines in their heads, meaning that subsequent examiners could evaluate the same split-screen image with a different baseline in mind. Whether the agreement observed during a comparison exceeds the baseline is, as FE Van Arsdale admits, "a subjective determination." Id. at 124.
The most troubling aspect of the methodology is the tautology at its heart. "AFTE requires an examiner to find ‘sufficient agreement’ between crime scene shells and test fired shells from the firearm in question in order to determine a match." Adams 444 F. Supp. 3d at 1261. Judge Garaufis of the Eastern District of New York explains why "sufficient agreement" is not an objective standard:
First, the sufficient agreement standard is circular and subjective. Reduced to its simplest terms, the AFTE Theory "declares that an examiner may state that two toolmarks have a ‘common origin’ when their features are in ‘sufficient agreement.’ " PCAST Report at 60. "It then defines ‘sufficient agreement’ as occurring when the examiner considers it a ‘practical impossibility’ that the toolmarks have different origins." Id. The NRC Report notes that the AFTE Theory "is the best guidance available for the field of toolmark identification, [but] does not even consider, let alone address, questions regarding variability, reliability, repeatability, or the number of correlations needed to achieve a given degree of confidence." NRC Report at 155. Without guidance as to the extent of commonality necessary to find "sufficient agreement," the AFTE Theory instructs examiners to draw identification conclusions from what is essentially a hunch—a hunch "based on the examiner's training and experience," AFTE Revised Theory of Identification , 43 AFTE Journal at 287—but still a hunch.
Moreover, the application of this circular standard is "subjective in nature ... based on the examiner's training and experience." AFTE Revised Theory of Identification , 43 AFTE Journal at 287. Ostensibly, one hundred firearms toolmark examiners could hold one hundred different personal standards of when two sets of toolmarks sufficiently agree, and all one hundred of these personal standards may accord with the AFTE Theory. Further, because the standard itself offers so little guidance on when an examiner should make an identification determination, some examiners may decide that the two sets of toolmarks were made by the same tool while others determine the toolmarks to be inconclusive and still others decide the toolmarks were made by different tools. To emphasize, these one hundred examiners
could come to these contradictory conclusions without a single examiner running afoul of the AFTE Theory.
United States v. Shipp , 422 F. Supp. 3d 762, 779–80 (E.D.N.Y. 2019).
As such, the AFTE methodology is not entirely replicable. Instead, those looking in must rely on validation studies of the methodology that attempt to measure the accuracy of technicians applying a vague standard that ultimately comes down to their training and experience. This is truly an unforced error, and forensic science would be improved by a concerted effort to reframe the methodology's procedures in concrete, objective terms. This factor weighs against admissibility under Daubert .
ii. Peer Review and Publication
The second of the Daubert factors, which considers whether the theory or technique "has been subjected to peer review and publication," weighs slightly against the Government. Daubert , 509 U.S. at 593–94, 113 S.Ct. 2786. As the Supreme Court emphasized in Daubert , "submission to the scrutiny of the scientific community is a component of ‘good science,’ in part because it increases the likelihood that substantive flaws in methodology will be detected." 509 U.S. at 593, 113 S.Ct. 2786. While the existence of peer reviewed literature can help determine a methodology's reliability under Daubert , the "fact of publication (or lack thereof) in a peer reviewed journal" is not dispositive. Id.
Here, the Court is aware of a number of studies, published in a variety of fora, but the Court is also troubled by the field's initial reticence to open itself up to criticism and debate. As has been noted at greater length in other decisions, the vast majority of studies on which examiners rely have been published in the AFTE Journal , a trade publication produced by the Association of Firearm and Toolmark Examiners. And while many courts have given weight to the AFTE Journal and the articles it publishes, the Court finds the justifications for doing so lacking. See Shipp , 422 F. Supp. 3d at 776 (discussing prior decisions finding the peer review factor weighs in favor of a finding of admissibility). Instead, the Court joins other recent decisions finding the AFTE Journal's bona fides as a scientific publication lacking and therefore of little weight under this Daubert prong. See, e.g., Adams , 444 F. Supp. 3d at 1265–66 ; Tibbs , 2019 WL 4359486 at *8–10.
Specifically, the Court is concerned that articles submitted to the AFTE Journal are reviewed by AFTE members and were, until recently, subject to an "open" pre-publication peer review process, meaning the author and reviewers knew one another's identities and are free to communicate directly during the review period. Tibbs , 2019 WL 4359486 at *8. Meanwhile, the AFTE Journal was "publicly" available only to those with a paid subscription or AFTE membership. Id. at *9. As a consequence, unlike other scientific journals, the AFTE Journal was not made available using free, publicly available search engines or indexed in databases made available through academic libraries. Id. "These limitations significantly diminish[ed] the stringency of the review that a study published in the AFTE Journal can be said to have undergone, even after its publication." Id.
For what it is worth, the Court notes that the AFTE Journal's articles’ abstracts and keywords were available to search through an index housed on AFTE's website. See Tibbs , 2019 WL 4359486 at *9 n.6.
Recently, possibly in response to criticism from the likes of Judge Edelman in Tibbs , AFTE has seemingly amended some of its practices. See AFTE Peer Review Process – January 2020 , AFTE, https://afte.org/afte-journal/afte-journal-peer-review-process (last visited Dec. 15, 2021). Now, AFTE uses a double-blind peer review "[w]here applicable," What Is The AFTE Journal? , AFTE, https://afte.org/afte-journal/what-is-the-journal (last visited Dec. 15, 2021), though it appears from the journal's website that non-AFTE members are not, as a matter of course, asked to review submitted manuscripts. See AFTE Peer Review Process – January 2020 (describing the peer review process and noting that "[d]uring the primary review process, the article may be forwarded to a subject matter expert outside of the Editorial Review Panel"). As such, the AFTE Journal peer review process still falls short. As explained more fully in Tibbs , members of the Review Panel, though likely trained and experienced professionals capable of firearms and toolmark examination, may not be experts in research design and methodology. Tibbs , 2019 WL 4359486 at *10. Moreover, and most importantly, these reviewers likely "have a vested, career-based interest in publishing studies that validate their own field and methodologies." Id.
The Court applauds the publication's changes and encourages AFTE and similar organizations to continue to open their publications up for criticism and review from the larger scientific community if they wish to meet Daubert's rigorous standard. The Court also encourages authors to take the initiative and publish their findings in journals geared towards the broader scientific community, opening themselves up to tougher lines of criticism even if their fields’ journals will not meet the moment. But the Court ultimately considers this factor to weigh slightly against the Government because of the NRC Reports, the PCAST Report, and subsequent validation studies, discussed below, that were either publicly published online or else were published in more reputable journals, such as the Journal of Forensic Sciences.
The Court is aware that it may seem perverse to consider the NRC Reports and PCAST Report, all of which were critical of the firearm/toolmark comparisons, as weighing in favor of the Government. But the peer review and publication processes have value beyond simply catching mistakes before they reach print; it also allows room for iterative criticism and improvement. As such, the critical reports carry weight in documenting both where groups of experts thought the field fell short and how the field has responded to those well-considered criticisms.
iii. The Known or Potential Rate of Error
This factor weighs strongly in the Government's favor. This factor is also the most important because, without appropriate estimates of accuracy, an examiner's subjective opinion that two items are similar, even if based on years of training and experience, is "scientifically meaningless." PCAST Report at 46. For that reason, the presence of multiple, appropriately designed, "black-box" studies is essential.
Appropriately designed black-box studies must have "open," rather than "closed" or "set-based" study designs. See PCAST Report at 106–10. Open studies ask examiners to determine whether a given unknown sample matches a specified sample. Id. at 10. Each problem is completely independent of the next. But almost all published studies, until recently used a "closed" design "where a match is always present for each unknown sample, and a ‘set-based’ design, where comparisons are made within a set of samples." Tibbs, 2019 WL 4359486 at *15 (citing PCAST Report at 106). Such studies lack external validity, as examiners conducting real-world comparisons have neither the luxury of knowing a true match is somewhere in front of them nor of making process-of-elimination-type inferences to reach their conclusions. Accordingly, the Court gives almost no weight to such studies and focuses instead on open black-box studies.
The Court is aware of four studies that meet these criteria. First, in the "Ames Laboratory Study," which was published online and endorsed by the PCAST Report, reported an estimated error rate of 1.51% (1 in 66), though the upper 95 percent one-sided confidence bound reflected the error rate could reasonably be as high as 2.2% (1 in 46). David P. Baldwin et al., A Study of False-Positive and False-negative Error Rates in Cartridge Case Comparisons (May 2016), https://www.ojp.gov/pdffiles1/nij/249874.pdf; PCAST Report at 124. Second, in the "Keisler Study" the observed error rate was 0.0%, though the study did not produce a confidence interval due to "non-normal distributions observed in reported data," and so the Court gives this study the least weight. Mark A. Keisler et al., Isolated Pairs Research Study , 50 AFTE J. 56, 58 (2018). Third, in the "Smith" study the error rate was estimated as 0.08% percent (1 in 1250), though reasonably as high as 0.5 percent (1 in 200). Fourth, the "Ames/FBI Study," pending publication, reports an estimated false positive rate of 0.933 percent (1 in 107), though reasonably as high as 1.57 percent (1 in 67). Finally, the Court also notes that, in the partly open "Miami-Dade" study, the estimated error rate was 2.0% (1 in 49), though reasonably as high as 4.7% (1 in 21). Thomas G. Fadul, Jr. et al., An Empirical Study to Improve the Scientific Foundation of Forensic Firearm and Tool Mark Identification Utilizing Consecutively Manufactured Glock EBIS barrels with the Same EBIS pattern , National Institute of Justice Grant #2010-DN-BX-K269 (Dec. 2013), www.ncjrs.gov/pdffiles1/nij/grants/244232.pdf.
In short, when the researchers conduct their studies, they necessarily use only a finite number of samples. PCAST Report at 51. For that reason, the error rates produced are necessarily estimates. Id. The confidence bound, a range that we can be 95 percent certain contains the true value, is therefore calculated to estimate how high the error rate could reasonably be. Id. These are the values that should be given to the jury, not the estimated error rate or the actual error rate in a given study's sample. Id.
These estimates utilize corrected data, as the Smith Study researchers found that five of the recorded errors were administrative in nature. Smith Study at 552. Although the PCAST Report counseled against using corrected data in validation studies, the Court is satisfied with the study's explanation and determination and accepts the corrected data.
But even these "open" studies have their drawbacks. The Smith Study, for example, uses corrected data, while the Keisler Study administered the tests to only 34 examiners, all AFTE members, and was published by the AFTE Journal. And the FBI study, seemingly the most comprehensive to date, is still pending publication and peer review. The most troubling concern, though, is that the studies allow the examiners to answer "inconclusive," even though the examiners know they are being tested. To be sure, incorrectly selecting "inconclusive" (a false-negative or Type-II error) in the field has little to no implications on the Daubert analysis: incorrect exclusions increase the likelihood that a guilty party is acquitted and therefore typically benefit the party objecting to proffered expert testimony. But providing examiners in the study setting the option to essentially "pass" on a question, when the reality is that there is a correct answer—the casing either was or was not fired from the reference firearm—fundamentally undermines the study's analysis of the methodology's foundational validity and that of the error rate. Given that the items to be examined are recovered in a laboratory setting, the high "inconclusive" response rate is at least notable, even if some samples might in fact have an insufficient number of features from which a conclusive determination can be made. Whether or not inconclusive responses should be permitted in a study is thus a close question, and one researchers should consider revisiting upon designing a new study.
Nevertheless, the Court concludes that there is now, with the studies conducted since the PCAST Report's publication, sufficient support to establish a known error rate for comparison using the AFTE methodology. As such, this factor weighs in favor of the Government.
iv. The Existence and Maintenance of Standards Controlling the Technique's Operation
Here, the operative standard is the AFTE methodology. As explained already, the AFTE methodology is highly subjective or else tautological. As such, the methodology does little to cabin a given examiner's analysis and "provides no objective yardstick to support or explicate the expert's opinion; instead, the expert is left to rely on her own thoughts and conclusions based only on the vagaries of her own training and experience." Tibbs , 2019 WL 4359486 at *21. This high level of subjectivity is, moreover, greater than the level of subjectivity accepted when, for example, a medical expert testifies as to whether a doctor met a certain accepted standard of care and also distinct from the kind of testimony a psychologist might give on a question such as a defendant's competence, which is an inherently ambiguous question. Shipp, 422 F. Supp. 3d at 780. This Daubert factor therefore weighs against the Government and against admissibility.
v. General Acceptance Within the Relevant Scientific Community
Finally, the Court turns to the level of acceptance within the relevant scientific community. The dispositive question here is how to best define the relevant scientific community. If the relevant community is limited to forensic examiners, the factor weighs in favor of the Government, as the theory of identification and source attribution is widely accepted within the community of people who purport to make such source identifications. See id. at *21 (noting that courts finding that AFTE theory has achieved general acceptance "have generally limited the scope of the so-called ‘relevant community’ to the specific community of firearms and toolmark examiners").
Doing so makes little sense though, for much the same reasons as reliance on the AFTE Journal does little to bolster the assertion that the methodology has undergone sufficient peer review. As James Cloud succinctly notes, adapting a Judge Rakoff quote, "Astrologers believe in the legitimacy of astrology.... And toolmark analysts believe in the reliability of firearms identification; their livelihoods depend on it." ECF No. 325 at 43.
The Court further notes that this "widespread acceptance within the law enforcement community may [also] have created a feedback loop that has inhibited the AFTE method from being further developed." Adams , 444 F. Supp. 3d at 1266. Finally, given that this methodology is solely used in the law enforcement context, where even good-faith actors are under pressure and subject to confirmation bias, the AFTE methodology may be subject to a limited "degree of intellectual rigor and detachment that counts as neutral scientific expertise." New York v. Ross , 68 Misc. 3d 899, 913, 129 N.Y.S.3d 629 (2020) (applying Frye standard) (citing Shipp , 422 F. Supp. 3d 762 ).
As such, the Court considers not only the acceptance within the narrower forensic sciences community but also the criticism from the broader scientific community. Most significantly, the Court gives weight to the PCAST Report's finding that toolmark comparisons lack foundational validity, but is ultimately convinced that the subsequent validations studies have addressed the most significant of the PCAST Report's concerns. The Court finds that this factor weighs slightly against the Government.
vi. FE Van Arsdale Properly Applied the AFTE Methodology
In his closing brief, James Cloud argues that the Government failed to elicit sufficient testimony at the hearing to establish that FE Van Arsdale reliably applied the AFTE methodology. ECF No. 425 at 12–14. The Court will reserve decision on this issue, pending further testimony from FE Van Arsdale at an upcoming hearing in this matter.
3. Limitations on FE Van Arsdale's Testimony
For these reasons, the Court concludes that FE Van Arsdale may testify regarding his analysis and comparison of the recovered cartridge casings to those fired from the recovered Ruger rifle. However, given the concerns outlined above, if FE Van Arsdale intends to go beyond testimony that merely notes the recovered cartridge casings could not be excluded as having been fired from the recovered firearm, the Court will inform the jury that: (1) only three studies that meet the minimum design standard have attempted to measure the accuracy of firearm/toolmark comparison and (2) these studies found false positive rates that could be as high as 1 in 46 in one study, 1 in 200 in the second study, and 1 in 67 in the third study, though this study has yet to be published and subjected to peer review.
B. The Fingerprint and Friction Ridge Analysis
Here, Donovan seeks to exclude or limit the friction ridge testimony on the basis that FE Watts should not be qualified as an expert, the ACE-V methodology she used fails Daubert's reliability prong, and because, even if ACE-V is reliable, FE Watts failed to validly apply the methodology. ECF No. 347. The Court agrees that, at this time, FE Watts is not qualified to testify as an expert. Given this, the Court need not address whether she validly applied the ACE-V methodology, though the Court analyzes the methodology itself and finds it reliable under Daubert .
1. Forensic Examiner Watts is Not Qualified as an Expert
Because Donovan Cloud objects to FE Watts being designated an expert, the Court must determine, as a threshold matter, whether FE Watts is "qualified as an expert by knowledge, skill, experience, training, or education." Fed. R. Evid. 702 ; see also Daubert II , 43 F.3d at 1315 (9th Cir. 1995). For the reasons set forth below, and despite the fact that FE Watts has previously been qualified by courts presiding over other cases, the Court finds FE Watts insufficiently qualified to testify as an expert in this case.
The FBI has employed FE Watts as a forensic examiner in their Latent Print Operations Unit at the FBI Laboratory in Quantico, Virginia since 2013. ECF No. 430 at 10. Prior to her current employment, FE Watts obtained an Associate of Arts and Science degree from Columbia Basin College, a Bachelor of Science in biochemistry and molecular biology from Washington State University, and a Master of Arts in forensic psychology from Marymount University. Id. at 10–11. She also worked as an intern with the University Police Department at Washington State University and with the FBI Laboratory's Latent Print Operations Unit. Id. at 12–13.
As a part of her training with the FBI, FE Watts underwent an 18-month training program. Id. at 14. The program was divided into three six-month segments. Id. The first segment consisted mainly of classroom instruction Id. at 14–15. The second segment required FE Watts to perform casework "under the guidance of a qualified examiner already employed within the [Latent Print Unit]". Id. at 15. In the third segment, FE Watts continued to perform casework, this time under the guidance of a different qualified examiner. Id. Throughout this training period, FE Watts underwent a variety of exercises and exams, some of which focused on the comparison methodology and some of which concerned other skillsets, such as in-court testimony. Id. at 15–16. At the end of her training period, FE Watts successfully completed, without error, a three-day examination and was qualified as an examiner. Id.
In recent years, FE Watts has made a series of errors in her work—some significant and some less so—which bear on her qualifications. The Court discusses the more significant incidents here. First, in July 2019, FE Watts, working as blind verifier of another examiner's comparison analysis, falsely excluded a palm print, even though the initial examiner had determined the palm print was an identification—a match with the known print. Id. at 147. No Corrective Action Report was issued as a result of this error, which FE Watts characterized as her "first and only error that [she] had made in casework." Id. The same month, FE Watts conducted her examination of the latent at issue in this case. See ECF 220-2.
FE Watts also made an error with the use of reagent. Id. at 148. In brief, while "processing a pistol," FE Watts observed a reddish residue, which she believed may have been blood. Id. In an effort to "increase contrast of the residue against the backdrop of the pistol" FE Watts, in addition to all the standard processing techniques, decided to use a liquid chemical known as Amido Black. Id. The reagent, once mixed, has an indefinite shelf life. Id. Even so, laboratory protocol requires the examiners to perform a reagent check, essentially confirming the reagent is working as intended. Id. at 148–49. FE Watts failed to perform the reagent check, which resulted in the placement of a "Concession" in that case's file. Id. at 148–50.
Extremely telling, in the Fall of 2020, FE Watts failed her annual proficiency test, falsely excluding one latent print as a match. See id. at 17. This proficiency test, administered by an external agency, was a requisite component of the FBI maintaining its accreditation. Id. at 16. For each test, examiners are given twelve latent prints and four subjects against which to compare those latents. Id. at 17–18. Examiners are "usually given about a month" to complete the exams. Id. at 18.
FE Watts attributes the error in large part to rushing to complete the exam—she completed the exam in "about three to four hours"—and also to the stress she was experiencing at the time. Id. at 18. As a result of the failed proficiency test, the Forensic Analysis Support Unit issued a Corrective Action Report and FE Watts was removed from comparison casework for "approximately a month and a half," and her work from the past year was audited for any other errors. Id. at 19, 145. The case review did not reveal any additional errors. Id. at 20.
Before returning to work, FE Watts was required to take a second proficiency exam, which she passed without making any mistakes. Id. But even on this exam FE Watts required a significant amount of help from her supervisor. At the hearing, FE Watts initially testified that she did not consult with anyone and testified that she was not allowed to seek help from her direct supervisor. Id. at 134–36; 156. In fact, FE Watts engaged in an "in-depth consultation" with her direct supervisor, Pete Peterson, regarding the proficiency test. Id. at 159 ("So I did consult on analysis, like, is this -- you know, am I -- can I trust what I am seeing and what I am thinking? Should I be considering anything else? Ultimately, I told him that I would, uh, go through a more rigorous process of elimination this time, in which I would compare every area as much as possible and every orientation."). Worrisome, however, FE Watts made that correction in her testimony only after being confronted with documented evidence of said consultation.
This in-depth consultation was memorialized in an e-mail from FE Peterson to Michele Thompson. Id. at 157–58; Ex. 2007. Notably, FE Watts initially thought the print was a palmprint, but was also concerned it might be a fingerprint. Ex. 2007. "As this was a proficiency test, [FE Peterson] initially tried a more Socratic method to the consultation by asking probing questions and challenging support." Id. But "this Socratic/challenging process" apparently was not well received, as [FE Watts] indicated that she was feeling ‘antagonized by the process.’ " Id. FE Peterson then "changed [his] approach" and provided some of his own analysis, as well as guidance as to what stage of analysis FE Watts had successfully completed and what her next steps should be. Id.
FE Watts returned to work, and between November 2020 and August 2021, qualified examiners reviewed all of her casework. ECF No. 430 at 20. During this time, FE Watts also worked on a new team, under a new supervisor, because of a "general reorganization." Id. at 165. Even though performance reviews are conducted on an annual basis, FE Watts’ supervisor decided to meet with FE Watts on a monthly basis. Id. at 167. FE Watts attributes this mostly to concerns over administrative errors and numerical metrics that met but did not exceed annual targets. See id. These meetings were conducted in addition to the normal interactions the two would have regarding, for example, typographical errors and technical corrections. Id. at 168.
While the Court is sympathetic to the circumstances surrounding FE Watt's struggles, the Court cannot in good conscience qualify FE Watts as an expert with the requisite skill to perform fingerprint comparisons when her two most recent proficiency exams either contained an error or required a significant amount of assistance from her supervisor. This finding is bolstered by the portions of FE Watt's testimony and performance reviews that touch on her skill, willingness to take correction, and confidence performing her work. See, e.g. , Exhibit 2007 at 3, 58.
The Court notes that only some of errors and supporting documentation were initially disclosed to the Clouds, and the Court was therefore required to reset the hearing on this matter to allow for additional discovery. See ECF No. 410. The Court was then particularly disturbed to learn that, according to FE Watts, most of these documents were never produced for the Court or defense attorneys in the last trial at which FE Watts was qualified to testify as an expert in fingerprint comparison. ECF No. 430 at 194–96 (discussing United States v. Clifton Lamar Dodd , No. 2:18-cr-243 (W.D. La.)).
Given this finding, the Court need not address Donovan Cloud's argument that FE Watts failed to reliably apply the ACE-V methodology when she examined the latent fingerprint and known fingerprints in this case. See ECF No. 347 at 63–65. Even so, the Court will proceed to analyze the methodology FE Watts purported to apply, as the Government may seek to find a replacement forensic examiner to perform a new, independent comparison of the latent print.
2. The ACE-V Methodology is Reliable
For much the same reasons the Court finds the AFTE method reliable, the Court finds the ACE-V method reliable. In fact, the PCAST Report found that fingerprint comparisons had foundational validity. Still, as with the firearms/toolmark testimony, the Court will limit any testimony as to fingerprint analysis to express only the levels of confidence supported by validly estimated rates of error.
i. Testability
This factor plays out in much the same way it did with the AFTE methodology. The proposition that a particular fingerprint came from a particular person is a falsifiable proposition. Like the AFTE methodology though, the ACE-V struggles with replicability. As with AFTE, there are numerous points of subjectivity, such as (1) the examiner's subjective choice to use certain areas of the fingerprint as points of comparison and (2) the examiner's personal assessment of what is and is not sufficient for a source identification or exclusion.
ii. Peer Review and Publication
This factor weighs slightly in favor of the Government. Research on fingerprint analysis has been published in a variety of fora and there is no indication that these publications suffer the shortcomings the Court identified with the AFTE Journal. See PCAST Report at 87–97 (summarizing studies and citing to a variety of sources). Of the two appropriately designed black-box studies, the first was conducted by the FBI and published "in a leading international science journal." Id. at 94 (citing Bradford T. Ulery et al., Accuracy and Reliability of Forensic Latent Fingerprint Decisions , 108 Proceedings of the National Academy of Sciences 7733 (2011)). The authors also published an open-access follow-up study the next year. Bradford T. Ulery et al., Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners , PLOS One 7(3), Mar. 2012, https://doi-org.ezproxy.taylors.edu.my/10.1371/journal.pone.0032800. The second black-box study, conducted by the Miami-Dade Police Department Forensic Sciences Bureau and funded at least in part by the National Institute of Justice, was not published in a peer-reviewed journal, but was at least posted online for free, opening it up to analysis in a way that the AFTE Journal publications are not. Accordingly, in light of all of these studies and the AFTE Report's finding of foundational validity, the Court finds that this factor weighs slight in the Government's favor.
iii. The Known or Potential Rate of Error
This factor also weighs in favor of the Government. Here, there are two black-box validation studies, one of which had follow-up. The first, the 2011 FBI study, reported 6 false positive identifications among 3628 non-matching pairs. The false positive error rate was therefore reasonably as high as 0.33 percent (1 in 306). PCAST Report at 94. As noted, in 2012, the same authors reported a follow-up study. Id. Seven months after the first test, 75 examiners returned to examine a subset of the latent-known pairs from the prior study. Id. Of the 476 non-matching pairs leading to conclusive examinations, there were zero false positives, corresponding to an error rate reasonably as high as 0.63 percent (1 in 160). According to the PCAST Report, "[t]hese results ... are broadly consistent with the false positive rate measured in the previous study." Id.
Third, the Miami-Dade Police Department Forensic Services Bureau conducted a study that was published on the internet but not peer-reviewed. Id. at 94–95. This study did not select non-matching pairs using a large database, as in the FBI studies. Id. at 95. Still, the study found 42 false positives among 995 conclusive examinations, corresponding to an error rate reasonably as high as 5.4 percent (1 in 18). Id. The paper observes that "in 35 of the erroneous identifications the participants appeared to have made a clerical error, but the authors could not determine this with certainty." Id. As such, the Court finds that consideration of the post hoc corrected data is inappropriate.
Importantly, these figures allow the examiners to choose "inconclusive" without penalty and without affecting the error rate. Again, while this is a knock against the external validity of the studies’ findings, there is also some merit to the decision to include the "inconclusive" answer choice. Moreover, the figures cited above correctly exclude the inconclusive responses from the error rate calculation, somewhat mitigating the responses’ effect.
Given the studies summarized above, the Court finds that this factor weighs in favor of the Government. As with the AFTE analysis, this is the most important factor, and although there are reasons to discount the studies, the Court is convinced that presenting the estimated error rates to the jury, at least one of which is highly favorable to the defense, sufficiently mitigates those concerns.
iv. The Existence and Maintenance of Standards Controlling the Technique's Operation
Here, the operative standard is the ACE-V methodology. As explained already, ACE-V, like AFTE, has a degree of subjectivity that prevents the uninitiated from entirely understanding precisely how and why the examiner reached her conclusion. This Daubert factor therefore weighs against the Government and against admissibility.
v. General Acceptance Within the Relevant Scientific Community
This factor is neutral. As with the AFTE method, the most important question here is how to best define the relevant scientific community. If the relevant community is limited to forensic examiners, the factor weighs heavily in favor of the Government, as the theory is widely accepted within the community of people who purport to make such source identifications. Again, doing so makes little sense, as the narrowly defined relevant community is incentivized to bolster the perceived validity of their profession and is weighed down by cognitive biases that obscure the deficiencies in any claims of validity. However, unlike with AFTE, fingerprint comparison is accepted more broadly than the AFTE method. Most significantly, PCAST deemed fingerprint comparison, and the ACE-V method specifically, to have foundational validity. This factor therefore weighs in favor of the Government.
3. Limitations on Fingerprint Testimony
For these reasons, the Court concludes that a qualified forensic examiner may testify regarding her analysis and comparison of the recovered latent fingerprint. However, given the concerns outlined above, if the examiner intends to go beyond testimony that merely notes the recovered cartridge casings could not be excluded as having been fired from the recovered firearm, the Court will inform the jury that: (1) only two studies that meet the minimum design standard have attempted to measure the accuracy of fingerprint comparison and (2) these studies found false positive rates that could be as high as 1 in 306 in one study, 1 in 18 in the second study.
CONCLUSION
For the foregoing reasons, the Court finds both the AFTE and ACE-V methodologies reliable under Daubert . The Court further finds that FE Van Arsdale is qualified to testify regarding firearm/toolmark comparisons, but the Court reserves decision whether FE Van Arsdale reliably applied the AFTE methodology in this case. Finally, the Court finds FE Watts is not qualified to testify in this manner, though the Court will provide the Government an opportunity to proffer a new fingerprint expert.
Accordingly, IT IS HEREBY ORDERED :
1. Defendant James Cloud's (01) Motion Exclude or Limit Toolmark Identification Evidence, ECF No. 325 , is GRANTED IN PART and DENIED IN PART . Any testimony regarding a comparison of the recovered cartridges to the cartridges known to have been fired by the recovered rifle are limited as described in this Order.
A. The Court reserves decision on whether FE Van Arsdale reliably applied the AFTE methodology in this case. The Government shall arrange for FE Van Arsdale at the upcoming hearing in this matter, currently scheduled for January 4, 2022 . Given the nature and time of this testimony, FE Van Arsdale may appear via video, so long as the Government files a notice of his intent to do so no later than December 28, 2021 .
2. Defendant Donovan Cloud's (02) Motion to Exclude Expert Testimony on Friction Ridge Analysis, ECF No. 347 , is GRANTED IN PART and DENIED IN PART .
A. Forensic Examiner Krystal N. Watts is not qualified to testify as an expert in friction ridge analysis in this case. The Government may proffer a new expert by no later than January 14, 2022 . The Government's proffer, if any, must be accompanied by a robust disclosure of any and all materials related to the new examiner's comparison and any records bearing on past performance as a forensic examiner.
B. Donovan Cloud may file any motion challenging this new witness’ qualifications and testimony, if any, by no later than January 21, 2022 , to be heard at the scheduled January 25, 2022 Pretrial Conference.