Summary
requiring "definitive" demonstration of need for disclosure of algorithm
Summary of this case from State v. PickettOpinion
DOCKET NO. A-0938-19T3
04-20-2020
STATE of New Jersey, Plaintiff-Appellant, v. David GHIGLIOTTY, Defendant-Respondent.
Michele C. Buckley, Special Deputy Attorney General/Acting Assistant Prosecutor, argued the cause for appellant (Lyndsay V. Ruotolo, Acting Union County Prosecutor, attorney; Michele C. Buckley, of counsel and on the brief). Elizabeth Cheryl Jarit, Deputy Public Defender, argued the cause for respondent (Joseph E. Krakora, Public Defender, attorney; Elizabeth Cheryl Jarit, of counsel and on the brief).
Michele C. Buckley, Special Deputy Attorney General/Acting Assistant Prosecutor, argued the cause for appellant (Lyndsay V. Ruotolo, Acting Union County Prosecutor, attorney; Michele C. Buckley, of counsel and on the brief).
Elizabeth Cheryl Jarit, Deputy Public Defender, argued the cause for respondent (Joseph E. Krakora, Public Defender, attorney; Elizabeth Cheryl Jarit, of counsel and on the brief).
Before Judges Fuentes, Haas and Enright.
The opinion of the court was delivered by
HAAS, J.A.D. In this appeal, we address the novel issue of whether a firearms toolmark identification expert's use of untested three-dimensional (3D) computer imaging technology known as BULLETTRAX, in conjunction with the traditional technique of comparing evidence and test bullets using a comparison microscope, requires that a Frye hearing be held to establish the scientific reliability of the BULLETTRAX machine and related software.
Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).
Following an evidentiary hearing, the trial court concluded that the State's expert relied upon the BULLETTRAX technology and the images it produced in concluding that a bullet fragment taken from the murder victim likely came from a handgun later seized from defendant David Ghigliotty. In so ruling, the court made extensive factual and credibility findings pertaining to the expert's testimony about his use of the images, and its findings are entitled to our deference on appeal. Under these circumstances, we conclude that the court correctly determined that a Frye hearing was necessary to demonstrate the reliability of the computer images of the bullets produced by BULLETTRAX before the expert would be permitted to testify at trial.
I.
To place this issue in its proper context, we begin by providing a general overview of the field of firearm identification, which "is a specialized sub-specialty of toolmark identification specifically related to the firearm mechanism's working surfaces." Thompson at 7-8, 28. In this case, the "tool" is the firearm. Thompson at 7; 2009 NRC Report at 150-51. The tool surfaces, which include the interior of the firearm barrel, the chamber, and parts of the action, "can produce toolmarks on fired and unfired ammunition components." Thompson at 7. Toolmarks are "inherently three-dimensional." 2008 NRC Report at 186.
The necessary contextual background information pertaining to firearm and toolmark identification presented in this section is primarily taken from four scientific reports that defendant submitted to the trial court. Portions of the reports were also cited in the trial court's opinion.
The four reports, and the citations we will use to refer to them, are as follows: (1) Robert M. Thompson, Firearm Identification in the Forensic Science Laboratory (National District Attorneys Association, Alexandria, VA), 2010 (Thompson); (2) National Research Council, Committee to Assess the Feasibility, Accuracy, and Technical Capability of a National Ballistics Database, Ballistic Imaging (Nat'l Academies Press, Washington, D.C.), 2008 (2008 NRC Report); (3) National Research Council, Committee on Identifying the Needs of the Forensic Science Community, Strengthening Forensic Science in the United States: A Path Forward (Nat'l Academies Press, Washington, D.C.), 2009 (2009 NRC Report); and (4) President's Council of Advisors on Science and Technology, Report to the President, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (Executive Office of the President), September 2016 (PCAST Report).
Toolmark evidence may consist of "class characteristics," which are not unique, and "individual characteristics," which are "microscopically dissimilar." Thompson at 8-10. "Class characteristics" are firearm features "shared by many items of the same type" and determined by the manufacturer that "help narrow the population of potential firearm sources" when doing an identification. Thompson at 8; 2009 NRC Report at 152. They include "the size of the cartridge chambered by the firearm, the orientation of the extractor and ejector, and the number, width, and twist direction of the land and grooves of the barrel rifling." Thompson at 8, 15; 2009 NRC Report at 152.
In contrast, "individual characteristics" of a firearm are "fine microscopic markings and textures" that "are random in nature, usually arising from the tool working surface incidental to manufacture, but can also be the result of use, wear, and possible care and/or abuse of the tool," and that form striated or impressed toolmarks on ammunition. Thompson at 9-11; 2009 NRC Report at 152. Because of their uniqueness, a firearm's individual characteristics make firearm identification possible, so long as the toolmarks imparted are "reproducible for comparisons." Thompson at 7-9.
"For the science of toolmark identification, the underlying hypothesis is that a toolmark can be identified to a specific tool that produced it, to the practical exclusion of all other tools." Thompson at 9-10; 2009 NRC Report at 150. Although it is impossible to prove the hypothesis "by testing all tools ever produced in the world," firearms examiners make identifications "based on observation and experimentation" which includes the consideration of "known non-match" toolmark comparisons. Thompson at 10; 2009 NRC Report at 152.
The science of firearm and toolmark identification is well-established, spanning over 100 years in the United States. Thompson at 8. "There is a foundation of knowledge about firearm and toolmark identification that has been organized over time and is described in forensic textbooks, scientific literature, reference material, training manuals, and peer reviewed scientific journals." Thompson at 28-29. The Association of Firearm and Toolmark Examiners (AFTE), an international body of practitioners, is the largest professional organization in the field and publishes a professional journal concerning firearm and toolmark science. Thompson at 11, 29.
Neither the underlying principles nor the methodology has changed significantly during the last 100 years. Thompson at 8. The AFTE recognizes that the "most widely accepted method used in conducting a toolmark examination is a side-by-side, microscopic comparison of the markings on a questioned material item to known source marks imparted by a tool." PCAST Report 104. A firearms toolmark examiner uses a comparison microscope to compare toolmarks on an evidence bullet with toolmarks present on a test fired bullet from the suspected weapon that is linked to either the crime scene or a suspect. Thompson at 10, 27. Class characteristics are evaluated first, followed by individual characteristics. Thompson at 26; 2009 NRC Report at 152; PCAST Report at 11, 104.
The design of the comparison microscope, the primary tool used by firearms toolmark examiners, has not changed in approximately eighty years. Thompson at 8. It consists of two compound microscopes that are joined together, "giving the examiner the ability to observe and compare two objects at the same time under magnification." Ibid. While the comparison microscope is considered "an absolutely necessary instrument" in making "the identification of toolmarks on fired bullets," both Thompson's article and the 2009 NRC Report note that examiners "might use other complimentary microscopic and photographic instrumentation." Thompson at 8; 2009 NRC Report at 153.
When an examiner finds that the toolmarks on an evidence bullet and a test bullet fired from the suspected weapon are in "sufficient agreement," a firearm identification can be made. Thompson at 9-10, 26. "[T]he final determination of a match is always done through direct physical comparison of the evidence by a firearms examiner, not the computer analysis of images." 2009 NRC Report at 153.
In 1992, the AFTE adopted a "Theory of Identification" and standardized the terms and conclusions that firearms examiners should use to describe examination results. Thompson at 11. Revised in 2011, the Theory of Identification recognizes that identification is "subjective in nature" and defines and explains the "sufficient agreement" standard used by examiners:
1. The theory of identification as it pertains to the comparison of toolmarks enables opinions of common origins to be made when the unique surface contours of two toolmarks are in "sufficient agreement."
2. This "sufficient agreement" is related to the significant duplication of random toolmarks as evidenced by a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows. Specifically,
the relative height or depth, width, curvature and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. Agreement is significant when the agreement in individual characteristics exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool. The statement that "sufficient agreement" exists between
two toolmarks means that the agreement of individual characteristics is of a quantity and quality that the likelihood of another tool making the mark is so remote as to be considered a practical impossibility.
3. Currently the interpretation of individualization/ identification is subjective in nature, founded on scientific principles and based on the examiner's training and experience.
[Theory of Identification as it Relates to Toolmarks: Revised, 43 AFTE J. 287 (2011) (emphasis added).]
Based on its Theory of Identification, the AFTE developed four potential conclusions for examiners to make following an investigation: (1) identification; (2) inconclusive; (3) elimination; and (4) unsuitable, meaning that the evidence was not suited for examination. Thompson at 11-12.
An "identification" is "[a]greement of a combination of individual characteristics and all discernible class characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool." Thompson at 11.
A finding of "inconclusive" results when there exists: (1) "[s]ome agreement of individual characteristics and all discernible class characteristics, but insufficient for an identification"; or (2) "[a]greement of all discernible class characteristics without agreement or disagreement of individual characteristics due to an absence, insufficiency, or lack of reproducibility"; or (3) "[a]greement of all discernible class characteristics and disagreement of individual characteristics, but insufficient for an elimination." Id. at 11-12. A finding of "elimination" means that there exists "[s]ignificant disagreement of discernible class characteristics and/or individual characteristics." Id. at 12. As of 2010, the error rate in actual casework had not been studied or determined. Id. at 29. However, "reviews of proficiency testing data show that the error rate for misidentifications for firearm evidence is approximately 1.0%, and for toolmark evidence it is approximately 1.3%." Ibid. An individual examiner's error rate should be considered in each case. Ibid.
II.
With this essential background in mind, we now turn to the facts of the present case. On June 22, 2004, the body of the victim, Taji Pile, was found on the side of the road with a bullet wound to the head. The police did not recover a murder weapon at the scene, but were able to retrieve three bullets from the victim's body.
During the investigation that followed, the police interviewed defendant, who was "a close friend" of the victim. Defendant stated he and the victim had used and sold marijuana together in the past, and that he last saw the victim on the day he was killed. The police did not arrest or charge defendant at that time.
On May 5, 2005, the police arrested defendant for unlawful possession of a handgun. The investigation revealed that the handgun had been purchased by the victim's brother in October 2003. Defendant was later convicted of the handgun charge.
Detective Gary Mayer, a firearms expert at the Union County Prosecutor's Office, was responsible for conducting a firearm identification test involving the three bullets found at the murder scene and the handgun seized from defendant. Only one of the projectiles recovered at the murder scene, a bullet jacket fragment that was "significantly damaged, was deemed intact enough to be suitable for comparison."
Using only a standard comparison microscope, Mayer compared the jacket fragment with test shots fired from the handgun. His comparison yielded "negative results" or "an elimination."
In 2015, more than ten years after the victim's murder, the prosecutor's office reopened its investigation. The detective assigned to the case asked Lieutenant Michael Sandford, who was assigned to the Union County Police Department's Firearms Identification Unit, to help him "understand why [Mayer's] original opinion ... was a negative [or an elimination]." Sandford twice told the detective that as a matter of standard practice, "we don't reexamine [what] another [firearms] examiner had examined," but "[a]fter maybe the third time" the detective asked for help, Sandford agreed to conduct the re-examination.
Sandford had thirty-three years of experience in the Firearms Identification Unit, and had testified as an expert in court approximately eighty-five times.
Toward that end, in 2016, Sandford completed a "bullet identification" on the ballistic evidence, or a study to determine whether a projectile is "a component or a bullet." Sandford determined that two of the recovered projectiles were "lead core ... from a bullet" that could not be used to conduct a microscopic toolmark comparison. The third projectile, however, was a "heavily damaged [bullet] jacket" that could be used for a microscopic toolmark comparison. Sandford was unable to determine what company had manufactured the bullet jacket.
Following the bullet identification study, Sandford conducted a microscopic comparison of the bullet jacket, Mayer's test shots, and test shots of his own, using a comparison microscope. He testified that the comparison microscope enabled him "to observe two components or two pieces of evidence side by side under a singular binocular eyepiece under the same lighting, magnification and orientation."
Initially, Sandford was "unable to come to a conclusive result" and therefore "couldn't support [Mayer's] opinion." He decided that he needed to examine the evidence "further" in order to "come to an opinion of [his] own." Thereafter, Sandford took approximately ten additional test shots, using ammunition from several different manufacturers because, in his experience, "a firearm would mark differently with different ammunition." He then compared the new test shots with the jacket fragment, and he also compared the new test shots with his prior test shots and Mayer's test shots, reviewing the evidence "many" times. Sandford testified that even with this additional work with the comparison microscope, he still could not reach a conclusion on the question of whether the fragment could have come from defendant's handgun.
In June 2016, Sandford and another firearms identification examiner from his unit, Sergeant Krzysztof Audinis, attended a training seminar in New Orleans led by Andrew Boyle, the firearms reference manager at Ultra Electronics Forensic Technology, Inc. (UEFT). Boyle's position was in the sales and marketing group, and he presented workshops and demonstrations related to the BULLETTRAX machine and its software component, Matchpoint.
At times, the trial court referred to BULLETTRAX as "BULLETRAX" or "BULLETRAX HD3D." However, the manufacturer's website uses BULLETTRAX as the nomenclature for the machine and we do so herein. See https://www.ultra-forensictechnology.com/en/our-products/ballistic-identification/bullettrax/ (accessed Mar. 23, 2020).
Boyle was well-versed in how BULLETTRAX worked from a user perspective, but he did not have a computer science background and was not a firearms examiner. Because this technology was developed by UEFT's computer programmers and software engineers, Boyle was not familiar with the specific algorithms and computer programming involved with the technology.
UEFT developed the BULLETTRAX machine in 2003.
Sandford testified that BULLETTRAX used an algorithm as part of the image acquisition process that "creates ... a digital representation or a numeric representation of what it sees under the microscope." However, he did not know who created the algorithm, how it was developed, or how many versions had been developed. He said he "didn't have much interest in the algorithm," and admitted that he did not "fully understand" it.
Boyle explained that BULLETTRAX uses computer automation and confocal microscopy to scan the surface of a bullet, read its topography in 3D, and create 2D and 3D images of a bullet's surface. If the subject bullet is damaged, the operator can program the machine to scan only portions of the bullet. After the operator manually positions the bullet, sets the scanning parameters, and presses the start button, the machine operates by itself.
The bullet's surface, rotated by a motor, is scanned with a confocal lens and the machine automatically makes adjustments to ensure optimal, consistent lighting and focus. According to Boyle, the scanner's capture area is 1.6 millimeters, meaning that small sections of the bullet are scanned and then "stitched" together, using an algorithm, into a single image. Acquisition of the image of a damaged bullet takes, on average, twenty to twenty-five minutes.
Once bullet images are acquired via BULLETTRAX, the images are viewed using Matchpoint, a software program. According to Boyle, Matchpoint's features are "designed to complement the workflow that [the examiners] are accustomed to" and allows them to emulate how a visual analysis using a comparison microscope would work, with certain advantages.
For example, Boyle testified that the examiner can compare up to six images at one time on a single screen as opposed to a single pair and switch back and forth between 2D and 3D views. The images of the bullet's surface can be viewed across a flat plane, with consistent lighting and focus applied to each, allowing the examiner to "have a bird's-eye view of not just one little area, like they would typically have on a comparison microscope." An examiner can reposition the images, manipulate the lighting, brightness, contrast and apply those changes instantaneously across all images, whereas such adjustments must be made manually when using a comparison microscope.
Boyle testified that following his 2016 AFTE workshop, at which he "show[ed] what we can do with bullet comparisons," Sandford reached out to the company's sales director and "made arrangements to visit [UEFT's office in] Montreal to have exhibits imaged into our system." Sandford explained that the use of BULLETTRAX and Matchpoint for the Pile investigation appealed to him because he "was looking ... for tools that would help [him] be more efficient in [his] examination" of the evidence. He testified that "a microscopic comparison on ... damaged projectiles is a very cumbersome examination" and he "felt that [he] needed some other tool to help [him] to complete this examination ... in a reasonable amount of time." Sandford noted the need to position the projectiles to ensure that "the area of interest that you're looking at [is] top-dead center" on both projectiles, which requires constant manual manipulation of the angles, lighting, and magnification.
Subsequently, Sandford brought the physical evidence to Montreal, and used the Matchpoint software to view images of the evidence created by BULLETTRAX. Boyle operated the BULLETTRAX machine and acquired 3D, black-and-white images of the surface of the damaged bullet jacket and four test shots. Thus, Boyle was responsible for positioning the projectiles into the machine, adjusting the brightness, and operating the computer program associated with it.
This was the first and only time Sandford had ever used BULLETTRAX in conducting a firearms identification study.
Thereafter, Sandford looked at the images side-by-side on a computer screen using Matchpoint "to try and target areas of interest to determine ... if [he] was going to go back and continue with further microscopic comparisons or not." He compared the bullet jacket to each of the four test shots. He also looked at one of Mayer's test shots. Boyle showed Sandford how to use the Matchpoint software, which included tools for flattening and manipulating the images, adjusting the brightness, zooming in, and "different overlays of ... color scaling."
During this process, Sandford identified five areas of interest, which he described as "four [LEAs] and ... a G[EA] area on the ... evidence item." According to Sandford, a "LEA" is "[a] land engraved impression in rifling on a projectile" and a "GEA" is "[a] groove engraved area."
Audinis accompanied Sandford to Montreal and while Sandford was working on Matchpoint, he communicated his findings to Audinis, who attempted to locate the areas of interest on the physical evidence using a comparison microscope. The partners remained in Montreal for three days, from June 20 through June 22, 2016.
Sandford explained that acquisition of the images through BULLETTRAX, which utilizes confocal microscopy, was advantageous "because [BULLETTRAX] takes away the influence of color when looking at [the] images" while "[a] comparison microscope ... uses color and color can influence you in different ways." Sandford testified that he incorporated some of the grayscale images from BULLETTRAX, in the form of "screen shots," into his subsequent written report to the prosecutor's office, although he asserted that he did not "make any conclusions" based on the BULLETTRAX images. Sandford placed markings on the screen shots when he came back to the lab to highlight the areas of interest he wanted to explore under the microscope.
When Sandford returned to his lab, he "waited a couple [of] weeks to clear [his] mind" before looking at the evidence again. Eventually, he used the screen shots from BULLETTRAX "like[ ] a GPS" to guide his comparison microscope analysis of the "areas of interest" and "came to an opinion of an identification or a positive identification."
More specifically, Sandford explained that he "found sufficient agreement in four different [LEAs] and a G[EA] ... between the evidence and a test shot." He testified that his positive identification of the bullet, as one that was likely shot from defendant's gun, was grounded in his "training and experience and education as a practitioner in firearms identification" and his handling of over 2300 cases. Although Sandford was not able to make a positive identification prior to using the BULLETTRAX technology, he testified that all of the conclusions and opinions he rendered were based on his "live comparison on a comparison microscope" as opposed to "photographs or a computer image."
Sandford testified that Audinis conducted a peer review, meaning an independent examination of the evidence, after he made his identification in the lab based upon his "microscopic comparison work only." Sandford acknowledged that a peer reviewer "is not supposed to be involved" in the initial examination at all or have any information about the first examiner's conclusion. However, he stated that Audinis worked with him on the case because they were a "small lab" of only two examiners.
Sandford further explained that his use of Matchpoint and BULLETTRAX helped him "define areas that [he] wanted to look at closer" and "[t]o avoid ... the cumbersome task of [manually] going around the entire circumference of a damaged bullet" as he "wanted to do [the examination] more efficiently." He said that not all of the areas of interest he flagged using BULLETTRAX led to a finding of sufficient agreement under the comparison microscope, and that, conversely, he had found areas of sufficient agreement that he "did not see in Montreal." Thus, Sandford asserted that even had he not used BULLETTRAX and Matchpoint to assist him, he "would have found those same areas of interest and identification points" anyway using the comparison microscope "if [he] kept at it and continued [his] investigation."
Nevertheless, Sandford acknowledged that the BULLETTRAX images served as a guide, helped him "move forward" with his examination of the evidence, and that he "relied on ... what [he] observed in ... Montreal to look at specific areas or certain areas ... rather than going through the entire circumference of the evidence." He said that there was "no way to put a time frame" on how long an investigation using a comparison microscope might take as it varied case to case.
With regard to Sandford's claim that the BULLETTRAX technology did not affect his subsequent positive identification, defense counsel asked him about an email exchange with a UEFT employee, Stacy Stern, while Sandford was still in Montreal. Stern wrote to Sandford: "I heard things may be looking good?" Sandford replied: "They just stepped over from looking good to outstanding." Sandford testified that his use of the word "outstanding" in the email meant "that the intent of what we went there for, we achieved that intent. To identify the areas [of interest] that we wanted to look at."
Sandford did not discuss the possibility of image distortion with Boyle but admitted that "it could have" affected how he located areas of interest. In this regard, Boyle testified that "one of the drawbacks" of using confocal microscopy in BULLETTRAX "is that it has a difficult time capturing information from especially steep slopes" and "deals better with ... a more stable shaped surface." This means that if there are areas with "especially steep slopes" on a bullet surface, then the computer-generated image could have "areas of missing data" that would appear as a "black region."
Boyle also stated that acquisition issues can result if a bullet surface is too shiny or too dark, and artificial "noise" (markings not actually present on the bullet) can show up in the image. He did not recall any steep slope issues or other distortion problems with the images acquired in this case. Boyle said that he was not aware of any validation studies done on BULLETTRAX concerning image acquisition or image viewing.
Sandford kept in touch with Boyle after his trip to Montreal and gave a presentation at the following year's AFTE training conference with him. Sandford admitted that he discussed the Pile case during his presentation and that a printed "summary of that training" mentioned "how 3D comparison ... recently led to a conclusive result in a decade-old cold case homicide." Boyle testified that the seminar "showcase[d] the benefits of how 3D analysis helped them work the case material" and "images were shown in regards to how the machine aided them in pinpointing areas of interest to explore on the comparison microscope."
III.
Based upon Sandford's positive identification of the bullet fragment as a projectile that had been shot from the gun seized from defendant in 2005, a grand jury indicted defendant on March 1, 2017 for the first-degree murder of Taji Pile, second-degree unlawful possession of a handgun, and second-degree unlawful possession of a firearm.
On April 5, 2018, defendant moved to preclude or limit Sandford's proposed expert testimony pursuant to Frye. The court scheduled a N.J.R.E. 104 hearing and ordered the State to produce "any and all manuals, policies or procedures from [UEFT]" for the BULLETTRAX software. UEFT refused to provide this information to the State and, therefore, it was unable to comply with the court's order.
The court thereafter conducted a three-day evidentiary hearing at which Sandford and Boyle testified as discussed above. At the conclusion of the hearing, the court rendered a lengthy written opinion and order granting defendant's motion for a Frye hearing to determine the reliability of the BULLETTRAX machine and the Matchpoint software. Before discussing that ruling in greater detail, we briefly summarize the legal principles governing the issue presented.
"[T]he Judiciary must ensure that proceedings are fair to both the accused and the victim. Trial judges partly fulfill that responsibility by serving as a gatekeeper. In that role, they must assess whether expert testimony is sufficiently reliable before it can be presented to a jury." State v. J.L.G., 234 N.J. 265, 307-08, 190 A.3d 442 (2018). "While juries would not always accord excessive weight to unreliable expert testimony, there is substantial danger that they would do so, precisely because the evidence is labeled ‘scientific’ and ‘expert.’ " State v. Cavallo, 88 N.J. 508, 518, 443 A.2d 1020 (1982).
Pursuant to N.J.R.E. 702, "[i]f scientific ... knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education may testify thereto in the form of an opinion or otherwise." To meet this requirement,
the proponent of expert evidence must establish three things: (1) the subject matter of the testimony must be "beyond the ken of the average juror"; (2) the field of inquiry "must be at a state of the art such that an expert's testimony could be sufficiently reliable"; and (3) "the witness must have sufficient expertise to offer the" testimony.
[ J.L.G., 234 N.J. at 280, 190 A.3d 442 (quoting State v. Kelly, 97 N.J. 178, 208, 478 A.2d 364 (1984) ).]
The three prongs cited above "are construed liberally in light of Rule 702's tilt in favor of the admissibility of expert testimony." State v. Jenewicz, 193 N.J. 440, 454, 940 A.2d 269 (2008). At issue in this case is the second prong concerning reliability of the field of inquiry. "In criminal cases, [our Supreme] Court has continued to rely on the Frye standard to assess reliability." J.L.G., 234 N.J. at 280, 190 A.3d 442 ; accord State v. Harvey, 151 N.J. 117, 170, 699 A.2d 596 (1997) ; State v. Torres, 183 N.J. 554, 568, 874 A.2d 1084 (2005).
In In re Accutane Litigation, 234 N.J. 340, 347-48, 398-99, 191 A.3d 560 (2018), the Supreme Court adopted the factors identified in Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 592-93, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993), which describe a methodology-based approach for determining scientific reliability, to "aid our trial courts in their role as the gatekeeper of scientific expert testimony in civil cases," but explicitly "retain[ed] the general acceptance test for reliability [as established in Frye,] in criminal matters." See also State v. Cassidy, 235 N.J. 482, 492, 197 A.3d 86 (2018) (noting that the Court has "not altered [its] adherence to the [Frye ] general acceptance test for reliability in criminal matters").
In Frye, 293 F. at 1014, the District of Columbia Court of Appeals explained:
Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.
The Frye test "requires trial judges to determine whether the science underlying the proposed expert testimony has ‘gained general acceptance in the particular field in which it belongs.’ " J.L.G., 234 N.J. at 280, 190 A.3d 442 (quoting Frye, 293 F. at 1014 ). "[T]here are three ways to establish general acceptance under Frye: expert testimony, authoritative scientific and legal writings, and judicial opinions." J.L.G., 234 N.J. at 281, 190 A.3d 442. "Proof of general acceptance does not mean that there must be complete agreement in the scientific community about the techniques, methodology, or procedures that underlie the scientific evidence." State v. Chun, 194 N.J. 54, 91-92, 943 A.2d 114 (2008) ; accord State v. McGuire, 419 N.J. Super. 88, 133, 16 A.3d 411 (App. Div. 2011).
"[T]ool mark analysis is not a newcomer to the courtroom." McGuire, 419 N.J. Super. at 130, 16 A.3d 411. "Testimony by tool mark experts has been admitted in New Jersey courts [under the Frye standard] without objection" and also "in other jurisdictions." Id. at 130-31, 16 A.3d 411 ; see State v. Behn, 375 N.J. Super. 409, 416-19, 868 A.2d 329 (App. Div. 2005) (discussing admitted testimony from "a ballistics and tool mark identification expert" linking bullets recovered from victim to a rifle that the defendant had purchased); State v. Cito, 213 N.J. Super. 296, 299, 517 A.2d 174 (App. Div. 1986) (discussing admitted testimony from a toolmark expert linking a screwdriver seized from the defendant's home to the scene of the crime).
IV.
Here, the trial court's concern was not with the general acceptance of the discipline of firearm toolmark identification, but rather with the new technique Sandford employed in reaching his conclusion. After making extensive factual findings pertaining to the testimony and the relevant scientific studies, it held "that it need not hold a Frye hearing to determine the general acceptance of toolmark identification." However, it conditioned the admissibility of Sandford's expert testimony upon a finding that BULLETTRAX and Matchpoint were scientifically reliable and held that a Frye hearing was warranted for that purpose.
While the court credited Sandford's testimony "that [BULLETTRAX] provided assistance in identifying areas of interest on the evidence bullet" and served as "a shortcut to save time in conducting the same review he otherwise would have made manually over the entire circumference of the evidence bullet and comparison bullets," it discredited Sandford's testimony that BULLETTRAX "did not inform his ultimate opinion in this matter." Instead, it found that "the [BULLETTRAX] images ... at least partially formed the basis of [Sandford's] opinion that the evidence bullet and the test fired shot in this case were a match" and were "so intertwined" with his "subsequent laboratory comparisons" that they "cannot be separated from his later identification simply because time passed."
Thus, the court found "that the images created by [BULLETTRAX] were an integral part of ... Sandford's opinion" that the bullet fragment came from defendant's gun. The court noted that Sandford testified that "he used the screenshots of the images he viewed on Matchpoint as a ‘GPS’ in order to readily identify the areas of the evidence bullet that he believed were ‘of interest’ when he examined the bullet under his confocal microscope back in New Jersey."
The court continued:
This [c]ourt is therefore unassuaged by ... Sandford's testimony that he could have made the identification without the use of [BULLETTRAX] because, stated simply, that is not what occurred here and to say that the identification was inevitable is purely speculative. This assertion is also contradicted by the findings of the original toolmark examiner [Mayer] who excluded the firearm in this case as a possible source of the evidence bullet.
The court also cited Sandford's email exchange with Stern while in Montreal in which he wrote that due to his use of BULLETTRAX, the situation "stepped over from looking good to outstanding," and found that his statement "call[ed] into question [his] claim that he never had an ‘ah-ha’ moment or a point in time where he thought there would probably be an identification while using" BULLETTRAX. It further found that Sandford's testimony that the images did not inform his ultimate opinion was "even more difficult to reconcile" since "Audinis ... was using a comparison microscope in the same room as ... Sandford, as he directed him to examine certain areas of the evidence bullet."
In addition, the court cited Sandford's joint presentation with Boyle at the 2017 AFTE conference "on how new 3-D comparison tools led to a conclusive identification in a decade-old cold case" as further evidence that, despite his assertions to the contrary, the BULLETTRAX technology Sandford used in Montreal clearly influenced his later conclusion that the bullet fragment was shot from defendant's gun.
As to the admissibility of evidence concerning this new technology in the absence of a Frye hearing, the court found, based upon Boyle's credible testimony, that BULLETTRAX "is a highly automated technology that does not merely photograph the bullet's surface, as suggested by the State, but instead digitally recreates the entire surface area." It also cited the fact that it "was unable to find—and the parties did not present—any published or unpublished decisions addressing the reliability or general acceptance of [BULLETTRAX] in connection with the examination and identification of ballistics."
As to the Matchpoint software, the court found that its reliability was "[e]qually unproven at this time." It added that the record lacked "any validation studies or indeed any records concerning the testing [BULLETTRAX] has undergone to determine the accuracy and reliability of the images it produces [and/or] the [Matchpoint] software that permits examiner interaction with those images."
Accordingly, the court ordered that because Sandford relied upon the untested BULLETTRAX technology in reaching his positive identification of the bullet fragment, a Frye hearing was needed to determine the scientific reliability of his proposed testimony concerning this device.
The court also addressed two other issues raised by defendant. First, defendant again demanded that the State provide him with the BULLETTRAX algorithms in advance of the Frye hearing. In support of this request, however, defendant relied solely upon defense counsel's request for this information, and did not submit a certification from its expert detailing the reasons why the production of the algorithms was necessary, or what other specific discovery concerning BULLETTRAX and Matchpoint might be warranted.
Nevertheless, citing Chun, 194 N.J. at 69-70, 943 A.2d 114, the court again found that production of the algorithms was necessary to ensure "defendant's right to a fair trial." Without citing anything else, the court found that the State's provision of this information would allow "the defense ... to effectively cross examine the State's witnesses at trial and to prepare for the Frye hearing" since "[t]he review and testing of algorithms and/or source coding utilized in a new or untested technology is required in order to determine its reliability." While the court stated it was amenable to making the production of the algorithms subject to a protective order, the terms of such an order were not delineated by the court or the parties.
Finally, defendant argued that if the BULLETTRAX technology ultimately passed the Frye test, Sandford should not be permitted to testify that he had come to his conclusion concerning the bullet fragment to a reasonable degree of scientific certainty. Instead, defendant asked that Sandford's testimony be limited to his claim that the evidence bullet was consistent with having been fired from the gun recovered from defendant, but not to the exclusion of all other guns. After examining the studies discussed earlier in this opinion and caselaw from other jurisdictions, the court granted this request even though if BULLETTRAX did not survive Frye scrutiny, the court's order limiting Sandford's expert testimony would be moot.
V.
We subsequently granted the State's motion for leave to appeal. On appeal, the State argues that the trial court erred by concluding that Sandford relied upon the BULLETTRAX technology in arriving at his conclusion that the bullet fragment came from defendant's gun. It alleges that Sandford denied basing his findings on what he observed in Montreal on the BULLETTRAX machine and, instead, his "ultimate results and conclusions" were made using only a standard comparison microscope. Therefore, the State claims that a Frye hearing concerning BULLETTRAX and Matchpoint was not necessary. We reject these contentions because the court's contrary determinations were squarely based upon its judgment that Sandford's testimony on this point was not credible. "[W]hen the result of the contest must turn on the truthfulness of witnesses, the superior advantage of the trial judge in seeing and hearing and appraising the disputants must ordinarily be regarded as the fulcrum on which the issue should be resolved." Abeles v. Adams Eng'g Co., 35 N.J. 411, 423-24, 173 A.2d 246 (1961). For this reason, appellate "[c]ourts generally defer to a trial court's credibility findings about the testimony of expert witnesses[ ]" and "accept the court's factual ‘findings to the extent that they are supported by substantial credible evidence in the record[.]’ " J.L.G., 234 N.J. at 301, 190 A.3d 442 (quoting Chun, 194 N.J. at 93, 943 A.2d 114 ).
Thus, "[a]n appellate court should not disturb the trial court's findings merely because ‘it might have reached a different conclusion were it the trial tribunal' or because ‘the trial court decided all evidence or inference conflicts in favor of one side’ in a close case." State v. Elders, 192 N.J. 224, 244, 927 A.2d 1250 (2007) (quoting State v. Johnson, 42 N.J. 146, 162, 199 A.2d 809 (1964) ). Accordingly, "[a] trial court's findings should be disturbed only if they are so clearly mistaken ‘that the interests of justice demand intervention and correction.’ " Ibid. (quoting Johnson, 42 N.J. at 162, 199 A.2d 809 ).
Applying this standard, we are satisfied that the trial court's factual findings and credibility determinations are fully supported by substantial credible evidence and, as such, are unassailable on appeal. The record makes clear that the BULLETTRAX images, which Sandford reviewed both while in Montreal and back in his lab in New Jersey, prompted him to reexamine the physical evidence under the comparison microscope. He even testified that he included some of the images in his report to the prosecutor's office. After reviewing the images and reexamining the evidence, he changed his ultimate conclusion from "inconclusive" to an "identification." In that regard, the BULLETTRAX images clearly aided and influenced the course of his investigation and informed his ultimate opinion, as the trial court properly found.
In reaching this conclusion, the court considered Sandford's repeated assertion that he was able to put aside everything he learned from examining the bullet fragment and the test bullets with the BULLETTRAX machine. However, the court rejected this claim based on its assessment of Sandford's credibility on the stand, coupled with the substantive evidence in the record that firmly demonstrated that the improved images Sandford obtained from the new technology clearly colored his final conclusion.
In this regard, it is important to recall that toolmark identification, while "founded on scientific principles," is "subjective in nature, ... and based on the examiner's training and experience." Theory of Identification, 43 AFTE J. at 287. Here, Sandford had previously been unable to make a finding on the comparability of the bullet fragment taken from the victim and the test bullets fired from defendant's gun. His predecessor, Mayer, had already reached a negative conclusion. It was only after Sandford had the opportunity to use the new BULLETTRAX technology that he was able to change his determination from inconclusive to positive.
The State claims that the trial court should have accepted Sandford's explanation that the BULLETTRAX merely provided him with "areas of interest" that he could later examine with the comparison microscope. But, this begs the question. With unlimited time to examine the bullet fragment and the test bullets using the microscope, Sandford admitted he could not reach a definitive conclusion before going to Montreal. It was only after the BULLETTRAX machine guided his analysis that the State's expert was able to find the areas of the evidence that ultimately led him to change his original findings. Under these circumstances, we are satisfied that the trial court properly rejected the State's argument that the BULLETTRAX technology played no role in Sandford's final opinion.
The trial court also pointed to other facts in the record that supported its determination that Sandford would not have been able to reach his conclusions if he had not had access to BULLETTRAX. For example, Sandford responded to an email sent by a UEFT employee by stating that his work on the evidence had gone "from looking good to outstanding" after he was able to use the new technology. Sandford later gave a seminar on BULLETTRAX that described how using the machine "led to a conclusive identification in a decade-old cold case." In addition, the court noted that it was "even more difficult to reconcile" Sandford's claim that he did not rely on BULLETTRAX to reach his conclusion because his partner, Audinis, was in Montreal at the same time using a comparison microscope to examine the evidence bullet as Sandford used the computer images as "a GPS" to guide him.
In sum, we discern no basis for disturbing the trial court's determination that BULLETTRAX "play[ed] a fundamental role in the ballistics examination in this case" and, like that court, we cannot "accept as plausible the assertion that [BULLETTRAX] did not form at least a partial basis for ... Sandford's ultimate conclusion."
In so ruling, we reject the State's reliance upon this court's decision in McGuire in support of its contention that Sandford made an independent evaluation of the evidence by examining it with a standard comparison microscope after employing BULLETTRAX and, therefore, his use of the new technology should be disregarded. In McGuire, a toolmark expert was called upon to opine whether a plastic garbage bag similar to one found in the defendant's apartment might have been used in connection with the murder of her husband. 419 N.J. Super. at 119, 124, 127-30, 16 A.3d 411. After examining the bags, the expert visited the plastic bag manufacturing plant, where he spoke with a plastics engineer who advised him that a distinctive "cliff" in the bag "could be caused by an operator putting one of the cutting blades in backwards." Id. at 128-29, 16 A.3d 411. The expert then personally examined the tool that cut the bags and observed how a blade might be replaced backwards. Id. at 133, 16 A.3d 411. Ultimately, the expert "concluded that the ‘cliffs’ in the bags in this case were likely caused by a backwards blade." Ibid. His conclusion was based, in part, on his conversation with the plastics engineer. Id. at 129, 16 A.3d 411.
On appeal, the defendant argued that the expert's opinion concerning the " ‘cliffs’ on [the] edges of the bags was inadmissible hearsay from an unidentified person and not a valid basis for expert testimony." Id. at 133, 16 A.3d 411. We disagreed, concluding that the expert had "formulated his own opinion" about the "cliffs." Ibid. It held that "[t]here is nothing improper about an expert obtaining information from an outside source, as long as the expert makes an independent evaluation of the information and is able to explain the basis for the opinion that he reached." Ibid.
Contrary to the State's contention, our decision in McGuire is plainly distinguishable from the matter at hand. In that case, the plastics engineer merely suggested that the expert consider whether the "cliff" in the bag could have been caused by an improperly placed blade. The expert then tested that hypothesis completely on his own, using his standard methods. Here, on the other hand, Sandford employed a completely new and untested machine to personally examine the bullet fragment and the test bullets, included images produced by BULLETTRAX in his report, and, as the trial court found, based his ultimate conclusion, at least in part, on his use of this novel technology, rather than the usual comparison microscope. Therefore, the State's argument on this point fails.
Because Sandford relied upon BULLETTRAX in arriving at his expert opinion, we also agree with the trial court that a Frye hearing was necessary to determine the reliability of this new technology. For evidence to be admissible under N.J.R.E. 702, "the expert must utilize a technique or analysis with ‘a sufficient scientific basis to produce uniform and reasonably reliable results so as to contribute materially to the ascertainment of the truth.’ " State v. J.R., 227 N.J. 393, 409, 152 A.3d 180 (2017) (quoting Kelly, 97 N.J. at 178, 478 A.2d 364 ). "Thus, the test in criminal cases [is] whether the scientific community generally accepts the evidence." Harvey, 151 N.J. at 170, 699 A.2d 596. To establish general acceptance, "the party proffering the evidence need not show infallibility of the technique nor unanimity of its acceptance in the scientific community." Cassidy, 235 N.J. at 492, 197 A.3d 86. "[T]he State's burden is to prove that the ... test and the interpretation of its results are non-experimental, demonstrable techniques that the relevant scientific community widely, but perhaps not unanimously, accepts as reliable." Harvey, 151 N.J. at 171, 699 A.2d 596.
An application of the Frye test at an evidentiary hearing was necessary in this case because BULLETTRAX is a new, untested device, operated by Matchpoint, a novel software product. As the trial court found, "BULLETTRAX is a highly automated technology that does not merely photograph the bullet's surface, as suggested by the State, but instead digitally recreates the entire surface area." The parties did not provide the court with any judicial opinions or authoritative scientific and legal writings demonstrating the reliability of this machine.
In addition, neither Sandford nor Boyle were experts in the science behind the BULLETTRAX system and, therefore, were unable to address whether it provided reliable images. In that regard, both witnesses conceded that BULLETTRAX created some degree of distortion when it "stitched together" the images of the bullet fragment and the test bullets that Sandford used to reach his conclusions. The trial court also correctly found that, for many of these same reasons, "the reliability of Matchpoint" was "[e]qually unproven at this time."
Under these circumstances, we affirm the trial court's determination that a Frye hearing was necessary to protect defendant's due process rights and ensure that the images produced by BULLETTRAX were sufficiently reliable to be admissible under N.J.R.E. 702.
VI.
Finally, we turn to the trial court's order requiring the State to provide defendant with "the BULLETTRAX ... and Matchpoint algorithms" in advance of the Frye hearing, and the court's ruling that if BULLETTRAX is found reliable following that hearing, Sandford "may testify that the evidence bullet in this case is consistent with having been fired from the weapon seized from ... defendant[,] but may not opine as to any degree of scientific certainty or to the exclusion of all other firearms." For the reasons that follow, we vacate each of these directives because we believe they were prematurely issued.
The trial court ordered the State to produce the BULLETTRAX and Matchpoint algorithms based solely upon defense counsel's request. While it is certainly possible that this information might be needed by defendant's experts to evaluate the reliability of the new technology, the defense did not present a certification from an expert in support of this claim for disclosure. Thus, there is currently nothing concrete in the record to support the court's conclusion that granting defendant "the opportunity to review the algorithms and elicit testimony concerning" BULLETTRAX is necessary "in order to completely explore and test the integrity of the images it produces."
Under these circumstances, defendant is required to make a more definitive showing of his need for this material to provide the court with a rational basis to order the State to attempt to produce it. In that regard, the trial court was aware that the algorithms are proprietary information within UEFT's, rather than the State's, sole possession. While the court was open to issuing a protective order to attempt to overcome UEFT's reluctance to disclose this information to the State, the parties did not submit suggested language to the court to assist it in attempting to craft and issue such an order.
Therefore, we vacate the court's order directing the turnover of the algorithms, and remand the discovery issues to the court for further consideration. The court must promptly conduct a case management conference with the parties to determine the most efficient way to proceed to identify the types of information that must be shared by them in advance of the Frye hearing. Resolution of discovery issues must be made after a N.J.R.E. 104 hearing to ensure the development of a proper, reviewable record that supports the court's ultimate decision.
We also hold that the court's order limiting the scope of Sandford's possible testimony at trial was premature. At this point, the State has not demonstrated that the BULLETTRAX and Matchpoint technology Sandford relied upon is sufficiently reliable under Frye to be admissible at trial. If the court ultimately concludes that the Frye standard has not been met, it is highly unlikely that Sandford would testify at trial as an expert witness.
On the other hand, if the court concludes that BULLETTRAX satisfies the Frye test, Sandford's testimony at trial would be based on a completely new technology. An expert's use of this technology was not addressed by any of the judicial decisions from other jurisdictions cited by the court in limiting Sandford's testimony. Those decisions only concerned witnesses who used the standard comparison microscope. Thus, if the court finds that the Frye standard has been met, Sandford's trial testimony may have a completely different, and perhaps more solid, basis than that of the experts who had their testimony limited by a court. Thus, we conclude that the court's placement of a limitation on Sandford's testimony constituted an advisory opinion which the court did not need to render at this stage of the proceedings. See G.H. v. Twp. of Galloway, 199 N.J. 135, 136, 971 A.2d 401 (2009) (noting that courts generally should not "answer abstract questions or give advisory opinions"). We therefore vacate this portion of the order. The court may revisit the issue of the scope of Sandford's trial testimony following the Frye hearing. VII.
In sum, we affirm the trial court's determination that a Frye hearing must be held to determine the scientific reliability of the BULLETTRAX and Matchpoint technology. We vacate the portions of the court's order concerning discovery of the algorithms, and its ruling on the scope of Sandford's trial testimony. We remand for further proceedings consistent with this opinion.
Affirmed in part; vacated in part; and remanded. We do not retain jurisdiction.