Commonwealth v. Meeks, No

Full title: COMMONWEALTH vs. JASON MEEKS. COMMONWEALTH vs. MICHAEL WARNER

Court: Commonwealth of Massachusetts Superior Court. SUFFOLK, SS

Date published: Sep 28, 2006

Nos. 2002-10961, 2003-10575 (Mass. Cmmw. Sep. 28, 2006)

Opinion

Nos. 2002-10961, 2003-10575.

September 28, 2006.

FINDINGS OF FACT, RULINGS OF LAW, AND ORDER ON DEFENDANTS' MOTIONS IN LIMINE TO EXCLUDE BALLISTICS EVIDENCE

RAYMOND J. BRASSARD, Justice of the Superior Court

INTRODUCTION

Defendant Jason Meeks ("Meeks") and defendant Michael Warner ("Warner") have been charged with murder in two separate, unrelated incidents. Their cases are joined in this motion solely for the purpose of seeking the exclusion of ballistics evidence in each of their respective cases in their separateDaubert-Lanigan motions. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 592-593 (1993);Commonwealth v. Lanigan, 419 Mass. 15, 26 (1994). Officer Martin Lydon ("Lydon") and Sergeant Detective Catherine Doherty ("Doherty"), both of the Firearms Examination Unit ("Unit") at the Boston Police Department ("BPD") evaluated the firearms identification evidence in both Meeks' case, Commonwealth v.Meeks, Crim. No. 2002-10961 (Suffolk Super. Ct.) and Warner's case, Commonwealth v. Warner, Crim. No. 2003-10575 (Suffolk Super. Ct.). This court conducted an evidentiary hearing over the course of ten days, from Wednesday, May 17, 2006, through Friday, June 2, 2006, during which time this court heard testimony from seven witnesses. For the following reasons, the defendants' motions to exclude firearms identification evidence in their respective cases are denied.

FINDINGS OF FACT

The following facts are taken from the witnesses' affidavits and their testimony at the evidentiary hearing, which this court found credible except where noted.

I. Basics of the Two Cases

As noted above, Warner's and Meeks' cases have no connection beyond the legal argument regarding the exclusion of ballistics evidence at their trials and the fact that Lydon and Doherty conducted the ballistics examinations in both cases. Although the Unit, and Lydon in particular, did not initially follow good firearms examination practices with respect to either case and the initial documentation of the examinations was inadequate, the later examinations were in accord with general practice. Additionally, the evidence was maintained in the same condition since the time of recovery,¹ and thus was available for the Unit's firearms examiners to re-examine.

1 All evidence is handled in the same manner. First, all evidence carries a "CC" number assigned to the case that never changes; the evidence usually receives this number at the scene. The Unit receives the evidence through a "window" from the hallway into the Unit. At this time, the Unit gives the evidence a Unit number, or BU#, and logs in the evidence to the Ballistics Case Management System using both the CC# and the "BU#." The BU# is an internal number that only the Unit uses to identify the evidence.

A. Commonwealth v. Warner

On December 19, 2000, Lydon responded to a crime scene at 66 Walnut Avenue, Roxbury, Massachusetts, where the alleged murder of John Rodrigues ("Rodrigues") had occurred. At that crime scene, Lydon recovered two CCI brand .25 caliber spent² shell casings ("Item #2" and "Item #3"), one .22 caliber spent shell casing ("Item #5"), and one .25 caliber spent copper jacketed bullet ("Item #6"). Lydon later retrieved from Boston Medical Center the .25 caliber spent copper jacketed bullet removed from Rodrigues' body.³

2 During his testimony, Lydon defined "spent" as meaning the piece of ammunition had been fired through the weapon.

3 Lydon testified that only evidence retrieved from a crime scene receives an item number, therefore this bullet removed from Rodrigues' body does not have a number. As indicated below with respect to bullets removed from the alleged victim in Meeks, if there had been more than one bullet removed from Rodrigues' body, Lydon would have numbered or lettered them in order to be able to distinguish between or among them. Evidence recovered from a crime scene such as a bullet or shell casing is placed in a small envelope; evidence recovered from an alleged victim's body is kept in a small bottle.

Lydon examined under the comparison microscope the two shell casings recovered from the scene, Items #2 and #3. After conducting this side-by-side examination, he found that they "shared sufficient ballistics characteristics to lead to the determination that both were fired from the same weapon." LydonWarner Report, Exhibit 8,⁴ at 1. Lydon then compared under the microscope the spent copper jacketed bullet recovered from the scene, Item #6, with the spent copper jacketed bullet removed from Rodrigues' body. On these spent bullets, Lydon examined the land-engraved areas, groove-engraved areas, and striations⁵ and concluded that they had "been fired from the same [as yet unrecovered] weapon." Id. ⁶

4 Except where otherwise noted, the cited exhibits are those admitted at the hearing before this court.

5 As discussed further below in the context of the manufacture of firearms, the barrels of pistols and revolvers have raised areas called "lands" and depressed areas called "grooves." An underlying premise of firearms examination is that, as the bullet travels through the barrel upon firing, the lands and grooves are imparted upon the bullets; the "land-engraved" areas on the spent bullet are depressed and the "groove-engraved" areas on the spent bullet are raised. The striations are the lines that appear within the land-engraved and groove-engraved areas. Another underlying premise of firearms examinations is that firearms examiners can examine those striations under the comparison microscope and identify or eliminate the spent bullet as having been fired from a specific firearm.

6 "It is not uncommon . . . for an examiner to write a report that states that sufficient microscopic agreement is present to suggest that the same tool made the series of toolmarks, but that a conclusive opinion can be rendered only after an examination of the responsible tool. Once the examiner has the tool, the working surface [e.g., the barrel] can be evaluated to determine if the tool produces a unique toolmark, or is one that contains subclass characteristics [discussed further below] that are capable of being transferred to toolmarked surfaces." Alfred Biasotti John Murdock, "Scientific Issues," in 4 David L. Faigman et al., Modern Scientific Evidence — The Law and Science of Expert Testimony § 36:7, at 410 (2005-2006 ed.) (Exhibit 83A) (emphasis in original). Given this apparent practice of firearms examiners, it may well be a valid criticism that Lydon identified the evidence as having been fired from the same firearm prior to actually possessing the firearm. As discussed further below, however, Lydon re-examined the evidence once a firearm was recovered and reached the same conclusion of identification. Therefore, any arguable unreliability was ultimately negated with the comparison between the test fires and the evidence.

On December 24, 2000, an Intratec .25 caliber semi-automatic pistol, Model Protec — 25 ("Intratec"),⁷ was recovered in Brookline, Massachusetts. Lydon did not learn of the existence of this firearm until July 1, 2001, when Detective Juan Torres ("Torres") of BPD's Homicide Unit, accompanied by a Brookline Police Officer, asked Lydon to examine the Intratec in connection with the Rodrigues homicide.⁸ Torres had information to the effect that the Intratec could have been involved in the Rodrigues homicide,⁹ but he did not divulge to Lydon any of the details he had received. He merely asked Lydon to determine if the Intratec was an operable firearm and, if it was, to conduct a test fire and to compare the test fires to the ballistics evidence connected to the Rodrigues homicide. Lydon understood the matter to be one of great urgency.¹⁰

7 Ronald Borgio, a Smith Wesson employee who testified on behalf of the Commonwealth, examined the Intratec firearm allegedly involved in the Rodrigues homicide. He noted the significant damage to the weapon, namely that the grips had been removed from the side area; that the side had been smashed in with significant force to cause deformation and to preclude the removal of the magazine without tools; and that part of a spring was protruding from the side of the frame. He observed what appeared to be significant corrosion on the interior of the barrel and minor rusting on the exterior of the barrel. Die-casting had been used for the frame and a low-quality steel had been used for the barrel, both of which are inferior materials to those used by Smith Wesson.
Although he does not know the manufacturing specifications for this type of pistol, Borgio could tell from his visual examination that the barrel had been manufactured out of a solid piece of steel and that it appeared to have been drilled, and, as the hole was straight and there were no gross marks from the drill, it was likely reamed and broached, terms that will be discussed further below. See Borgio Hearing Testimony, p. 78, ll. 14-19. He also observed that the breech face had been made with a rotary tool because the breech face does not go through the ejector port. The Intratec's extractor looked thinner to Borgio than that in its Smith Wesson counterpart and it looked like it had been made out of sheet metal, thereby suggesting a different machining process than that used in the Smith Wesson.

8 At first, this Intratec weapon did not enter the Unit through the "window" because its relevance was unknown. The firearm would have been returned to Brookline if nothing had come of Lydon's test firing. Once Lydon identified that firearm as having been involved in the Rodrigues homicide, it was logged in to the Unit.

9 Torres testified that he had previously asked an examiner in the Unit to conduct an examination of ballistics evidence on two other, unrelated occasions; on one occasion, the examiner found a match, and on the other occasion, the examiner found an exclusion.

10 Lydon also testified that he regularly receives "hot tips" that require ballistics examinations, the majority of which result in exclusions.

When Lydon received the Intratec on July 1, 2001, he found that it was not in good condition, and that it looked like a vehicle had driven over it. With some difficulty, and using only his hands and no tools to load ammunition into the Intratec,¹¹ Lydon managed to fire a test round into a water tank; Torres and the Brookline Police Officer were present during the test firing.¹² Lydon's notes do not indicate how many test fires he performed that day. While on the stand, however, Lydon looked inside the evidence envelopes, and he testified that he used aluminum "CCI" ammunition for one of the test fires ("CCI test fire") and "Federal Cartridge" ammunition for two of the test fires ("Federal Cartridge test fires"). See Exhibit 50 (envelope containing test fires). The shells retrieved at the scene were both brass. Lydon retrieved the spent test fire shell casings, and, that same day, he compared one of them under the comparison microscope with Items #2 and #3. He also retrieved the spent test fire bullets and compared one of them under the comparison microscope with Item #6 and with the bullet recovered from Rodrigues' body. Lydon's report does not indicate whether he used the CCI test fire or the Federal Cartridge test fire(s) for these examinations.

11 The Intratec was damaged in such a way that the magazine would not feed the ammunition fully into the chamber because "the magazine was jammed into the gripframe. . . ." Lydon Warner Report, Exhibit 8, at 3. As a result, Lydon had to pull back the slide and feed the bullet into the chamber manually. The effect of this procedure is that there is a lack of extractor and ejector makrs on the fired shell casing.

12 There is some dispute regarding whether anyone witnessed Lydon's test fire of the Intratec weapon. Although resolution of this dispute does not affect the reliability of Lydon's examination, this court concludes that the evidence, specifically the Brookline Police Department report dated July 13, 2001 ("Brookline Report, Exhibit 45"), supports the finding that Torres and the Brookline Police Officer witnessed Lydon's test fire. In that report, Sergeant William Riley ("Riley") wrote that he and Officer Carroll brought the Intreatec weapon to BPD and met with Torres. "The weapon was brought to the . . . Unit and examined by Officer Martin Lydon. Officer Lydon examined the weapon and with some effort pulled the slide to the rear, inserted a cartridge, pushed the slide forward and then fired the weapon. Officer Lydon repeated this two more times." Brookline Report, Exhibit 45. Riley further wrote that, on July 12, 2001, he "spoke to the ADA Orlando Dimambro who advised [Riley] that the complaints [concerning the Intratec weapon] had been dismissed with out [sic] prejudice at the Commonwealths [sic] request. The reason for the dismissal was the impression that the [Intratec] weapon was inoperable, however upon examination by the Boston Police the weapon is operable, and was fired 3 times in [Riley's] presence." Id. (emphasis added). With respect to this report, Lydon testified that he does not recall that anyone witnessed his test firing of the Intratec, but that he believes Riley's report.

With respect to the shell casings, Items #2 and #3, Lydon examined the breech face markings at the rear of the shells. He found three areas of examination on the heads of the casings. He determined that the head-stamped area showed that the two shell casings were "likely" fired from the same weapon as the test fired casing on the basis of these markings. Because he was unable to conclude with definiteness that the two casings had been fired from the same firearm as the test fired casing on the basis of these markings alone, Lydon looked at the chamber markings. See Exhibit 20 (photograph of chamber marks). "Chamber marks," according to Lydon, arise when the cartridge swells in the chamber area during the firing process. Upon determining that the chamber markings on all three casings matched, Lydon concluded that the two shell casings recovered from the scene, Items #2 and #3, were fired from the Intratec. With respect to the bullets, Lydon found that the individual markings and striations were in sufficient agreement and concluded that both the spent bullet recovered from the scene, Item #6, and the bullet recovered from Rodrigues' body were fired from the Intratec. He recorded all of these conclusions in his report.

In his report, Lydon wrote that he compared the recovered shell casings and bullets to the test fire without detailing the examination. He rendered a generalized opinion, writing, "[f]rom these comparisons a determination was made that the two .25 caliber casings [Items #2 and #3], and the two spent .25 caliber bullets [Item #6 and the victim bullet] . . . were fired from the Intratec .25 caliber pistol . . . submitted with this case." Lydon Warner Report, Exhibit 8, at 3. No photographs were taken of either the shell casings or the bullets, consistent with the procedure in place at that time.¹³

13 The Unit's firearms examiners did not regularly take photographs in the course of their firearms examinations until 2004 when Doherty implemented the procedure. Photographs are never taken of areas that are different.

Years later, Lydon and Doherty re-examined this ballistics evidence and took photographs of the shell casings and the bullets. Doherty became involved in this case in February 2006 for her "supervisory review" simultaneous with Lydon's "pre-trial review."¹⁴ Doherty did her own comparison microscope examination of the bullet removed from the victim; she concluded that that bullet had been fired from the recovered Intratec weapon. She also took photographs of the areas she used for identification. See Exhibits 18, 19, 21, 22, 23 (photographs). Doherty could not recall, and her notes do not indicate, if she took these photographs in February 2006 or March 2006. Doherty also examined the shell casings in this case. She compared Item #2 and Item #3 to each other and to the CCI test fire from the recovered Intratec weapon. She photographed the comparison between either Item #2 or Item #3¹⁵ and the CCI test fire. Exhibit 20 (photograph). Again, Doherty could not recall the date on which she took that photograph.

14 Doherty's typed report on the Warner evidence is undated, but it refers to the attached "Cartridge Case Worksheet" which is dated May 21, 2006, thereby indicating that she generated this report around that same time. See Doherty Warner Report, Exhibit 61, at 2. During her testimony, Doherty stated that she took two of the pages of handwritten notes on February 18, 2006, presumably during her supervisory review of the evidence. There is no date written anywhere on this page of notes, but this court credits Doherty's testimony.

15 Doherty could not recall whether she used Item #2 or Item #3 for the comparison, and her notes do not contain this information. See Doherty Warner Report, Exhibit 61.

On May 21, 2006, as part of her own pre-trial review¹⁶ of the evidence, Doherty re-examined the evidence, filed a report, see Doherty Warner Report, Exhibit 61, and took additional photographs.¹⁷ See Exhibits 67, 68, 69, 70, 71 (photographs). Doherty examined the recovered bullets, first comparing Item #6 to the bullet removed from the victim. Her report includes a description of the examination, including the notation that she positioned both bullets with their noses facing left and that she used a "large score mark . . . as beginning reference point." Doherty Warner Report, Exhibit 61, at 1. Rotating both bullets counterclockwise, she found "clear markings for identification" on what she named land-engraved area #3.Id. She further noted that all of the bullets' land-engraved areas "show sufficient agreement. Both bullets fired from same weapon." Id. at 2.

16 Doherty characterized this review as "pre-trial" rather than "supervisory" because it was prior to her own in-court testimony at the hearing on these motions.

17 The final page of handwritten notes attached to Doherty's report indicates the date on which she took certain photographs. Doherty testified that although the date appears to read "3/18/06," it actually reads " 5/18/06." (Emphasis added). She came to this conclusion based on her own personal knowledge that she took the photographs listed on this page in May 2006 during the course of the hearing before this court. The court credits her testimony.

Second, Doherty compared the bullet removed from the victim to the Intratec test fire.¹⁸ Her report on this examination was similarly detailed, noting the same large score mark used for orientation and the counterclockwise rotation to land-engraved area "#3." She concluded that "[b]oth share sufficient agreement, sufficient characteristics, meets, exceeds CMS."¹⁹ Id. Third, she compared Item #6 to the Intratec test fire, making almost identical comments in her report. See id. Finally, Doherty concluded that "[a]ll bullet evidence, both submitted bullets and test fires[,] share sufficient characteristics in all [land-engraved areas] when using score mark as reference starting point, to state all bullets fired from the same Intratec weapon [recovered in this case]. . . ." Id.

18 Doherty's report indicates that she used the test fire from an envelope of test fires dated July 11, 2001.

19 CMS stands for "consecutively matched striae," a method of firearms examination. This method is discussed further below.

At this same time, Doherty compared the shell casings found at the scene, Items #2 and #3, to a test fired shell casing.²⁰ First Doherty compared Items #2 and #3 to each other and concluded that they "display similar characteristics relating to the orientation of extractor markings and ejector markings. Microscopic comparison reveal [sic] detailed markings in firing pin markings from ejector and location of extractor, as well as chamber markings indicate both shell casings fired from same .25 caliber semi-automatic weapon." Id. at 2-3. Second, Doherty compared the test fires against each other, finding they shared "similar characteristics in firing pin and extractor location[,] . . . [although they] lack markings due to difficulty in loading weapon." Id. at 3. Third, Doherty compared one of the test fires to Item #2, and, fourth, she compared the test fire to Item #3. Her conclusion as to each, although rendered in separate sentences, was the same, specifically that there was sufficient agreement in the chamber markings such that Items #2 and #3 were fired from the Intratec.

20 Doherty's report merely indicates she compared Items #2 and #3 to a "[r]epresentative test fire" without providing further information as to which test fire, the CCI or the Federal Cartridge, she used. See Doherty Warner Report, Exhibit 61, at 3. On the final handwritten page of her report, Doherty wrote that she photographed her comparisons of Item #2 to the CCI test fire and of Item #3 to the CCI test fire. See id. at 7.

On a "Cartridge Case Worksheet" attached to Doherty's report, Doherty wrote in the date of May 21, 2006, and included the following handwritten note: "2 shell casings (CCI)²¹ share sufficient characteristics examined against test fired . . . casing. ID area marked. Chamber markings ID'd share sufficient individual characteristics — fired from Intratec . . . Marked with `X' barrel and chamber cast for examination." Id. at 4 (footnote added). On this worksheet, Doherty also drew a sketch of the breech face and of the "headstamp."²²

21 As noted above, Lydon testified that in July 2001, he performed one test fire using CCI ammunition and two test fires using Federal Cartridge ammunition. Lydon gave this testimony after looking in the envelope of test fires admitted as evidence. See Exhibit 51 (envelope). It is unclear, then, why Doherty's notes show that there are two CCI test fires.

22 The Cartridge Case Worksheet does not indicate which shell casing Doherty drew (Item #2, Item #3, or the test fire).

Doherty also made a cast of the interior of the Intratec's barrel using Mikrosil.²³ See Exhibit 72 (envelope containing casts).²⁴ This cast of the barrel enabled her to identify marks that continued through the length of the cast, i.e., class and sub-class characteristics. Doherty was able to distinguish these long marks from individual characteristics²⁵ that do not run the entire length of the barrel. Additionally, Doherty used a borescope²⁶ to examine the inside of the barrel.

23 Essentially, to make such a cast, two chemicals are mixed together and compressed into the weapon's barrel. The Mikrosil hardens inside the barrel, then the examiner uses a long toothpick to pull the cast out of the barrel.

24 Three casts of the Intratec's barrel were admitted as Exhibit 72, and Doherty described the casts to the court. The first, #1, showed the muzzle end of the barrel; the depressed areas were the lands of the barrel, which would be the land-engraved areas of a bullet, and the raised areas were the grooves of the barrel, which would be the groove-engraved areas of a bullet. The second cast, #2, showed the beginning of the barrel where the projectile would sit, and the chamber markings were visible. The third cast, #3, was of the breech area of the weapon.

25 This term, as well as the terms "class characteristics" and "sub-class characteristics," will be defined and discussed further below.

26 Doherty described a borescope as a microscope and flashlight on the end of a stick. She noted that an examiner could also use a comparison microscope to examine the barrel of a weapon by taking away one of the stages; this method would produce the same effect as using a borescope.

Lydon re-examined the evidence in May 2006, making sketches²⁷ in which he compared Item #6 with the test fired bullet from the Intratec. Exhibit 48. Rather than "mimic" what Lydon sees under the microscope, the sketches indicate where Lydon saw "significant" marks on land-engraved areas #1 through #6. Lydon also noted a gross mark²⁸ on groove-engraved area #1, located under land #1; he used this groove-engraved area for orientation, to line up the two bullets, not for identification. The gross mark on the groove-engraved area is a sub-class characteristic, Lydon testified, because it is too thick to be an accidental marking, because it is a straight line that does not vary, and because it is located in a groove-engraved area, an area that is more susceptible to sub-class characteristics. In a paragraph at the bottom of the series of sketches, Lydon wrote that "[l]and[-engraved areas] 2, 3, and 4 have varied amounts of agreement. 5 1 only near groove[-engraved areas]. #6 near groove[-engraved areas] and [one] mark toward forward end of bullet. Gross markings on groove[-engraved area] between land[-engraved area] #1 and land[-engraved area] #6. Found sufficient agreement between test fire and bullet from scene (cone #6)." Id. at 1-2.

27 Later in his testimony, Lydon stated that once Doherty became commander of the Unit in 2004, he began "drawing diagrams."

28 Although the witnesses did not so testify, it is likely that this "gross mark" Lydon used for orientation is the same "large score mark" that Doherty used as a beginning reference point in her examinations. See Doherty Warner Report, Exhibit 61, at 1.

Lydon therefore examined the evidence from the Warner case on four different occasions: initially in December 2000; when he received the Intratec pistol in July 2001; in February 2006 when he learned that a court date had been scheduled for this case; and in May 2006 for the hearing before this court on Warner'sDaubert-Lanigan motion. Doherty examined the evidence on at least three different occasions as well, in February and May 2006, and possibly in March 2006; Doherty photographed the examinations in February 2006 and May 2006. The lack of documentation throughout the initial examinations, although troublesome, does not affect the end result, that after each examination, both Lydon and Doherty reached the same conclusions, that the spent shell casings and bullets connected to the Rodrigues homicide were fired from the Intratec. Moreover, Lydon and Doherty better documented their later examinations. Although these more detailed notes and reports still contain omissions, they reflect an attempt to accurately describe the examination processes, thereby contributing to subsequent reproducibility as well as meaningful opinion testimony.

B. Commonwealth v. Meeks

On November 1, 2001, Lydon responded to a crime scene at 123 Blue Hill Avenue in Roxbury, Massachusetts, where Alvaro Sanders ("Sanders") was allegedly shot, and he recovered two spent .38/.357²⁹ bullets ("Item #1" and "Item #11"). On November 7, 2001, Detective Carl Washington ("Washington")³⁰ received two glass jars from the Medical Examiner's office each of which contained one spent .38/.357 caliber bullet ("Item A" and "Item B")³¹ that had been removed from Sanders' body.

29 According to Lydon, this "dual designation" is unavoidable because the lengths of a spent .38 bullet and a spent .357 bullet are the same. However, one can fire a .38 bullet from a .357 firearm, but one cannot fire a .357 bullet from a .38 firearm.

30 Washington is now retired.

31 As Lydon looked at Items A and B in the courtroom, he noted that Item A is the larger of the two bullets and that Item B is actually in two pieces as a result of damage it had sustained.

With respect to each of these pieces of ballistics evidence, Lydon filled out BPD "Ballistics Worksheets" on which he noted the caliber and their recovery location; he noted the weight of Items A and B as well. See Exhibit 47. The worksheets have spaces for the examiner to note the widths of the land- and groove-engraved areas, but Lydon did not enter this information with respect to any of the evidence.³² There is also a space for the examiner to enter information about the firearm itself, but, as no firearm had been recovered at that time, these spaces remained blank.

32 Lydon noted in his testimony that these worksheets are not important in a homicide case.

Lydon examined Items #1 and #11 under the comparison microscope and determined that there were not enough markings on these spent bullets to compare because of the amount of damage³³ they had sustained; he could not see the land- and groove-engraved areas on these bullets. He did note, however, that Item #11 had "one land[-engraved area] that is discernable [sic] to the point where it's [sic] width was able to be compared with the width of the land[-engraved areas] on [Items A and B] . . . and the width of the land[-engraved areas] on these 3 bullets was consistent." Lydon Meeks Report, Exhibit 9, at 2. Lydon also examined Items A and B under the comparison microscope and saw that they, too, had been badly damaged so he could not see all of the land- and groove-engraved areas. Lydon noted the damage to Items A and B in his report, and concluded that he "found [them] to have been fired from the same [as yet unrecovered] weapon." Id. At this point, he could not tell the type of weapon that had fired those bullets, but he knew the class characteristics of the gun included five lands, five grooves, and a right twist.

33 Lydon defined "terminal damage" as the damage sustained when a projectile reaches a target. Bullets that have been fired and that have sustained damage are harder to study not only because the damage can smooth out or roughen up the surfaces, but also because pieces of the bullet can break off as a result of the damage. Damage to lead bullets is not unusual because the metal is softer than "jacketed bullets."

On January 8, 2002, Lydon and Washington reported to a crime scene in Roxbury, Massachusetts. A Smith Wesson .357 revolver³⁴ was recovered at 29 Dennis Street, Roxbury, and one spent copper jacketed .38/.357 bullet was recovered around the corner from 29 Dennis Street at 43 Winthrop Street, Roxbury.³⁵ Washington did the initial testing on these items, and, in a report, titled "Ballistician's Office" and dated January 22, 2002, he wrote only that he examined three .357 cartridges and that, "[a]fter microscopic examination, compared with the test fires[,] it was found to have been all fired in the above weapon[,] a [.]357 Smith Wesson. . . ." Lydon Meeks Report, Exhibit 9, at 5.

34 As he did with the Intratec, above, Ronald Borgio a Smith Wesson employee, examined the Smith Wesson revolver allegedly involved in the Sanders homicide and noted that it is a .357 magnum caliber revolver capable of firing six rounds, and that it has a four inch barrel. Based on his knowledge of Smith Wesson serial numbers, Borgio concluded that this revolver was "probably produced sometime around 1980" in Springfield, Massachusetts. Borgio Hearing Testimony, p. 9, l. 16. Currently, Smith Wesson in Springfield uses Computer Numerical Control ("CNC") machines to manufacture its firearms; these machines are automatic as opposed to manual. In 1980, however, the Springfield plant had "relatively few, if any, CNC machines. . . ." Id. at p. 38, ll. 13-14. Although the types of tools have not changed with the use of CNC machines, and although the process is essentially the same, prior to CNC machines, "operators would have had to use fixtures to locate the six [cylinder] holes. And they would have had to drill those holes, and then in a separate operation ream the holes." Id. ll. 21-24. With CNC, "there is no fixturing. The machine is programed to drill and then ream the holes. So the concept and the cutting process is the same. The degree of automation is higher today than it was in 1980." Id. at p. 39, ll. 2-6. In Borgio's opinion, this revolver was not reamed or broached with a CNC machine. With this type of revolver, the unfired ammunition moves from the cylinder to the barrel; the tool that files the area between the cylinder and the barrel is a handtool both at the present time as well as in 1980.
According to Borgio, the frame of the Smith Wesson allegedly involved in the Sanders homicide appeared to be in good functioning order, the cylinder lined up correctly and locked. Borgio observed no corrosion in the chamber or inside the barrel, although he could see some residue of unburned powder on the inside of the barrel. He could tell that the revolver had not been fired often, but, as he could see powder marks in each of the cylinders, he knew that it had been fired at least six times; he speculated that the revolver could have been fired as few as fifty times or as many as one hundred and twenty times. Looking down into the barrel, Borgio could see the lands and grooves. He concluded that the revolver might have been dropped on the front sight, but it did not appear to have been damaged as a result.

35 During his cross-examination of Lydon, defense counsel asserted that the distance between the two locations is less than 0.4 miles; upon examining the mapquest.com map defense counsel showed him, Lydon confirmed that distance. See Exhibit 41. While Lydon acknowledged that his report clearly states that both the bullet and the firearm were recovered at 29 Dennis Street, he noted that he responded to 29 Dennis Street initially, therefore that was the address he used consistently; if he recovered anything else that was related to the 29 Dennis Street crime scene, even if found elsewhere, he would use "29 Dennis Street" as its identifier.

On August 8, 2002, Daniel Coleman ("Coleman"), who was, at that time, a sergeant detective in BPD's Homicide Unit,³⁶ contacted Lydon and told him that he had reason to believe that the Smith Wesson .357 revolver recovered on January 8, 2002, had been used in the Sanders' homicide. Coleman did not disclose any information to Lydon; he merely asked that a firearms examiner review the evidence.

36 Coleman is now the Deputy Superintendent of the BPD's Homicide Unit.

Thereafter,³⁷ Lydon test fired the Smith Wesson and compared the test fire with one of the bullets recovered from Sanders' body; Lydon made no notation in his report, dated August 12, 2002, whether he used Item A or Item B³⁸ for the comparison and, during his testimony, he could not recall which bullet he had used.³⁹ Upon examining the two bullets, Lydon found that striations on "the one good land[-engraved area] from the victim bullet matched one of the land[-engraved areas] from the test fire bullet[,]" and he concluded that the bullet removed from Sanders' body had been fired from the Smith Wesson .357 revolver recovered in January 2002. Lydon Meeks Report, Exhibit 9, at 3. Although his report does not contain any information to this effect, Lydon testified that he also attempted to compare Items #1 and #11 with the test fired bullet, but they were both too damaged to enable him to identify or eliminate them as having been fired from the same revolver.

37 Even though his report indicates that he examined this evidence on August 8, 2002, Lydon testified that he could not recall if that was the correct date; it could have been that day or it could have been one or two days later. It was not uncommon for Lydon to write up his report a day or more after having conducted the examination.

38 Although he did not identify the bullet he compared in his August 2002 report, Lydon testified that the bullet he matched to the test fire was Item A. It is his usual practice to test both bullets in evidence against the test fire, but he did not do so in this case. He explained that it was likely that Item B had sustained too much damage to compare it to the test fired bullet; that notwithstanding, he could match at least one area on Item B to Item A in August 2002 despite the very small areas capable of being examined on each bullet. Lydon explained, in a hypothetical example, that "Zone 1" of Bullet A matches the test fired bullet; "Zone 2" of Bullet A matches Bullet B; Bullet B is too damaged to see "Zone 1," so it cannot be matched to the test fired bullet.

39 During his direct examination, Lydon took Item A and Item B out of their respective jars in order to determine which one appeared in the enlarged photographs presented to the court. Upon looking at the two bullets, Lydon believed that he had returned them to the wrong jars, so, while on the stand, Lydon switched the two bullets. On cross-examination, Lydon revealed that he had examined Items A and B further and determined that he had not put the two bullets in the wrong jars, therefore his testimony on direct (to the effect that he had returned the bullets to the wrong jars) had been incorrect. He placed each bullet in its proper jar.

Additionally, Lydon compared the test fired bullet from the Smith Wesson .357 revolver recovered at 29 Dennis Street with the bullet recovered at 43 Winthrop Street and concluded, as Washington had, that they had been fired from the same weapon, namely the recovered Smith Wesson revolver. Finally, Lydon compared the bullet recovered on January 8, 2002, at 43 Winthrop Street with one of the bullets removed from Sanders' body. Again, Lydon did not note at the time whether he had used Item A or Item B, and he could not recall when testifying. He concluded that "one of the land[-engraved areas] of this bullet [recovered at 43 Winthrop Street] matched the land[-engraved area] from the victim bullet." Id. "In conclusion, the bullet removed from the victim's body [either Item A or B], the bullet recovered on 1/8/02 at 29 Dennis St., and the test fire bullet from the Smith and Wesson revolver from the Dennis St. case, each have 5 land[-engraved areas], 5 groove[-engraved areas], and a right twist, and all 3 share individual characteristics on at least 1 land[-engraved area]." Id.

In response to a defense counsel request for photomicrographs some time in early 2005, Lydon re-compared one of the bullets removed from Sanders' body with the August 2002 test fired bullet from the recovered Smith Wesson revolver. After examining the two bullets over a period of approximately one week,⁴⁰ Lydon was unable to re-locate "the one good land[-engraved area]" on whichever one of the victim bullets he had found it on in August 2002. See id. at 1, 3.

40 Lydon testified that he did not examine the evidence relating to the Sanders' homicide exclusively for one week; he would examine the evidence for a time, put it aside to work on something else, then go back to the evidence again.

Doherty recalled that Lydon had approached her about needing photographs taken of his examination. She knew Lydon had previously found a match, but she had limited information about the case. She testified that her initial understanding was that Lydon needed photographs, not that he was having difficulty finding the area of correspondence. Lydon then performed another test fire of the recovered Smith Wesson revolver. Doherty compared the January 2005 test fire to Item A recovered from the body.

According to Lydon's report, he observed on the January 2005 test fired bullet five land-engraved areas, five groove-engraved areas, and a right twist. "The bullet removed from the victim's body ha[d] those characteristics, but only three land[-engraved areas] and two groove[-engraved areas] [that were] of any use in [his] ballistic comparison due to damage sustained by this bullet." Id. at 1. Lydon testified that he found these areas of significance after Doherty pointed them out; in her own testimony, Doherty stated that she did not recall that she found an area that Lydon had not, but that she personally observed three areas of identification on the land-engraved area of the evidence bullet that led her to conclude that there was sufficient agreement in the matching striae of Item A and the test fire. Doherty also took photographs of the evidence. See Exhibits 14A, 15A, 16A.⁴¹ Lydon testified that it is not rare for two examiners to find different areas of significance⁴² on the same piece of evidence. Doherty corroborated this testimony, stating that she has occasionally found other areas of comparison based on the use of light and rotation under the scope.

41 Exhibits 14, 15, and 16 are photocopies of the actual digital photographs which are Exhibits 14A, 15A, and 16A. Exhibits 15A and 16A are photographs of the land-engraved areas and Exhibit 14A is a photograph of a groove-engraved area. Doherty testified that she did not consider the groove-engraved area to be an area of significance; when defense counsel asked her why she bothered taking a photograph of that area, she said she did not know.

42 Marks that are "areas of significance" are not necessarily marks that result in an identification. For example, with respect to the January 2005 test fired bullet and the bullet recovered from the victim's body, Lydon found three good land-engraved areas and two groove-engraved areas on which he could observe marks or areas of significance; the fact that he only used two land-engraved areas and one groove-engraved area to make an identification merely means that there was a lack of matching patterns on the remaining land-engraved area and groove-engraved area. Based on his own experience, Lydon asserted that it would be impossible to find a match based on two land-engraved areas and one groove-engraved area and to conclude nonetheless that the lack of matching patterns on the remaining land-engraved area and groove-engraved area negated that identification.

Doherty and Lydon revisited the evidence from this case in February 2006 for Lydon's pretrial review and for Doherty's supervisory review. They both compared Item A and one of the test fired bullets under the comparison microscope, and they made sketches and further documentation.⁴³ Doherty also testified that she compared Item A to one of the test fires and took photographs in March 2006,⁴⁴ and that she compared Item B to one of the test fires sometime in May 2006.⁴⁵ She also compared Items A and B to each other.

43 It appears that the only documentation of these February 2006 reviews is one page of handwritten notes attached to Doherty's typed report. See Doherty Meeks Report, Exhibit 60, at 9.

44 Similarly, the only documentation of this March 2006 examination is one page of handwritten notes attached to Doherty's typed report. See Doherty Meeks Report, Exhibit 60, at 8.

45 Doherty testified that she did this examination of Item B after learning of Lydon's testimony at the hearing before this court that he did not compare Item B to the test fired bullets; in fact, she testified, she was asked to re-examine the evidence because of the confusion that had resulted from Lydon's testimony. At this time, she also took photographs. See Exhibits 62, 63, 64 (all three are copies of photographs of a land-engraved area of Item B, on the left, and one of the test fires, on the right; each copy is a closer view to display the matching striae); Exhibit 65 (copy of a photograph separating, and not comparing, Item B from the test fire to show that Item B is deformed and to show the overall contour of the land- and groove-engraved areas); Exhibit 66 (copy of a photograph showing the orientation of the two bullets and showing that Item B is mangled and deformed). Doherty discussed these photographs during her testimony and indicated areas of similarity by drawing a rectangle around them.

In her undated typed report,⁴⁶ Doherty described her examination of the evidence in some detail. First, she compared Item #1 to Item #11 and found that, although they "share similar width of [land-engraved areas], these observable [land-engraved areas] in the above bullets are inconclusive as to having been fired from the same weapon." Doherty Meeks Report, Exhibit 60, at 1. Second, Doherty compared land-engraved area "#1 (marked with black sharpie)" of Item A to Item B's "middle [land-engraved area], (deformed, folded over)," and found there were "matching stria[e] at base of [land-engraved areas] on both specimens (4x, 4x CMS)."⁴⁷ Id. Doherty therefore concluded that Items A and B were "fired from the same gun. . . ." Id.

46 In the handwritten notes attached to Doherty's typed report, Exhibit 60, Doherty wrote that she compared Item A to Items #1 and #11 and to Washington's test fire dated January 8, 2002. Her handwritten notes also indicate that she matched Item A to Item B and compared Item B to the January 8, 2002, test fire. There was some confusion during Doherty's testimony regarding when, exactly, she took these notes. Doherty concluded that the third page of handwritten notes, containing in the bottom right-hand corner the date May 22, 2006, was actually the first page of her notes from that date; the second page of handwritten notes was the second page of her notes from that date; and the first page of handwritten notes was the last page of notes from that date. Confusion also resulted from the fact that Doherty's "5" in this date notation looks like a "3." She testified that she interprets the number as a "5" because she specifically recalls doing the examination and taking the notes. This portion of Doherty's testimony took place on Wednesday, May 31, 2006. The court credits this testimony. Doherty therefore only examined Item B in May 2006 at which time she found that it had been fired from the same weapon as Item A and the bullets test fired from the recovered Smith Wesson. See Doherty Meeks Report, Exhibit 60. As Doherty's typed report discusses her examination of Item B, it is reasonable for this court to conclude that Doherty wrote this typed report in May 2006, after Lydon testified at the hearing on these motions.

47 Doherty wrote that "[t]aking into account the change to evidence submitted and the composition of the evidence (lead bullets . . .), consecutive matching stria were located within the respective evidence bullets and photographed, whenever possible." Doherty Meeks Report, Exhibit 60, at 3.

Third, Doherty compared land-engraved area "#3" of the one "fully visible" land-engraved area on Item #11 and concluded that there were "[m]atching stria[e] [on those two areas] . . . (marked with black sharpie), and matching CMS (3x, 4x) . . . [such that Item] `A' was fired from the same gun as [Item] # `11'." Id. at 2. Fourth, Doherty noted that test fires of the recovered Smith Wesson revolver were taken on January 8, 2002, and that "additional test fires were done during . . . February 2006. These test fires were examined for markings, and several were chosen for further examination against the evidence. . . ."Id. Fifth, Doherty compared "[t]est fires"⁴⁸ to the bullet recovered at Dennis Street; she "positively identified [the latter] as having been fired from the [recovered] .357 Smith and Wesson revolver. . . ." Id.

48 Doherty does not specify which test fires she used for this examination, or whether she used all of the test fires.

Sixth, Doherty compared the "[t]est fires"⁴⁹ to Items A and B and Items #1 and #11. She concluded that they "[a]ll share characteristics of being fired from a .38/.357 Smith and Wesson weapon having a right-hand twist and equal [land-engraved area] dimensions. The groove engraved areas which were clear on the evidence bullets [Items A, B, #1, and #11] were examined against the groove engraved areas of the test fired bullets, but were not used for identification, instead they were used for orientation, as there were multiple identifiable stria[e] in these areas."Id. at 2-3 (footnote added).

49 Again, Doherty did not specify which or how many test fires she used for this examination.

In this written report, Doherty also discusses the Mikrosil cast that she took of the barrel of the Smith Wesson. See Exhibit 73 (envelope containing cast). As part of her examination, she cut this cast into two pieces and examined both under the comparison microscope, focusing on the continuity, or carry-over, of lines in the groove-engraved areas caused by the tool that did the rifling. This examination "revealed areas in the groove engraved areas that ran the length of the cast, indicating the possibility of sub-class characteristics being present, and therefore the land engraved areas were examined for identifiable areas suitable for photography as a representation of areas used for identification." Doherty Meeks Report, Exhibit 60, at 3.

Thus, Lydon examined the ballistics evidence in the Meeks case five times: initially in November 2001; in August 2002 when the Smith Wesson was brought to the Unit; in January 2005, both alone and with Doherty, after ADA Haggan asked for photographs of the identifications; in February 2006 as a case review in preparation for the hearing before this court; and in May 2006 during the hearing. Similarly, Doherty examined the evidence four times. The lack of detailed documentation with respect to the examinations of the evidence in this case prevented Lydon from clearly articulating the examination process during his hearing testimony. Not only was Lydon unable to recall which test fires he used during which examination, but he was also unable to recall which of the bullets recovered from the victim he examined. Additionally, Doherty's notes generated some confusion in that it was unclear on what date they were made. That notwithstanding, and despite Lydon's January 2005 difficulties, neither Lydon nor Doherty deviated from their identification of Items A and B and Items #1 and #11 as having been fired from the recovered Smith Wesson revolver. Moreover, Doherty's May 2006 report contains significant detail, contributing to subsequent reproducibility.

II. Tour

On Wednesday, May 17, 2006, this court, accompanied by the Commonwealth and defense counsel, toured the Unit in the BPD's Roxbury, Massachusetts, headquarters. A detective in the Unit uninvolved in either of these cases conducted the tour; Doherty was present. The tour began with the three stations comprising the Integrated Ballistics Identification System ("IBIS"). IBIS is a database into which operators scan images of spent shell casings and projectiles.⁵⁰ The BPD's IBIS operators are civilians, not police officers. In order to input an image into IBIS, the civilian operator places the shell casing or projectile into a grip, similar to that on a microscope, and a picture of the piece of evidence appears on the computer screen at a magnification resolution of approximately 125.⁵¹ For shell casings, for example, the operator takes one image of the entire casing, one image of the breech face impression, and one image of the firing pin impression; the ejector marks are not inputted into IBIS.

50 Spent projectiles are the same as spent bullets. This court uses the two terms interchangeably. Additionally, spent shell casings are the same as spent cartridge cases, and the court uses those terms interchangeably as well.

51 This resolution has never been upgraded, therefore an IBIS operator can view images from 1995 and 2005 side-by-side. The only change IBIS has implemented is with its lighting. Originally, IBIS only used oblique lighting, and DrugFire, the FBI's ballistics database, used reverse imaging. When DrugFire merged with IBIS, IBIS began to provide both types of lighting.

Once the images are inputted, they go to the main server; the computer then gives the pieces of evidence a score based on a mathematical algorithm and, applying that score, creates a list of possible match candidates. The search results are limited regionally,⁵² but, as IBIS is a national database, the operator can expand the search to other regions throughout the country. The operators can also customize the search by area of comparison, focusing only on the "best breech face score," for example. Clicking on each possibility that the computer has listed brings up an image of that possible match. The operator then compares that image with the image of the piece of evidence in order to determine if s/he should print out the images to give to Doherty who would then contact the firearms examiner assigned to the piece of evidence.

52 Massachusetts is in "Region 10."

The tour also included two of the Unit's three comparison microscopes; a laboratory where, for example, examiners make casts of firearms barrels; the test firing room and its contents; and the ammunition room, which contains not only boxes of various types of ammunition but also recovered firearms ready for destruction. The Unit detective explained that the comparative microscope uses oblique lighting; the examiner manipulates the angle of the light to obtain the best view of the bullets or shell casings. The pieces of evidence can be moved as well. The comparative microscope starts with a default of a 1.25 magnification which the examiner can increase to forty.

Another Unit detective test fired a gun, both into the tank of water so that the expelled bullets could be retrieved, and into an empty metal container to test that the gun functioned properly. The tour of the Unit also included the evidence room, and the BPD detective pointed out the Unit's method of using cardboard boxes to hold the all of the evidence in one case. There appeared to be two sizes of boxes, a larger size that would accommodate a firearm as well as other ballistics evidence and a smaller size that would not accommodate a firearm.⁵³

53 It is unclear if the size of box changes as the case progresses, nor is it clear what size box Lydon used initially in either Warner's or Meeks' case. Lydon brought the evidence boxes for each of these cases with him when he testified, and the evidence from both of the defendants' cases necessitated the larger box.

The tour was helpful and aided the court's understanding of the evidence.

III. Firearms Examination

Firearms examiners Mary-Jacque Mann and Peter Striupaitis both testified that an assumption underlying the field of firearms examinations is that the manufacturing process of firearms leaves unique marks on the surface of firearms; firing the firearm imparts those unique marks onto the projectiles and shell casings such that, as a result of the unique marks that tools have imparted, trained examiners are able to identify those marks as having been made by a particular firearm. Peter Striupaitis, also a former AFTE⁵⁴ president, testified that these principles are generally accepted among the firearms and toolmark community, a community that consists of approximately 1,100 individuals worldwide.

54 AFTE is the Association of Firearm and Toolmark Examiners, discussed further below.

According to Peter Striupaitis, firearms investigations began in the early twentieth century with Calvin Goddard, and they are now widely conducted throughout the United States on local, county, state, and federal levels. The comparison microscope has historically been used for firearms examinations and its use is still widespread. Mary-Jacque Mann pointed out that the way in which an examiner observes those discernible marks on projectiles and shell casings under the comparison microscope differs from one examiner to another, and the human element of firearms examinations can likely never be eliminated. There is no universal standard as to when a match is made or not made; rather, it is based on the experience and training of the examiner. Mary-Jacque Mann also acknowledged that there is no database for an examiner to look to when making examinations. She opined that a database would not be helpful because barrels can change over time; therefore, the marks they leave (individual characteristics) will also change over time. In her view, a statistical approach is neither practical nor necessary.

As noted, examiners in the field of firearms examination operate under the assumption that tools leave unique marks on the surfaces of firearms during their manufacture and that those marks, in turn, are impressed onto the surfaces of fired shell casings and projectiles. Relying on their training and experience, examiners are then able to reach conclusions about fired shell casings and projectiles based on those imparted markings. The manufacturing and firing processes of firearms support this assumption.

A. The Manufacturing Process

Ronald Borgio ("Borgio"), an engineer employed since 1978 at Smith Wesson in Springfield, Massachusetts, testified for the Commonwealth on the manufacturing process, function, and performance of firearms, particularly Smith Wesson firearms. Borgio's primary duty in his current position as Smith Wesson's Director of Total Quality Management is to ensure that the manufacturing process is to specification as to each firearm. Borgio testified that he is familiar with the premise that microscopic marks are left on the interior of the barrel as a result of manufacture and are thereafter transferred onto the bullet and shell casing upon being fired. Although he has not studied the marks left on cartridge cases and projectiles, he has worked with firearms examiners and observed such markings, and he has looked through a comparison microscope approximately twelve times over the course of his career. Additionally, his background and education as an engineer enable him to understand the random markings that occur on the interiors of barrels.⁵⁵

55 Borgio testified that, based on his knowledge, he does not know of any design process that "would result in two identical barrels. . . . [i]dentical meaning [with respect to] the microscopic marks[.]" Borgio Hearing Testimony, p. 76, ll. 15-20.

The barrel of a Smith Wesson firearm, whether it is a pistol or a revolver,⁵⁶ begins as a separate piece of solid steel approximately six inches long. Certain pieces of powered machinery then create the barrel; these tools cause random markings to appear on fired projectiles. First, after the excess metal is machined away, the drill passes through to make an initial hole; this hole is the opening in the barrel through which a projectile passes, although it is not yet in its final configuration.⁵⁷ The drill does not leave the same exact hole each time; rather, it may leave a hole thousandths of an inch larger or smaller than another hole. The drill cuts away chips of metal and forces them through the barrel.⁵⁸ Ideally, Borgio testified, those chips are flushed out of the barrel completely, although some chips can remain in the barrel.

56 The two firearms at issue in the Warner and Meeks cases are a pistol and a revolver, respectively. Where there is no difference between the two with respect to their manufacture or firing processes, this court will use the term "firearm;" otherwise, this court will use the appropriate term.

57 In a pistol, the back part of this hole where the ammunition rests prior to firing is also the chamber. See Borgio Hearing Testimony, p. 62, ll. 2-6. In a revolver, there is more than one chamber in which, generally, six pieces of ammunition sit prior to being advanced into the barrel; each of those chambers is manufactured separately, as discussed further below.

58 Borgio defined the "flutes" as the raised parts of the drill that do the cutting; "[t]he unraised part of the drill is the chip clearance so the chips could get out." Borgio Hearing Testimony, p. 35, ll. 2-4.

Second, as the barrel is locked in a receptacle in a vertical position and constantly squirted with coolant,⁵⁹ the reamer rotates in and out of the barrel, rotating the entire time at a certain RPM.⁶⁰ This tool provides a straight, precision hole with the correct diameter, or caliber, and it removes any marks that the drill left.⁶¹ The reamer may have to remove more or less metal from each barrel depending on the drilling process that preceded it.⁶² As the drill leaves each barrel in a slightly different condition, so too does the reamer. Additionally, the operator does not place the reamer into each barrel in exactly the same position, and there is no way to determine which part of the reamer begins to insert itself into the barrel. See Borgio Hearing Testimony, p. 74, ll. 18-24-p. 75, ll. 1-9. The marks that the reamer leaves along the interior of the barrel are perpendicular to the barrel; these marks remain on what will become the lands of the barrel.⁶³ Although the barrel does not rotate during this fifteen-to-twenty second process, Borgio testified that there is a slight vibration while the barrel is in the fixture; this movement causes random markings to form on the barrel's interior.

59 This coolant is either an oil-based or water-based solution that is designed to cool the tool and flush out the "chips" that the reamer takes out.

60 "The reamer goes down until it clears the bottom of the barrel, and the machine reverses itself. . . . [T]hat's not a precision process. . . . [T]he reamer may go down an eighth of an inch further on one than the other, simply because the machine reversal switch acted quicker on barrel A than it did on barrel B." Borgio Hearing Testimony, p. 84, ll. 14-21.

61 The chips of metal that the reamer removes "end up being removed in the flutes of the reamer, or flushed through the barrel because when the reamer is going through the barrel and it's being done with a flood of coolant, the barrel has an open end at the bottom. So some chips can get flushed through a head [sic] of the reamer, some can get trapped in the flutes of the reamer, and some can exit the top of the barrel when the reamer is extracted." Borgio Hearing Testimony, p. 53, ll. 3-13. Additionally, for example, "the chips that are left on the flute can cause a mark of their own, in addition to what the cutting surfaces of the reamer are doing." Id. at p. 54, ll. 6-9. The presence of these chips can "reduce the effectiveness of the reamer." Id. at p. 71, ll. 23-24.

62 The marks the reamer leaves on the interior of the barrel as it rotates out "tend to be minor compared to the marks that were made when the reamer was actually removing material on its entrance into the barrel." Borgio Hearing Testimony, p. 40, ll. 18-21.

63 In fact, the interior of the barrel is a single land area interrupted only by the markings that the reamer has left. Borgio pointed out that the differences in the spacing of these marks the reamer leaves is likely only 0.005 to 0.010 of an inch; one would therefore have to measure the marks to discern the difference.

Borgio noted that a pistol undergoes a secondary reaming process because, in a pistol as contrasted with a revolver, the chamber is part of the barrel. The pistol's chamber is reamed by a separate reamer called a "taper reamer" because the top end of the chamber is slightly larger in diameter than the bottom end.Id. at p. 64, l. 1-5. The chamber taper reamer has a piece of metal, called a "pilot," on its tip that is the diameter of the barrel and guides the chamber reamer three-quarters of an inch into the chamber.⁶⁴ Id. at p. 82, ll. 14-22. The taper reamer leaves "direct" markings on the chamber, whereas the drill and barrel reamer leave random, "indirect" markings on the chamber.⁶⁵ Revolvers have chambers, or "cylinders," that are separate from the barrel.⁶⁶ Each of those chambers is drilled and reamed, but, as with the pistol's chamber, not broached. See id. at p. 93, ll. 22-24-p. 94, ll. 1-7.

64 If the taper reamer went beyond the chamber further into the barrel, it would affect the caliber because its diameter is larger than the barrel.

65 "[T]he final chamber reamer removes the marks, the direct marks left by the" drill and the barrel reamer. Borgio Hearing Testimony, p. 83, ll. 17-19. Those marks become indirect after the chamber is reamed. Borgio explained that the barrel reamer leaves "indirect" marks on the chamber "[b]ecause if the [barrel] reamer left a little more material, then the chamber [taper] reamer has got to take out more material than it did on the previous barrel. . . . [Thus,] indirectly it affects the final chamber reamer by the amount of material that's left." Id. at p. 83, ll. 20-24-p. 84, ll. 1-2.

66 The revolver is manufactured in such a way that there is a minimal gap between the front of the cylinder and the back edge of the barrel. "[I]n the manufacturing process, [assemblers] insert the barrel into the [revolver's outer] frame and close the cylinder. And the cylinder tends to bump into the barrel. And . . . the assemblers . . . take a file and they file this surface very carefully until they get about a four to ten thousandths of an inch gap." Borgio Hearing Testimony, p. 13, ll. 5-11. This file is a hand tool.

Third, with a machine clamping the barrel under oil in a horizontal position,⁶⁷ the operator puts the broach into chamber end of the barrel. The broach is a round file approximately twenty-eight to thirty inches long that is cut away in certain areas. See Exhibit 32 (photograph of broach taken by Borgio). The broaching takes approximately ten seconds. Even though it is round, the broach does not rotate; rather, several hundred pounds of force pull it through the barrel from one end to the other without reversing.⁶⁸ The broach creates the rifling characteristics, such as five lands, five grooves, and a right twist, by creating the grooves, or depressions in the barrel. Each groove ultimately measures approximately 0.003 of an inch in width, and each tooth of the broach cuts a fraction of that amount. The lands are what remain once the grooves are cut away, with the markings on the lands having already been left by the reamer; the broach does not touch the reamed surfaces.⁶⁹

67 As the broaching is done in oil, the larger pieces of debris from the broaching process are caught in a screen designed for that purpose, while the smaller, microscopic chips stay in the oil.

68 Borgio noted that in a pistol, the chamber end of the barrel is slightly wider than the rest of the barrel because that is where the ammunition sits prior to firing. As the broach passes through the barrel, it does not touch the chamber end because the broach is narrower than that end. A pistol's chamber is therefore not broached.

69 Just as with the reaming process, vibration occurs while the barrel is clamped and the broach is pulled through. This vibration creates "chatter . . . or cutter marks." Borgio Hearing Testimony, p. 72, ll. 23-24.

Some barrels⁷⁰ go through a fourth machining process called "honing." This process takes place after the reaming and before the broaching; Borgio also referred to this process as a "second reaming." Borgio Hearing Testimony, p. 69. l. 6. With honing, a circular stone is inserted into the barrel to provide a smooth, "cosmetic finish" to the interior of the barrel, changing the surface finish of the barrel so when looking down the barrel, one sees a satin finish. Id. at p. 66, ll. 22-23. Honing is a rotating process, like reaming, therefore it leaves "annular" marks in the barrel. Id. at p. 67, l. 23. These marks change the marks made by the reamer in an individual, random fashion. Borgio analogized the honing of a barrel to the sanding of a piece of wood and the reaming process to the planing of wood.⁷¹

70 Borgio could not tell if the Smith Wesson revolver allegedly used in the Meeks case had been honed.

71 Upon inspecting the Smith Wesson revolver allegedly used in the Sanders homicide, Borgio could not tell if the barrel had been honed, and he did not know if Smith Wesson used the honing process in 1980, when he believed that revolver had been manufactured.

Borgio also testified about the tools used on pistols⁷² that result in random markings on shell casings. First, the firing pin in a pistol, located in the center of the pistol's slide,⁷³ starts out as a long bar of steel. A machine feeds out this bar of steel as a tool, a "cutter," shapes the firing pin's tip, its barrel, its safety groove, and finally the surface for its hammer. See id. at p. 98, ll. 1-7. The machine cuts the resulting firing pin off from the bar of steel when it is approximately two and a half inches long.⁷⁴ See Exhibit 36 (photograph of "Firing Pin Manufacturing" taken by Borgio).

72 Borgio testified that "the cartridge cases usually remain in the revolver." Borgio Hearing Testimony, p. 94, ll. 11-12. Therefore, as the cartridge case is not necessarily ejected from a revolver's chamber, or cylinder, the markings that firing impresses upon it are not relevant to this discussion.

73 The slide is a separate piece of the pistol located above the barrel; the slide is moveable. Borgio did not define the term "slide." An on-line encyclopedia provides the following: "The slide is the part of a semi-automatic pistol that moves during the operating cycle and generally houses the firing pin or striker, extractor, and serves as the bolt face. It is spring loaded so that once it has moved to its rearmost point in the firing cycle, spring tension brings it back to chamber a fresh cartridge." http://en.wikipedia.org/wiki/Pistol_slide (last visited September 25, 2006).

74 In a revolver, as contrasted with a pistol, the tip has the same rounded configuration but the rear part of the revolver's firing pin is a "hammer nose." Borgio Hearing Testimony, p. 99, ll. 4-5.

As contrasted with a revolver, a pistol's slide has an extractor and an ejector. The extractor part of the slide has a groove that, upon firing, comes in contact with a groove on the cartridge case in order to expel the cartridge case through the ejection port. To create that groove, a rotating cutter machines the underside of the extractor's lip while the extractor is held vertically in a machine. See Exhibit 38 (photograph of "Extractor Hook Operation" taken by Borgio). The rotating cutter spins at approximately 100 RPMs and is moved slowly across the extractor, then a lubricant flushes the cutter and extractor of the debris. This metal removal operation and the actual machining of the extractor leave random marks on the extractor.

Finally, a broaching tool is used to make a smooth, flat surface on the breech face against which the cartridge case will rest.⁷⁵ See Exhibit 33 (photograph of "Slide Breech Face Operation" taken by Borgio). Without a smooth surface on the breech face, the slide "could not accept a cartridge." Borgio Hearing Testimony, p. 92, ll. 22-23. This broach enters the ejection port vertically and has sharp teeth⁷⁶ to remove unwanted material.⁷⁷ Like the barrel broach, the breech face broach "does it[s] . . . job on one pass" without an extraction. Id. at p. 89, l. 16. Unlike the barrel broach, which is round in shape and submerged under oil, this half-inch wide tool is flat, and coolant oil accompanies it as it machines the breech face. The marks that this breech face broaching leaves on the breech face are random.

75 "[O]n the slide the broach is the final tool on the breach [sic] face." Borgio Hearing Testimony, p. 101, ll. 13-14. Borgio brought two slides with him to court; the first, Exhibit 34, is a slide before the breech broaching process; the second, Exhibit 35, is a slide after the breech broaching process.

76 "[T]he teeth down below are shallower, because each tooth is just a slight amount larger than the predecessor tooth. So each tooth removes just a little bit more material." Borgio Hearing Testimony, p. 90, ll. 1-5.

77 Borgio likened this tool to "a very coarse file." Borgio Hearing Testimony, p. 87, l. 20.

Borgio cited a number of variables that result from the manufacture of a firearm that could affect the marks left on projectiles and shell casings, including the following: the degree of cleanliness of the coolant; the manufacture of the cutting tools (i.e., the reamer and the broach); the degree of sharpness of the cutting tools;⁷⁸ the presence of microscopic chips in the oil solution; the composition of the steel; and the heat treating done to the metal.

78 The broach and reamer must be resharpened every fifty to one hundred times they are used; their cutting edges are dulled each time they are used, therefore it is impossible to cut each time in exactly the same manner.

B. The Firing Process

Borgio also testified about the firing processes of pistols and revolvers, and their effects on cartridge cases and bullets. During the firing process of a pistol, the ammunition loads into the chamber area from the magazine, the slide moves forward, the breech face on the slide comes up against the rear of the cartridge case in the chamber, and the sides of the cartridge case come into contact with the chamber. The 30,000 to 35,000 pounds per square inch of pressure at the moment of firing forces the back of the cartridge case against the breech face and causes it to expand against the chamber. Any marks on the pistol's breech face and chamber are imparted onto the cartridge case through this process. The firing pin is located in the center of the pistol's slide, therefore part of it makes contact with the cartridge case.⁷⁹ When a firearm is fired, the hammer hits the back of the firing pin and sends it forward; the tip of the firing pin leaves an impression on the cartridge case. "The residual force on the cartridge case forces the slide rearward, which ejects the fired case." Id. at p. 61, l. 3-5. The cartridge case also comes into contact with the extractor and ejector in a pistol, as contrasted with a revolver, and the fired shell casing is ejected through the top ejection area. These marks on spent shell casings are "impression" marks, thereby "denot[ing] perpendicular movement of the `tool' relative to the surface marked." Alfred Biasotti John Murdock, "Scientific Issues," in 4 David L. Faigman et al., Modern Scientific Evidence — The Law and Science of Expert Testimony § 36:7, Figure 1, at 405 (2005-2006 ed.) ("Biasotti Murdock, Exhibit 83A"); see id. at 406 n. 4 (stating that impressed toolmarks "are produced when a tool [such as a firearm's breech face] is placed against another object [here, the shell casing] and enough force is applied to the tool so that it leaves an impression").

79 The firing pin in the revolver ignites the primer in the ammunition.

Simultaneously, in both a pistol and a revolver, the pressure resulting from the firing forces the bullet, or projectile, down the barrel and out through its muzzle, or front, end at a speed of twelve to thirteen hundred feet per second. Passing through the barrel leaves striations on the projectile; the raised lands on the barrel leave striations on the now depressed "land-engraved" areas of the projectile, and the depressed grooves of the barrel leave striations on the now raised "groove-engraved" areas of the projectile. These marks on spent projectiles are "striated" marks, thereby "denot[ing] lateral movement of a `tool' relative to the surface marked or lateral movement of the surface marked relative to the tool." Id. at 405; see id. at 406 n. 4 (noting that striated toolmarks "are produced when a tool [here, the barrel of a firearm] is placed against another object [here, the projectile] and with pressure applied, the tool is moved across the object producing a striated mark").

C. The Specifics of Firearms Examinations

Firearms examiners render conclusions with respect to ballistics evidence based on their belief in the "physical phenomena . . . [of] the imparting or transfer of a presumably unique combination of patterns or contours from one solid surface to another." Biasotti Murdock, Exhibit 83A, § 36:7, at 404. "Toolmarks produced by firearm components and other tools typically are a combination of both impression and striated marks." Id. at 406. The Association of Firearms and Toolmark Examiners ("AFTE"), a voluntary organization for firearm and toolmark examiners that holds conferences each year and issues a seasonal peer reviewed publication, the AFTE Journal, defines impressed toolmarks as "[M]arks produced when a tool [i.e., a barrel] is placed against another object [i.e., a bullet or shell casing] and enough force is applied to the tool so that it leaves an impression." "Identification as it Relates to Toolmarks," 30 AFTE Journal 86, 88 (Winter 1998) ("AFTE Theory, Exhibit 28"). Striated toolmarks, in turn, are "[m]arks produced when a tool is placed against another object and with pressure applied, the tool is moved across the object producing a striated mark. FRICTION MARKS, ABRASION MARKS and SCRATCH MARKS are terms commonly used when referring to striated marks." Id. (capitalization in original).

Firearms examiners examine the impression marks and the striated marks on spent projectiles and shell casings. The types of marks are classified in three ways: 1) class characteristics; 2) sub-class characteristics; and 3) individual characteristics. Class characteristics are those characteristics that the firearm manufacture intends to result from the manufacturing process, namely the number and width of lands and grooves, and the direction in which those marks twist, either left or right. See Biasotti Murdock, Exhibit 83A, § 36:7, at 404 n. 1. Class characteristics "are often visible to the naked eye. . . . [and] will be the same for any bullet fired from any firearm of the same make and model. . . ." Lisa J. Steele, "All we want you to do is confirm what we already know," 38 Criminal Law Bulletin 466, **4 (2002). Certain class characteristics, such as the extractor or ejector marks impressed upon a spent shell casing, "can be used to identify the make of the firearm from which the case was fired . . . but generally are not to be used to identify the individual firearm from which the cartridge was fired. . . ." Lamagna Affidavit, ¶ 11; see Biasotti Murdock, Exhibit 83A, § 36:7, at 406 n. 4 ("The class characteristics . . . can indicate the type of tool [i.e., firearm] used to produce the mark.").

Firearms examiners, conversely, can base an identification on individual characteristics which "are considered unique to that tool and therefore are believed to distinguish it from all other tools." Biasotti Murdock, Exhibit 83A, § 36:7, at 404 n. 1. "The object is to determine whether the individual characteristics [e.g., of two bullets] are so similar that one and the same tool (. . . a particular gun barrel) must have produced both" of them. Adina Schwartz, "A Systemic Challenge to the Reliability and Admissibility of Firearms and Toolmark Identification," 6 Columb. Sci. Tech. L.Rev. 1, 5 (2005). Individual characteristics are "[m]arks produced by the random imperfections or irregularities of tool surfaces. These random imperfections or irregularities are produced incidental to manufacture and/or caused by use, corrosion, or damage. They are unique to that tool and distinguish it from all other tools." AFTE Theory, Exhibit 28, at 87; see Biasotti Murdock, Exhibit 83A, § 37:7, at 404 n. 1. "For instance, if a particular firearm had a broken firing pin nose, or distinctive machine toolmarks on the firing pin nose or breech face, it would leave a certain mark on the primer and/or cartridge case head that perhaps no other firearm would leave." Lamagna Affidavit, ¶ 12; see id. at ¶ 28 ("Finer toolmarks normally form the Individual characteristics.").

Sub-class characteristics "are toolmarks that, because of their well defined, continuous over virtually all of the tool working surface, often prominent, and sometimes equally spaced appearance without changing significantly over some distance, can be suspected of being found on other similarly manufactured tool working surfaces." Biasotti Murdock, Exhibit 83A, § 36:8, at 409; Id. § 36:7, at 404 n. 1; see Schwartz, supra ("Subclass characteristics, which are present in only some toolmarks, arise when manufacturing processes create batches of tools with similarities in appearance, size, or surface finish distinguishing them from other tools of the same type.").⁸⁰ These marks are "[d]iscernable [sic] surface features of an object which are more restrictive tha[n] CLASS CHARACTERISTICS in that they . . .: (1) [are] [p]roduced incidental to manufacture; (2) [a]re significant in that they relate to a smaller group source (a subset of the class to which they belong); and (3) [c]an arise from a source that changes over time." AFTE Theory, Exhibit 28, at 88 (capitalization in original); see Biasotti Murdock, Exhibit 83A, § 36:7, at 407 n. 7. These marks are therefore not unique to a single firearm, thereby requiring firearms examiners to "consider the possibility of sub class . . . carry over on consecutively manufactured tool working surfaces [e.g., a firearm's barrel] before positively identifying a toolmark as having been made by a particular tool, to the exclusion of all other tools." Id. § 36:8, at 410 (emphasis added).⁸¹ AFTE recommends that "[c]aution should be exercised in distinguishing SUBCLASS CHARACTERISTICS from INDIVIDUAL CHARACTERISTICS." AFTE Theory, Exhibit 28, at 88 (capitalization in original).

80 In "Sub Class Characteristics of Sequentially Rifled 38 Special S W Revolver Barrels," the authors, Frederic A. Tulleners and James S. Hamiel discussed a study of ten sequentially manufactured revolvers. 31 AFTE Journal 117, 117 (Spring 1999) (Exhibit 29). The examinations demonstrated that "subclass characteristics were not found on the land impressions of the fired lead bullets or on the land or groove impressions of the copper-jacketed bullets." Id. (underlining in original); see id. at 118, 121.

81 "When there is . . . a chance that microscopic subclass characteristics, having their origin in the manufacturing process, can be present in the type of toolmarks recovered in a series of crimes, a more cautious approach should be taken. It is not uncommon in these cases for the examiner to write a report that states that sufficient microscopic agreement is present to suggest that the same tool made the series of toolmarks, but that a conclusive opinion can be rendered only after an examination of the responsible tool. Once the examiner has the tool, the working surface can be evaluated [e.g., by taking a cast of the barrel] to determine if the tool produces a unique toolmark, or is one that contains subclass characteristics that are capable of being transferred to toolmarked surfaces." Biasotti Murdock, Exhibit 83A, § 36:8, at 410. Here, although it is a legitimate criticism of Lydon that he initially rendered conclusions of identification in the Meeks and Warner cases without having a firearm, the firearms were eventually recovered and Lydon's conclusions did not change. His failing to withhold his conclusion and merely finding that there was a "suggestion" of identification prior to the recovery of the firearm does not render his conclusions unreliable. Appropriate cross-examination as to this point, however, much like cross to the effect that Lydon failed to document his examinations, may seek to question the weight or value of his testimony.

In examining individual characteristics, "[t]he theory of identification, as it relates to toolmarks, adopted by [AFTE] . . . gives a nonquantitative answer to the question of how much agreement is needed." Biasotti Murdock, Exhibit 83A, § 39:9, at 411 (footnote omitted). "The theory of identification as it pertains to the comparison of toolmarks enables opinions of common origin to be made when the unique surface contours of two toolmarks are in "sufficient agreement." AFTE Theory, Exhibit 28, at 86. "`[S]ufficient agreement' is related to the significant duplication of random toolmarks as evidence by the correspondence of a pattern or combination of patterns of surface contours."Id. (emphasis added). A firearms examiner determines significance by conducting a

"comparative examination of two or more sets of surface contour patterns comprised of individual peaks. Specifically, the relative height or depth, width, curvature and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours."

Id. The AFTE Theory of Identification describes agreement as significant "when it exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools ["best known non-matches"] and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool ["known matches"]." Id.

At the same time, the AFTE Theory of Identification encourages firearms examiners to "report the objective observations that support the findings of [firearms] examinations. The examiner should be conservative when reporting the significance of these observations." Id. (emphasis added). The range of objective conclusions the AFTE Theory of Identification sets forth are as follows:

"1. IDENTIFICATION — Agreement of a combination of individual characteristics and all discernable [sic] class characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool.

"2. INCONCLUSIVE —

A. Some agreement of individual characteristics and all discernable [sic] class characteristics, but insufficient for an identification.

B. Agreement of all discernable [sic] class characteristics without agreement or disagreement of individual characteristics due to an absence, insufficiency, or lack of reproducibility.

C. Agreement of all discernable [sic] class characteristics and disagreement of individual characteristics, but insufficient for an elimination.

"3. ELIMINATION — Significant disagreement of discernable [sic] class characteristics and/or individual characteristics.

"4. UNSUITABLE — Unsuitable for microscopic examination."⁸²

82 The AFTE Theory of Identification defines "Unsuitable" as "Unsuitable for microscopic examination." AFTE Theory, Exhibit 28, at 87. In Biasotti Murdock, Exhibit 83A, which purports to quote the AFTE Theory of Identification, the authors define "Unsuitable" as "Unsuitable for comparison."

Id. at 86-87 (capitalization in original); see Biasotti Murdock, Exhibit 83A, § 36:9, at 412.

"There is no question . . . that conclusions of identity in firearms . . . are possible." Biasotti Murdock, Exhibit 83A, § 36:10, at 413. "[T]he interpretation of the . . . identification is subjective in nature, founded on scientific principles and based on the examiner's training and experience." AFTE Theory, Exhibit 28, at 86 (emphases added); see Biasotti Murdock, Exhibit 83A, § 36:9, at 413 ("[T]he interpretation that forms the basis for these conclusions is subjective."). The "scientific principles" have been "continually tested empirically and [have] stood the test of time, resulting in the general principle (Theory) adopted by [AFTE] in 1992." Biasotti Murdock, Exhibit 83A, at 411. "The studies leading up to this theory have been peer reviewed, published, and thus have been available for replication by the relevant scientific community of forensic scientists." Id.

"Consecutive matching striae" ("CMS") is another method of firearms examination; it solely concerns striated toolmarks as opposed to impression toolmarks. See Biasotti Murdock, Exhibit 83A, § 36:7, at 406 n. 4. "The concept of `consecutiveness' is a simplified way of expressing the matching of a segment of contour, or a pattern of matching individual characteristics in a striated toolmark." Id. § 36:12, at 418. This method looks for striae, or lines, on spent bullets "that correspond or match with respect to each striae's width, depth and contour and are of sufficient length to assure that striae are parallel to one another. . . . [In applying this method,] only the ridges are counted and not the valleys between the ridges." Id. ⁸³ "CMS requires examiners to compare the striae comprising the individual characteristics of toolmarks." Schwartz, supra, at 16. "[T]he threshold for identifying a particular tool as the source of a three-dimensional toolmark [such as that a barrel leaves on a projectile] is a match between evidence and test toolomarks of one group of six consecutive matching striae or two different groups of at least three consecutive matching striae in the same relative position." Id.

83 CMS is distinguished from the percent of matching lines that Biasotti employed in his 1957 study, discussed further below. " Percent matching lines denotes the percent of matching striae without regard to consecutiveness." Biasotti Murdock, Exhibit 83A, § 36:12, at 418 n. 7.

"These results, therefore, support the validity of the hypothesis adopted by [AFTE]. . . ." Biasotti Murdock, Exhibit 83A, § 36:12, at 418. As discussed further below, however, CMS is not the sole method that firearms examiners utilize. Rather, firearms examiners generally use CMS as further support for the conclusions they reached using the AFTE Theory of Identification.

IV. Evidence in Support of the Reliability of the Field of Firearms Examination

This court heard testimony from four firearms examiners: Lydon and Doherty, BPD firearms examiners who conducted the examinations in the Meeks and Warner cases; and Mary-Jacque Mann and Peter Striupaitis, former and current firearm examiners, respectively, who testified on behalf of the Commonwealth. These four firearms examiners are proponents of the methods currently utilized by firearms examiners. From their testimony, combined with the above principles of the field of firearms examination in general, this court learned that the field of firearms examination is self-contained, with firearms examiners engaging in peer review on an immediate lab-specific level. Although the process of documenting examinations varies from lab to lab, the firearms examiners utilize the same method in conducting their examinations, a method that they developed through training and experience.

Moreover, "[f]irearms and toolmark examiners apply science and scientific methods, procedures and instruments in a practical way, but most are more skilled in the art of applying those methods and procedures than they are in the basic sciences involved." Biasotti Murdock, Exhibit 83A, § 36:9, at 412. "The examiner qualified to render . . . conclusions should be familiar with . . . [various studies and] the quantity and quality of matching agreement found in comparisons of toolmarks known to have been produced by different tools. (Known non-matches.)"Id. § 36:10, at 413-414. The firearms examiners who testified at this hearing all possess these qualities.⁸⁴

84 In "The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion," by D. Michael Risinger, Michael J. Saks, William C. Thompson, and Robert Rosenthal, the authors set out a hypothetical "new system of forensic science laboratories with the goal of producing results of maximal accuracy." 90 Calif. L.Rev. 1, 42 (Jan. 2002) ("Risinger Saks, Exhibit 44"). They suggested a series of options in establishing such a system:

"Either laboratories would be set up in the new system as arms of criminal law enforcement, or laboratories would be freestanding entities available to both prosecution and defense.

Either examiners in the new system would be drawn largely from the ranks of current enforcement officers, or examiners in the new system would be recruited from the ranks of people interested in science with no pre-existing law enforcement bias.

Either examiners in the new system would be socialized in such a way, and would interact with case detectives in such a way, as to feel themselves an integral part of a law enforcement `team,' or examiners would be insulated from such influences and trained to form no such role view, but instead to view their role solely in terms of the maximal integrity and maximal accuracy of their own results.

Either examiners in the new system would be exposed to much domain-extraneous information in the process of conducting an examination, including the emotionally gripping details of the underlying case and the hopes and expectation of the case detectives, or specific procedures would be put in place to separate relevant information from extraneous information, and to insulate the examiners from exposure to the latter."

Id. at 42-43. While the second option in each of the above-quoted paragraphs would minimize or eliminate confirmation bias, there are of course budgetary and other considerations. As to the third point, this court finds that the officers and examiners in these cases take seriously their duty to report the truth. As to the fourth point, "domain extraneous information" was minimal in these cases, and such a practice is clearly desirable. As set forth below, the firearms examiners who testified in support of the field of firearms examinations demonstrate that while the methods they employ are imperfect, a number of factors, including the consistency of their conclusions and in-lab peer review, render their examinations reliable.

A. Officer Martin Lydon

Lydon is a nineteen-year veteran of the BPD. In 1999, he was assigned to the BPD's Unit where he still works as a firearms examiner.⁸⁵ Upon this assignment, Lydon underwent a two-year training period during which time he did not have sole responsibility for the cases on which he worked. Rather, he worked with veteran examiners ("trainers") who trained him on the comparison microscope, walked him through his case-work, and reviewed Lydon's conclusion once he had completed an evaluation of a piece of evidence.

85 Lydon is one of the Unit's two firearms examiners; the other firearms examiner is Detective Tyrone Camper ("Camper").

Over the course of this training period, Lydon attended six armorer's schools, test fired over 200 weapons, conducted several hundred microscopic comparisons of spent shell casings, and dozens of microscopic comparisons of spent bullets. Lydon Affidavit, ¶ 3.⁸⁶ Although Lydon's training period concluded after two years, he considers his training on-going as he continues to examine known matches and known non-matches whenever he has free time, and he continually develops his knowledge of the degree of similarity between known matches and known non-matches. To that end, since completing his training, Lydon has worked on more than 1,000 cases, test fired more than 800 weapons, and conducted over 1,500 microscopic comparisons of spent shell casings and over 200 comparisons of spent bullets. He has attended various training conferences both on- and off-site, and has toured the factories of various firearms manufacturers in order to watch first-hand the process of manufacturing firearms. Lydon has participated in proficiency testing in 2003 and 2005; both tests were associated with the American Society of Crime Laboratory Directors ("ASCLD").⁸⁷ Lydon did not receive any "misidentifications"⁸⁸ in the tests. Lydon also uses the Unit's "library" that contains periodicals and books related to firearms and ammunition.

86 Lydon submitted two separate affidavits, one for Warner's case and one for Meeks' case. These affidavits are identical through and including paragraph twelve; starting at paragraph thirteen, Lydon describes his role in the defendants' cases, focusing on Warner's case in his Warner affidavit, and focusing on Meeks' case in his Meeks affidavit.

87 ASCLD is an independent organization that offers certification to examiners and labs; the BPD is in the process of trying to be certified by ASCLD, discussed further below.

88 According to Lydon's testimony, a "misidentification" refers to the incorrect conclusion that two pieces of evidence match when in fact they do not; this result is sometimes referred to as a false positive. (Presumably a "misidentification" would also include an incorrect conclusion that two pieces of evidence do not match when in fact they do; this result would be a false negative.) A "misidentification" does not include the situation where the examiner's conclusion is "inconclusive" but there is actually a match. "Inconclusive," according to Lydon, means that the class characteristics of two pieces of evidence are the same and some markings may be in agreement, but there are not enough markings to conclude definitely that the two pieces of evidence match. Even if those two pieces of evidence do in fact match, the examiner who found that the markings were inconclusive did not make a "misidentification," according to Lydon. As noted above, the proficiency tests score all findings of "inconclusive" as incorrect.

Based on his experience and training as a firearms examiner, Lydon accepts the premise that firearms leave random but identifiable markings on bullets and shell casings, and he accepts the AFTE Theory of Identification. Lydon spent about half of each week during his training period examining old evidence under the comparison microscope in order to distinguish among the characteristics he viewed. He studied "known non-matches" by taking two firearms of the same caliber and model, firing them, and examining both bullets, known not to be from the same firearm, under the comparison microscope. This method of training taught Lydon the nature and degree of similarity found between known non-matches. The "best non-match" is the known non-match the examiner has analyzed which had the most similar markings the examiner has personally observed;⁸⁹ this "best known non-match" method is individual to each examiner who develops an "eye" for seeing striations over the course of his/her training and experience. When examining bullets or shell casings, Lydon looks for more agreement between the markings than the best non-match he has ever seen, thereby basing his conclusions entirely on his own experience; there is no database or other source displaying the "best non-match."

89 Lydon's understanding of the concept of best known non-matches is consistent with the AFTE Theory of Identification. See AFTE Theory, Exhibit 28, at 86.

Lydon also studied "known matches" in a similar manner, firing bullets from the same firearm and examining them under the microscope. This technique taught Lydon the difference in, and significance of, the similarities between two non-matches and between two matches. He was trained to look at the random markings for "patterns" in the numbers of striations in certain locations on the bullet and to look at the marks left on shell casings. Although there is a "reticle" on the comparison microscopes, Lydon does not measure the striations because the markings on the reticle are not fine enough to be of any use. Lydon distinguished his "patterns" method from the CMS method that other examiners use. Lydon does not use CMS.

Lydon employs a "conservative approach" in examining ballistics evidence, looking only at the evidence in front of him and not making any pre-determined decisions. Using this approach, Lydon first attempts to eliminate the evidence, such as a spent bullet from a crime scene and a test fire, as non-matches by using class characteristics and individual characteristics.⁹⁰ For example, if the land- and groove-engraved areas of two spent bullets do not match up, then the examination results in an exclusion. If the land- and groove-engraved areas do match in terms of spacing and location, thereby indicating that the two spent bullets were fired from the same type of gun, then Lydon examines the individual characteristics. He first identifies a land-engraved area that he would characterize as an area of significance, or area of interest, on one bullet and marks it with a marker; then he "spins" the other bullet to search for a matching land-engraved area. If they do not correspond, then the examination results in an exclusion.

90 In Lydon's opinion, bullets are more difficult to match than shell casings because, with bullets, he is examining small areas of striations whereas with shell casings he has more areas to use for comparisons.

If Lydon cannot eliminate the evidence, he then attempts to find "sufficient" markings to identify the two pieces of evidence as having come from the same firearm. Based on his experience with known non-matches, Lydon can identify a match by lining up a sufficiently random pattern on one projectile to that on another. The fact that he may find differences would not prevent him from declaring a match because "there will always be differences."⁹¹ If Lydon is not completely certain that the two pieces of evidence match, he concludes that the markings are too "inconclusive" to result in a match. Lydon will not change his "inconclusive" determination even where another examiner has looked at the same evidence and found a match unless Lydon is able to see the points of agreement on which the other examiner based his/her finding of a match.

91 In response to this court's question of whether there will always be similarities, too, Lydon testified that given the amount of markings that appear on any given spent projectile, there is a high probability that an examiner would find similarities, but he has not studied this issue.

To Lydon's knowledge, no other examiner who has re-examined Lydon's work since he completed his training has disagreed with his conclusions. He has changed his own mind when he has been "on the cusp" with respect to an examination after another BPD examiner has reviewed the evidence and persuaded Lydon that there is a match.⁹² He recalled four occasions when his match opinions were re-examined by non-BPD examiners, and on each occasion, his conclusions were upheld.

92 In response to a question from this court, Lydon testified that he has never been uncertain then concluded there was no match; rather, he has concluded "inconclusive."

With respect to sub-class characteristics, Lydon testified that a firearms examiner has to be aware of their existence and that they appear more frequently on groove-engraved areas, but their presence does not preclude an identification. See Frederic A. Tulleners James S. Hamiel, "Sub Class Characteristics of Sequentially Rifled 38 Special S W Revolver Barrels," 31 AFTE Journal 117 (Spring 1999) ("Tulleners Sub-Class, Exhibit 29") (discussing study of ten sequentially manufactured revolvers that demonstrated that subclass characteristics were found only on groove-engraved areas of test fired bullets and not on land-engraved areas). Sub-class characteristics are gross markings that do not "group" in such a way that they form patterns parallel to the projectile in the same way that individual characteristics do. He could not describe the difference any other way, and stressed that his experience as a firearms examiner assists him in distinguishing sub-class characteristics from individual characteristics. Lydon began reading about sub-class characteristics after he testified inUnited States v. Green, Crim. No. 02-10301-NG (D. Mass), in November 2005. He believes that the examinations he conducted before that date are not in question because his research of sub-class characteristics did not cause him to make any changes in his examination methods.

In 2004, Doherty became commander of the BPD Firearms Examination Unit. She has implemented various procedures in the examination of ballistics evidence, and the Unit is in the process of creating a written protocol setting out examination procedures. Among the changes that Doherty implemented is the requirement that the examiners generate examination reports that are more detailed than previous reports. For example, rather than simply setting out his conclusion that the markings on two pieces of evidence match, do not match, or are inconclusive, Lydon now creates a separate report in which he describes the firearm, if any, the bullets and/or shell casings, and the areas of agreement.⁹³ Doherty also requires that the examiner contact her once a match is found so that she can conduct her own examination of the evidence under the comparison microscope in order to confirm or negate the examiner's conclusion. Also at this time, either Doherty or O'Shea takes photographs of the areas of agreement between the pieces of evidence. Before a case goes to trial, examiners are required to conduct a re-examination of their own work. At that time, Doherty conducts her own supervisory examination as well and takes pictures.

93 As part of his testimony, Lydon suggested that this separate, more detailed report, would contain such information as: "found runs of seven to ten land[-engraved areas], striations found on two land[-engraved areas], . . . ."

B. Sergeant Detective Catherine Doherty

Doherty is the commander of the BPD's Unit. She began at the Unit in February 1995 as an officer assigned to implement IBIS into the Unit. She left the Unit in 1997 and returned in late 2004 when she became the commander. Her formal education includes a bachelor's degree in criminal justice from Curry College and she has taken classes at the Boston University Graduate School of Medical Science.

With respect to her implementing IBIS in the BPD, Doherty's primary responsibilities included photographing, examining, and inputting into IBIS both backlogged evidence from the beginning of 1994 and new evidence that came into the Unit. Doherty also examined evidence after it had been inputted into IBIS. When she found a potential match on the screen, she would notify the firearms examiner and follow through with the physical examination under the comparison microscope. During this time period, Doherty saw test fires from the same types of weapons, thus contributing to her understanding of known non-matches. She also examined "unknown" evidence, meaning evidence from crime scenes without knowing anything about the firearm from which it was fired.

Doherty also attended IBIS training outside of the BPD. She attended conferences in California, Maryland, New York, and Washington D.C., including a conference at which Joseph Masson ("Masson"), a well-known firearms examiner, spoke. Doherty is familiar with Masson's belief that as IBIS' database increases, there is a greater likelihood of similarities presenting themselves, especially with respect to sub-class characteristics, but she also understands Masson's view as recognizing that IBIS is a useful database to examine a vast amount of evidence in one place and to assist in identifying sub-class characteristics. See Joseph J. Masson, "Confidence Level Variations in Firearms Identifications through Computerized Technology," 29 AFTE Journal 42, 43 (Winter 1997) ("Masson, Exhibit 75").⁹⁴

94 Doherty is familiar with this article by Joseph Masson. Masson, Exhibit 75. She also is familiar with the 2001 study of Frederic Tulleners, titled "Technical Evaluation: Feasibility of a Ballistics Imaging Database for all New Handgun Sales," Exhibit 76, and with the report by Jan De Kinder, titled "AB1717 Report [of] Technical Evaluation: Feasibility of a Ballistics Imaging Database for all New Handgun Sales," Exhibit 77, which was conducted in response to Frederic Tulleners' study.

At the same time she was undergoing IBIS training, Doherty also requested and received training in firearms examination. For this training, Doherty took apart weapons and learned about their manufacture and operation; she test fired the weapons and examined the bullets and shell casings; she responded to crime scenes with other examiners to retrieve evidence; and she attended armorer's schools. Doherty also sat with a firearms examiner on a daily basis to look at evidence with the comparison microscope. There was also a training microscope attached to the comparison microscope that enabled Doherty to see what the examiner was seeing. This training period lasted approximately two years. After she became a detective in 1996, Doherty conducted firearms examinations on her own, including testifying in court to defend her work. Doherty left the Unit in 1997 and was promoted to sergeant. After various positions with the BPD, she returned to the Unit as commander in late 2004 under the mandate that she impose more rigorous procedures with respect to firearms examinations.

Doherty's training in firearms investigation still continues, and she has attended conferences, shot shows, and armorer's courses both on- and off-site. Doherty has also read many of the books and periodicals in the Unit's "library," and she has access to the AFTE Journals from the past twenty years. She randomly pulls articles from the AFTE Journal on a regular basis and passes them around the Unit; the examiners must "sign-off" that they have read the article. Doherty has also conducted her own study of consecutive serialed weapons by taking test fires from the Glock firearms that the BPD uses⁹⁵ and compiling a database of two test fires for each gun, specifically test fire #1 and test fire #850. As to each firearm, Doherty has always been able to match the markings on the casing of test fire #1 to the markings on the casing of test fire #850. Additionally, by studying the cartridge cases of the test fires, Doherty added to her understanding of the similarities and differences between known non-matches.⁹⁶

95 Doherty testified that all BPD officers are required to go to the firing range two times per year and fire fifty rounds each time.

96 The court recognizes that Doherty's study of the BPD Glock firearms was in no sense "blind;" in each instance, Doherty knew that the evidence she was comparing was in fact fired or not fired by the same firearm.

Since becoming commander of the Unit, Doherty has implemented a number of changes with respect to firearms investigations. Training, both on-site and off-site, was among her priorities, as was requiring further documentation of past⁹⁷ cases such as supplemental reports, photographs,⁹⁸ and sketches. She has set up a training program for firearms examiners with a recommended length of two years. Doherty testified that she is working toward a higher level of professionalism with respect to documenting case investigations. For each new investigation, the firearms examiners are required to take notes and draw sketches contemporaneous with the initial examination. Photographs⁹⁹ are also required, but only in priority cases, such as homicides, or in a case of "geographic separation." Geographic separation occurs when the bullets and/or shell casings are found in one location and the firearm is found in another location, or when IBIS identifies a connection between two separate shootings. Doherty also requires her examiners to engage in "pre-trial review"¹⁰⁰ before a scheduled court date.¹⁰¹

97 The "past cases" are those cases for which examiners conducted investigations prior to Doherty's becoming commander of the Unit.

98 Doherty noted in her testimony that the enlarged photographs used in the hearing before this court are not part of the Unit's normal procedure. The photographs used during this hearing were blown up for purposes of illustration. Normally, the examiner takes at least two photographs per case.

99 Only Doherty and Sergeant James O'Shea are able to take photographs, both for supervisory reasons and for training reasons. During the tour of the Unit, Doherty demonstrated the process of photographing the evidence while under the microscope with one of the BPD's digital cameras. In her hearing testimony, Doherty testified that the Unit has three cameras, each of which cost approximately $1200 to $1400. The camera Doherty used during the tour has 7.1 mega-pixel, and the microscope company designed a special lens that attaches to the lens of the camera; this camera does not have a flash. In order to photograph the evidence, Doherty removes one of the eyepieces of the microscope and fits the lens of the camera into that area. She also uses the camera's own magnification to further enlarge the representation of the markings. Comparison microscopes with cameras installed within them do exist, but the BPD does not have one yet. Those types of comparison microscopes cost approximately $67,000.
Doherty testified that photographs are merely representations of areas of identification. She decides where to photograph based on the best marked areas, where striae would show most clearly in photographs. Doherty pointed out that photographs do not show the same level of detail one sees under the comparison microscope, partly because the photograph can blur and partly because the screen of the digital camera is so small that the photographer cannot see that the finer details did not show up on the photograph until after the photograph is printed.

100 Doherty always examines evidence after it has undergone a pre-trial review unless the case does not involve an examination of ammunition and the examiner is merely going to court with a firearm.

101 During Doherty's testimony, she was asked about Lydon's and Camper's testimony in the federal case of United States v.Green. Crim. No. 02-10301-NG (D. Mass.). In that case, the court (Gertner, J.) excluded their testimony because they did not have photographs, they did not have any documentation, and they had not looked at the evidence prior to their testimony. See Exhibit J of Memorandum in Support of Defendant's Motion in Limine to Exclude Ballistics Evidence, p. 40 l. 25-p. 41 ll. 1-3 (court noting in a hearing with attorneys during Green trial that Lydon and Camper "took the stand with no notes, no diagrams, no photographs, no . . . nothing . . . other than asking the jury to trust them that when they looked at . . . the samples, they were similar"); Exhibit K of Memorandum in Support of Defendant's Motion in Limine to Exclude Ballistics Evidence, p. 128 l. 20-25 (court stating in her instruction to jury that testimony of Lydon and Camper "has been excluded from the case. [The jury] may look at their testimony with respect to the physical evidence that they gathered, but [they] should ignore any testimony, any conclusions that they may have drawn with regard to the similarities or differences between the evidence that they saw under the microscope"). They gave this excluded testimony in late 2005, a year after Doherty had implemented her "pre-trial review" rule. Doherty testified that she was not aware of this exclusion and did not know why neither Lydon nor Camper had any photographs or documentation. Doherty's testimony on this point is concerning. Improved procedures are effective, obviously, only when they are followed. In the cases before this court, such procedures were, ultimately, followed to a significant extent.

Doherty is seeking ASCLD accreditation for the Unit which requires, in part, that one examiner per lab take a proficiency test each year;¹⁰² that certain safety procedures are followed; that administrative policies are in writing; and that certain levels of documentation are maintained. She is aware that the Collaborative Testing Services, Inc. ("CTS"), is the only ASCLD-approved testing service for firearms examiners.¹⁰³ BPD has used CTS standardized tests for its firearms examiners since 2003. All CTS exams have three possible answers: identifications, exclusions, or inconclusives. If an examiner chooses "inconclusive," however, s/he will always be marked wrong.¹⁰⁴ The directions to the test inform the test-taker of the type of weapons from which the pieces of evidence were fired. According to Doherty, however, this aspect of the test is of no consequence because, although that information would enable the test-taker to "look up" the class characteristics of that type of weapon, the examiner's focus is on the individual characteristics.

102 Doherty testified that, instead, she is going to require that each examiner in the Unit take one test per year.

103 Doherty is aware of the study Joseph L. Peterson ("Peterson") and Penelope N. Markham ("Markham") did of CTS test results from 1978 through 1991, titled "Crime Laboratory Proficiency Testing Results, 1978-1991, II: Resolving Questions of Common Origin," 40 J. Forensic Sci. 1009 (1995). Peterson and Markham concluded that there was a 12% error rate. Doherty pointed out that anyone, not just firearms examiners, could take the tests they studied; that Peterson and Markham only calculated the error rate based on wrong answers; and that wrong answers included "inconclusive" responses. Doherty also testified that CTS does not conduct quality control of the pieces of evidence it sends out as part of its tests, therefore an "inconclusive" could legitimately be an inconclusive.

104 In her testimony, Doherty explained how the CTS tests work, that the "Item 1" an examiner looks at in the BPD Unit is fired from the same weapon as the "Item 1" that an examiner looks at in a California ballistics unit; thus, each Item 1 is fired from the same weapon but it is not the same piece of evidence.

Doherty brought with her two Summary Reports from two CTS tests that the BPD firearms examiners took, both of which contain the following disclaimer on the first page:

"This report contains the data received from the participants in this test. Since these participants are located in many countries around the world, and it is their option how the samples are to be used (e.g., training exercise, known or blind proficiency testing, research and development of new techniques, etc.), the results compiled in the Summary Report are not intended to be an overview of the quality of work performed in the profession and cannot be interpreted as such. The Summary Comments are included for the benefit of participants to assist with maintaining or enhancing the quality of their results. These comments are not intended to reflect the general state of the art within the profession."

2003 CTS Test, Exhibit 79; 2005 CTS Test, Exhibit 80. Doherty agrees with the disclaimer that the results of CTS testing in general should not be used as an indicator of proficiency of the field as a whole. She uses the tests to find the individual error rate for her own lab.

As commander, Doherty's role is that of a supervisor.¹⁰⁵ She spends approximately forty percent of her time reviewing the examiners' case work, and she takes out recently examined evidence and re-examines it on a daily basis. For these reviews, she examines the evidence under the comparison microscope.

105 The other supervisor in the Unit is Detective James O'Shea. He conducts the same supervisory tasks as Doherty.

With respect to her understanding of firearms examination, Doherty accepts the basic, century-old principle that the tooling of weapons gives each weapon a unique working surface on any area of the weapon that is tooled, and an examiner is trained to compare those unique markings. The unique markings can appear from the magazine lip, from the extractor, from the impression of the firing pin on the back of the primer, from the chamber, from the barrel, and from the ejector. An examiner's ability to identify and compare those markings results from his/her education, training, personal study, and own past examinations. Based on her own research and studies, Doherty is not aware of any portion of the firearms examination community that disputes these principles.

The concept of "known non-matches" is very important to a firearms examiner, Doherty believes, because one has to know the limits of known non-matches before one can make an identification. Known non-matches have "considerable agreement" of gross markings, or sub-class characteristics; individual characteristics do not agree. When pressed by defense counsel,¹⁰⁶ Doherty testified that "considerable agreement" refers to similar contours and toolings and is measured solely by one's own experience in conducting examinations and one's own research. "Considerable," in Doherty's opinion, relates to how the markings appear, not to any number of markings.¹⁰⁷

106 During his cross-examination of Doherty, defense counsel directed her attention to an article by John E. Murdock ("Murdock") titled "Some Suggested Court Questions to Test Criteria for Identification Qualifications." 24 AFTE Journal 69 (January 1992) ("Murdock, Exhibit 78"). Murdock suggests that the attorney ask the following question of the examiner: "Isn't it true that there can be, on occasion, a considerable amount of agreement in comparisons of this sort [i.e., known non-matches], especially if the width of the mark being compared is quite small. . . ." Id. at 70. The answer the examiner should give, Murdock writes, is "yes, but if no, you [the attorney] could refer the witness to a 1975 article on boltcutters or Murdock/Biasotti's `State of the art . . .' article[.]" Id. at 71 (footnotes omitted). The defense attorney pressed this point, even though Doherty had effectively already answered "yes" to a similar question and even though Doherty testified that the "boltcutter" article to which Murdock cited and which states that there is agreement among smaller striated marks of known non-matches does not apply to firearms identification. Doherty testified that any agreement in the smaller striated marks of known non-matches would be superficial at best; a closer examination would display the difference.

107 Doherty testified that she is familiar with Biasotti's 1957 study in which he concluded that fifteen to twenty percent of the striations on known non-matches can agree. Doherty pointed out that Biasotti was merely counting lines, not looking at patterns. Biasotti was not engaging in CMS when he conducted this study.

Doherty uses the AFTE Theory of Identification to make identifications, and she uses a cautious approach to her examinations, attempting first to exclude the evidence before finding an inclusion. To find a match, Doherty, like Lydon, looks for sufficient agreement in the pattern and striae to such a degree that another firearm could not have made those markings. Doherty uses the CMS approach as an additional tool in her examinations to check her work. She recognizes that CMS is also a "means of communication" such that, for example, one examiner speaking to another examiner over the telephone can convey an area of identification on projectiles, or, for example, an examiner can speak to others in the Unit about a case. Doherty maintains that this process is not subjective because the examiner is describing what the mark is and where it is.

A potential problem that could arise in an examination is an examiner's basing an opinion on an area that is too small. In this situation, Doherty acknowledged, agreement may be found where there is no match. Examiners, however, do not look at one small area but at the relative positions of random marks. Their examinations are based on the "objective standards" of their training, meaning examiners must examine known non-matches to find the best, they must educate themselves on the literature in the field, and they must take advantage of the "collective knowledge" of the community of examiners. This collective knowledge is best shared through photographs of examinations, and the AFTE Journal often prints such photographs to accompany articles.

C. Mary-Jacque Mann

Mary-Jacque Mann ("Mann") testified on behalf of the Commonwealth concerning her experiences as a firearms examiner. Mann worked as a trace evidence analyst for six years, 1982-1988, at the New Mexico State Police Crime Lab where she used a comparison microscope for her examinations of trace evidence. She became involved in firearms examinations during the nine years, 1989-1998, she spent working at the National Fish Wildlife ("NFW") Forensic Laboratory in Ashland, Oregon; there, she spent approximately sixty percent of her time doing that type of work. Mann no longer works as a firearms examiner.¹⁰⁸

108 In his affidavit in support of the defendants' motion, David Lamagna, a forensic examiner with some firearms examination training, wrote that, in his experience, "individuals like Ms. Mann who do have scientific training and education typically do not remain in the forensic field where they are often times supervised and report to police personnel who have no scientific training or appreciation for the work that they are doing." Lamagna Affidavit, ¶ 63. This point does not persuade this court to doubt the credibility of Mann or any of the other witnesses. Many people in many walks of life perceive that their supervisors are not as capable as they are.

Mann took a firearms examination class in her master's program,¹⁰⁹ and then continued her training when she worked at the NFW lab in Oregon. She trained on the comparison microscope with experienced firearms examiners from the Oregon State Police Crime Lab.¹¹⁰ Her training lasted during her entire time at the NFW lab, from approximately 1989 to 1998. She also attended five AFTE seminars, two firearms examiners workshops, and the Oregon State Police in-service training. Mann also trained other firearms examiners in the use of the scanning electron microscope, discussed further below, including firearms examiners from the Oregon State Police. She considers herself to have been a very conservative firearms examiner who was over-zealous in documenting her examinations and seeking out peer review when she identified a match.¹¹¹

109 This firearms examination course was taught by a firearms examiner from the FBI and among the topics covered was identifications. For this class, Mann did a project using the scanning electron microscope to conduct firearms examinations.

110 The Oregon State Police were in constant collaboration with NFW during Mann's time there.

111 Mann testified that only during training did she also have another examiner review her exclusions and inconclusives. She recalled two occasions where she questioned whether there was a match, and both times the more experienced examiner found a match.

Generally, upon receiving ballistics evidence at the lab, Mann would take certain steps. First, she made a photocopy of the piece of evidence in the state in which it was received. She then conducted an examination of the evidence on the comparison microscope. She considered this "traditional approach" to firearms examination to be adequate for most case work because most of the time, an examiner is able to identify matches. There is an "art" to using the comparison microscope, to angling the light to enable one to view significant markings. Mann used the comparison microscope in an attempt to find an elimination using class characteristics. If she were unable to do so because she found that the class characteristics matched, she examined the individual characteristics.¹¹²

112 Throughout the time she worked at the NFW Lab, sub-class characteristics did not enter into her training or her examinations.

To find a match, she would have to observe an "overwhelming agreement" of the microstriations on the pieces of evidence; this level of agreement is a subjective determination that cannot be objectified, or articulated in objective terms. She looked at the quality, or appearance, of the striae more so than the quantity; she never measured or counted striae. All sciences, in her view, operate under certain assumptions. When she reached a match opinion, Mann sought peer review by another lab.

If a firearm had been recovered, Mann conducted at least six test fires, using both the same type of ammunition as the evidence in question as well as a different type of ammunition. The hardness of the metal would make a difference with respect to the depth of the markings,¹¹³ therefore this process eliminated any question as to whether certain marks were the result of a particular type of ammunition.¹¹⁴ With these test fires, Mann first inter-compared them, checking for consistency of pattern on the consecutively fired bullets and shell casings and making an independent observation of the differences and similarities. Mann then extended her examination to the evidence bullets and shell casings. She took photographs of both types of ammunition if she found a match, and she labeled each individual photograph with the letter or number she assigned to each particular test fire and piece of evidence. She never put multiple test fires together in one envelope without identifying each piece with some identifying marking.

113 See, e.g., Frederic A. Tulleners, "Technical Evaluation: Feasibility of a Ballistics Imaging Database for All New Handgun Sales" at 1-4-1-5 (2001) (Exhibit 76) (concluding after performance tests using different types of ammunition that "impression may not mark in a similar manner with the same level of detail" on cartridge case of all types of ammunition).

114 For example, Mann testified, if a mark appeared on both types of ammunition, she would conclude that the mark came from the firearm.

After using the comparison microscope, Mann conducted a parallel examination on NFW's scanning electron microscope ("SEM").¹¹⁵ The SEM provides a superior level of documentation of matches than the comparison microscope and allows the examiner to see striations within striations. The SEM's increased magnification permits the examiner to see an area in focus from top to bottom, and the SEM produces better photomicrographs than the comparison microscope. Mann's purpose in using the SEM was primarily to enhance her comparison microscope work. The SEM examination enhanced her work in that if she identified two pieces of evidence under the comparison microscope as having been fired from the same firearm by matching their microstriations, an SEM examination permitted her to see the striations within the microstriations. These observations helped her to explain her findings as well. Although Mann would also use the SEM to check her comparison microscope work, she did not use the SEM for eliminations because, with its high magnification, class characteristics cannot be observed.¹¹⁶ Mann never had a situation where she found a match on the comparison microscope and reversed her opinion based on her SEM examination.

115 The SEM is much larger than a comparison microscope; the SEM that Mann used in Oregon took up an entire room. The SEM operates in a vacuum and looks only at the surface of an object; the SEM cannot look through a substance on the evidence to the surface below, therefore the evidence must be very clean before an SEM examination can occur. Unlike the comparison microscope which uses light to form the image, the SEM uses electrons. The images of the evidence are viewed on cathode ray tube monitors. The most common use of the SEM in forensic sciences is the analysis of gun shot residue.
The SEM also has a "firearms comparison stage" that permits the SEM to be used for firearms examinations, although the SEM is not used routinely in the general community of firearms examiners, partly because of its cost and partly because it takes longer to use than the comparison microscope. The firearms comparison stage is a specialized unit into which firearms evidence is placed. With the SEM, an examiner can do a side-by-side comparison of two pieces of ballistics evidence by viewing them on the monitor; one view is "live" while the other is "stored." Both pieces of evidence are in the SEM at the same time, however, so the examiner is able to manipulate them.

116 During her time at the NFW lab, Mann conducted a study using fractal images with the hope of finding a more objective way of examining a projectile. Mann used an instrument, borrowed for the purpose of her study, that documented the topography of the surface of a projectile; the instrument produced a chart of the microstriations on land- and groove-engraved areas. Mann published a summary of this study, and she presented her findings at a conference.

Throughout each step of her examination process, Mann documented what she saw, both with "extremely detailed" notes¹¹⁷ and with photographs. Documentation is a critical part of the scientific method, and the documents served as a reminder of what she had seen.¹¹⁸ She filled in a printed sheet documenting what she saw with respect to class characteristics, including the caliber and weight of the projectiles, and the number and width¹¹⁹ of the land- and groove-engraved areas. With respect to shell casings, Mann documented the size and shape of the firing pin. She usually drew a diagram of the positions of the extractor and ejector marks, and the location of the firing pin relative to the extractor and ejector marks. Mann noted that the examinations are non-destructive and do not cause any alteration to significant characteristics of the evidence. For that reason, there is no problem if documentation is not produced during the initial examination but rather during a subsequent examination. This documentation not only represents and memorializes the examiner's findings but also minimizes claims of bias. The preferred method, however, is for the examiner to document the initial examination as s/he is performing the work.

117 Mann estimated that she took fewer than twenty pages of notes per examination. She noted that her "excessive documentation" is not a universal practice but her own practice that she developed as a result of her scientific background.

118 In Mann's view, documentation, reproducibility, and accountability are important, as is the acknowledgment that these examinations have been conducted successfully for decades. By documentation, Mann means taking photographs, drawing sketches, and taking notes. With respect to reproducibility, Mann uses the example of microstriations. Upon locating a pattern, Mann would determine if that pattern was reproducible by examining a series of test fires from the same firearm; this practice is consistent with the general practices in the field of firearms examinations. See Biasotti Murdock, Exhibit 83A, § 36:8, at 409 ("Appropriately prepared test toolmarks, having the general appearance of the evidence toolmarks, are . . . compared to one another to see if the tool is capable of leaving reproducible toolmarks."). Finally, by accountability, Mann means that the examiner is able to articulate the reason for his/her having found a match to another individual such as another examiner, a judge, or a jury.

119 To measure the land- and groove-engraved areas, Mann used a stage micrometer located on the eyepiece of the comparison microscope. She described the effect of this measurement tool as putting a clear ruler over everything she viewed under the microscope.

With respect to photographs, Mann located an area of significance on two pieces of evidence then rotated the evidence to view the area from all angles before coming to a conclusion, but she did not necessarily photograph the evidence at all angles with respect to that one area. The photographs she took were not meant to display the "reasons" for finding a match; rather, she took photographs when she found a match as a representation of what she had seen.¹²⁰

120 Mann testified that during the time she was at the NFW lab, she used a Polaroid camera to photograph the evidence. She admitted that her boss thought that she often "over-documented," budget-wise, with photographs.

D. Peter Striupaitis

Peter Striupaitis ("Striupaitis") is in his twenty-seventh year as a firearm and toolmark examiner and he currently works at the Northeastern Illinois Crime Lab and as a contractor for the Illinois State Police where he trains firearms and toolmark examiners. Striupaitis has a bachelor's of arts degree in criminal justice from the University of Illinois, Chicago, and a master's degree in corrections and criminal justice from Chicago State University. He began training as a firearms and toolmark examiner in 1980 with the Illinois State Police's two-year documented training program that consisted of written and oral tests. He was not a sworn officer, but a civilian as were the other firearms examiners. Striupaitis has been a member of AFTE¹²¹ since 1981, and he has held various positions in AFTE including vice president and president. Striupaitis believes that approximately 900 of the 1,100 firearms examiners in the world are AFTE members. Since 1980, his work has been directly connected to firearms and toolmark examination¹²² and, in his current positions, the substance of his daily work involves examining evidence and entering evidence into IBIS. He takes a CTS test¹²³ each year, with the most recent being in January 2006. Over the course of his career, he has examined tens of thousands of firearms and hundreds of thousands of spent bullets and shell casings.

121 Membership in AFTE is voluntary. Since 1999, AFTE has offered a process by which examiners can become certified in three different areas, range proximity testing (gunshot residue), toolmark identification, and firearms identification. Twenty-three examiners are certified as toolmark examiners, and fifty-three examiners are certified as firearms examiners; there is some overlap between these two groups. Striupaitis knows of some, but not many, examiners who are certified in all three areas. See United States v. Mahone, 453 F.3d 68, 71 (1st Cir. 2006) ("It is not required that experts be blue-ribbon practitioners with optional certifications.").
The certification process includes a written portion that tests the examiner's knowledge of, for example, firearms examination, and a practical portion that involves examining specimens under the comparison microscope while monitored by a tester. The only real advantage that comes from becoming AFTE certified, according to Striupaitis, is that the examiner achieves recognition in a program created by his peers. Striupaitis has not sought certification, but he has taken proficiency tests over the course of his career, and he passed all of them.

122 Striupaitis spent approximately seven years as an administrator. In that position, he did not examine evidence on a daily basis, but he maintained his proficiency through testing and case work.

123 Striupaitis noted what he considers a flaw in CTS' tests: (1) those administering and evaluating the test do not look at every specimen that is sent out to be examined; and (2) an "inconclusive" answer could, in fact, be correct, but it is marked wrong. Once the inconclusive answers are taken out of the calculation, Striupaitis points out, the error rate is very low. See also Biasotti Murdock, Exhibit 83A, § 36:11, at 415-416 (setting forth flaws of proficiency testing).

Just as Lydon, Doherty, and Mann testified, Striupaitis is a proponent of the AFTE Theory of Identification that requires an examiner to find "sufficient agreement," both quantitatively and qualitatively, between specimens before finding a match. The sufficient agreement must exceed the agreement of the examiner's best known non-match. This identification is not absolute, however, because no one examiner can fire every existing firearm and examine the spent bullets and shell casings. A firearm examiner's ultimate conclusion is subjective, but based on objective observations. Therefore, examiners can only make identifications to a reasonable degree of firearm identification certainty. Striupaitis does not know of any firearms examiner who uses CMS in lieu of the AFTE Theory of Identification; those who use CMS also use the AFTE Theory of Identification. In fact, Striupaitis has attended AFTE conferences since 1981 where he has met other examiners and attended various seminars, and he does not know of any examiner who disagrees with the AFTE Theory of Identification or the underlying principles of the field. There are, of course, various disagreements among examiners in the field,¹²⁴ and the examiners discuss many of those disagreements at the AFTE conferences, but he testified that the AFTE Theory of Identification is generally accepted among the community of firearms examiners. Based on all of the evidence presented at the hearing, the court credits Striupaitis' testimony as to the AFTE Theory of Identification.

124 Striupaitis also testified that there can be disagreements between two examiners in a lab, with one seeing an inconclusive and the other seeing a match. In this instance, a third examiner is brought in to "referee;" if an error is found in either examiner's examination, corrective measures are taken.

Just like Lydon, Doherty, and Mann, Striupaitis uses the "pattern matching" method in his firearms examinations. With this method, Striupaitis looks for significant agreement in the totality of the pattern, including agreement of the curvature, the spatial relationship, and the depth of the two pieces of evidence. He does not look for a percentage of agreement.¹²⁵ Instead, with his trained eye, he has developed an "internal database" of best known non-matches that enables him to make identifications based on sufficient agreement between marks by comparing one similarity or difference to another. This method of pattern matching is generally accepted in the firearms examination community.

125 Striupaitis is familiar with Biasotti's 1959 study of percentage of agreement among known matches and known non-matches. With this study, Striupaitis testified, Biasotti intended to develop a mathematical model for pattern recognition. The fact that the percentage of matching striations between known non-matches is not dramatically different from the percentage of matching striations between known matches, however, has no effect on Striupaitis' work. As an examiner, he looks at the contour, height, width, and spatial relations of the markings on a piece of ballistics evidence; essentially, he looks for patterns, something Biasotti did not take into consideration. See, e.g., Biasotti Murdock, Exhibit 83A, § 36:12, at 418 n. 7. The contour, height, width, and spatial relations will not match on known non-matches. By "contour,' Striupaitis testified that he means the dimensions, characteristics, quality, and shape of the markings. If the two pieces of evidence were fired from the same firearm, all of those aspects will match.

As noted, Striupaitis maintains that a firearm examiner's ultimate opinion is subjective, but based on objective observations. Procedures that examiners use to validate their subjective opinions include documentation and peer review. Examiners must document their work during their examinations, and they must have the case work reviewed by another examiner. Striupaitis has used both of these procedures throughout his career, although the Illinois State Police did not require review until the mid-1980s after he had worked there for a few years. There, an examiner is required to have his/her work reviewed not only when s/he finds a match but also when the firearm and the evidence have the same class characteristics but the examiner finds an elimination based on individual characteristics.¹²⁶ Striupaitis still uses this verification method, and he minimizes the possibility of confirmation bias¹²⁷ by merely asking the other examiner to look at some evidence that he has in the comparison microscope without disclosing his conclusion.

126 This practice is superior to that followed at the Unit when only a match opinion is routinely reviewed by a supervisor, thereby creating a type of confirmation bias.

127 David Lamagna, who submitted an affidavit in support of the defendants' motions, defines "confirmation bias" as

"the extent to which an examiner sees what he expects to see. In the vast majority of cases, firearms examiners are told whether the investigators expect or hope that the evidence was fired through a particular recovered firearm. In those circumstances, `confirmation bias can cause firearms examiners to over-estimate the quality or quantity of striae and other toolmarks when they have external reasons to expect a match and to under-estimate the quality and quantity of striae and other toolmarks when they have external reasons to expect a non-match. So long as the criteria is wholly subjective, confirmation bias may be impossible to avoid.'"

Lamagna Affidavit, ¶ 43, quoting Steele, supra, at **11.
"As extraneous contextual effects are more pronounced, greater distortions can arise. . . . It is important to establish empirically that experts can be influenced by extraneous contexts." Itiel E. Dror, et al., "Contextual information renders experts vulnerable to making erroneous identifications," 156 Forensic Sci. Int'l 74, 76, 77 (2006) (discussing "first research study to experimentally examine the possible impacts of extraneous context in the real world of biometric and forensic science"). But see Risinger Saks, Exhibit 44, at 14 (noting that "perception is directed by expectations but not controlled by them; it involves the pickup of real information"). "Examinations in forensic science labs are currently the equivalent of show-ups in the eyewitness realm. In both settings, the test is structured to be single-suspect, implying that the correct suspect is in hand and preventing the ultimate decisionmakers (the courts) from evaluating the quality of the test and the likely validity of the results." Risinger Saks, Exhibit 44, at 48 (footnote omitted). This practice results in the potential for confirmation bias to result. A possible technique to "provide a solution to this problem . . . [is] an evidence lineup. In an evidence lineup, the examiner would be presented with multiple specimens, some of which were `foils.' The examiner would, of course, be blind to which items of evidence in the evidence lineup are foils and which are the true questioned evidence." Id.

Throughout his career as a firearms and toolmark examiner, Striupaitis has followed his own basic method of examination. First, upon receiving the evidence in sealed condition, Striupaitis makes notations regarding the firearm's characteristics, including its caliber and whether he found any trace evidence in or on it, and the evidence's characteristics, such as its weight, its measurements, and its physical condition. He receives damaged projectiles from time to time, but the damage does not necessarily preclude him from conducting an effective examination. Second, he test fires the firearm into a water tank; he always takes at least two test fires in order to determine reproducibility.¹²⁸ He labels the test fires so he can distinguish between or among them, then he compares the test fires to each other under the comparison microscope. He documents the fact of having test fired the firearm, but he does not document that he compared the test fires to each other; he compares them in order to form a "standard" as to the firearm's test fires.

128 Striupaitis testified that he has seen instances where different ammunition has not reproduced on shell casings. Some ammunition may "imprint" differently because of differences in primer hardness, in powder, and/or in color. Even though the imprint of markings may not be as vivid or clear on some types of ammunition, however, identifications are not precluded.

Third, he removes one of the test fires from under the comparison microscope and replaces it with the evidence in question. He moves the specimen to find areas of agreement or disagreement. Fourth, he reaches his conclusion, and, finally, he has another examiner review his work by looking at the evidence under the comparison microscope.¹²⁹ The examiner who verifies Striupaitis' work does not produce a separate report but does stamp and initial Striupaitis' report.

129 As noted above, Striupaitis has another examiner review his work when he has made an identification or if he has found the two pieces of evidence which have the same class characteristics but he reaches the conclusion of elimination.

Striupaitis himself documents the entire process and takes photographs. The comparison of a specimen to the test fire can last from hours to days depending on the condition of the evidence. Striupaitis knows of no false positives he ever made. On one occasion, however, after examining a firearm and a spent bullet and reaching the conclusion of inconclusive, a quality assurance process was conducted, and Striupaitis re-examined the evidence. At that point, Striupaitis concluded that there was a match.¹³⁰

130 While it is likely that "the [firearms examination] field is in a poor position to calculate error rates[,]" the evidence supports a finding that labs are able to measure error rate on an individual level. Biasotti Murdock, Exhibit 83A, § 36:11, at 414.

When Striupaitis first began working as a firearms examiner, sub-class characteristics were not a concern because no one had defined them in the 1980s, although examiners had noticed them.¹³¹ Striupaitis described sub-class characteristics as discernible characteristics on the groove-engraved areas of an object caused by a tool with a defect, such as a broach¹³² with an imperfection, and repeated throughout the manufacturing process. An examiner cannot rely on sub-class characteristics to make an identification because they may be present on a number of firearms. An examiner can locate sub-class characteristics through observation of the spent projectile or shell casing or by looking at the barrel of the firearm. Sub-class characteristics would occur through the length of the barrel,¹³³ so, to determine if a mark is a sub-class characteristic, an examiner can look into the barrel with a borescope, a standard instrument for this purpose. The examiner can also take a cast of the barrel, a generally accepted means of identifying sub-class characteristics. The examiner then examines the barrel cast with and/or without the comparison microscope.

131 Striupaitis recalled that, thereafter, nineteen articles appeared about those characteristics, seventeen of them in theAFTE Journal.

132 Striupaitis testified that the effect of an imperfection on a reamer might cause sub-class characteristics.

133 Striupaitis testified that if the mark were to stop and restart down the length of the barrel and not run continuously, it would not be a sub-class characteristic but an individual characteristic. He did not, however, explain why this is the case.

Striupaitis conducted two studies about which he testified. These studies confirmed his understanding of the field of firearms investigation, that firearms examiners can match bullets and shell casings to the firearms from which they were fired by examining the markings impressed upon them. First, in the early 1980s, Striupaitis designed a "wear study" of consecutively fired weapons to determine if individual characteristics maintained themselves and whether multiple firings affect an examiner's ability to identify matches. In that study he fired a Raven .25 caliber semi-automatic firearm 501 times using jacketed ammunition, then examined the bullets and the shell casings, comparing the first to the second, the first to the third, etc. On the spent shell casings, he looked at markings on the breech face, the head, and the primer areas. He looked at the chamber markings, but it is not his practice to make identifications based on chamber markings.¹³⁴ He did find some changes over the 501 firings, but he observed no discernible difference between bullet #1 and bullet #501, and each bullet and shell casing was identifiable.¹³⁵ Although his study did not result in this conclusion, Striupaitis acknowledges that some degree of wear and/or damage could prevent identification; in that event, however, there would be a false negative, not a false positive.

134 Striupaitis testified that chamber markings are "fine" and his standard operating procedure is to rely on chamber markings only if it is necessary because, for example, he cannot find any breech face marks, etc. He has never had to make an identification based on chamber markings.

135 Again, it is clear that this study was not "blind."

Second Striupaitis received a grant to conduct a firearms examination study at some point between 2002 and 2004.¹³⁶ In this validation study, Striupaitis studied consecutively rifled barrels. He obtained six consecutively rifled firearms from two different manufacturers, and thereafter fired the twelve firearms two times each. First, he entered the two spent projectiles from each into IBIS.¹³⁷ Second, Striupaitis distributed the test fires among a group of twenty-to-thirty examiners who had volunteered to be part of the study. Each examiner was given six projectiles numbered one through six ("known bullets") and six projectiles ("unknown bullets") that they had to match to the known bullets. The examiners correctly matched all of the unknown bullets to the known bullets; there were no misidentifications. This study by Striupaitis appears to have been conducted on a "blind" basis.

136 Striupaitis could not recall if he did this work from 2002 to 2003 or from 2003 to 2004.

137 Striupaitis also entered the spent bullets into "Cyclops," a video imagery program for firearms examiners. Cyclops utilizes a laser operation that produces graph-like images of the contour of the bullet. The intent was to integrate this system with IBIS.

V. Opponents of the Field of Firearms Examination

This court also heard testimony from Adina Schwartz, an academician whose experience in the field of firearms examination is limited to reading literature in the field and publishing her own critical articles. Adina Schwartz testified on behalf of the defendants, urging that the principles underlying the field of firearms examination are unreliable. The defendants also provided an affidavit from David Lamagna, a forensic examiner and investigator.¹³⁸ David Lamagna's background in firearms examination is limited; he does not conduct firearms examinations on a day-to-day basis, and his training in the field occurred during the years 1994 and 1999 and included attending armorer's schools and instruction courses. Both Adina Schwartz and David Lamagna raise important issues. This court concludes, however, that the field of firearms examination is reliable.

138 The Commonwealth agreed not to move to strike Lamagna's affidavit as long as the Commonwealth could file testimony Lamagna gave at an evidentiary hearing in United States v.Prochilo, Crim. No. 96-10321-DPW (D. Mass. April 28, 2000) (Woodlock, J.) and at the trial of United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.). The Commonwealth has filed those transcripts as Exhibits C and D to the Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence.

A. Adina Schwartz

Adina Schwartz ("Schwartz") is currently a professor at John Jay College of Criminal Justice in the undergraduate and master's programs, and at the Criminal Justice Ph.D Program of the Graduate Center at City University of New York. She began her career as an academician and philosophy professor at Yale University. She left that field in 1982 at which time she pursued her law degree from Yale University. Schwartz worked as a civil litigator for approximately three years, then she did appellate work for the federal defender's office. She began publishing articles dealing with scientific evidence and the law in 1994, about the time Daubert, 509 U.S. 579 (1993), was decided; thereafter, she became interested in toolmark identification of which firearms identification is a subset. She has since immersed herself in the literature of toolmark identification.¹³⁹ Among her own publications is "Firearms and Toolmark Identification" in Jane Campbell Moriarty, Psychological and Scientific Evidence in Criminal Trials, West (2004 edition).

139 Schwartz testified that she finds the literature on toolmark and firearms identification to be "off the cuff" and "chatty."

Schwartz has never been trained as a firearms examiner or conducted a firearms examination,¹⁴⁰ but she has gone to a ballistics lab and looked at cartridge cases under a comparison microscope on two occasions. She has never conducted a test concerning the changes in toolmarks over time, nor has she ever written or taken a proficiency test in the field of firearms investigations. She has never attended an armorer's school, watched the manufacture of a firearm, spoken with firearm manufacturers, or fired a gun. The hearing in these cases was the first time in which she testified in court as an expert witness.

140 When asked during cross-examination whether she knows how to compare marks between projectiles or shell casings, Schwartz testified that she believes no one can.

Schwartz believes that the claim of toolmark examiners that they can identify a particular tool as having made a particular mark is unreliable, especially given that the standard is, essentially, that the examiner will know a match when s/he sees it.¹⁴¹ To be reliable, in her view, examiners must be able to rely on a database and statistics associated with that database as well as proficiency testing. Because toolmark examiners have the benefit of none of those features, there is a bias that permeates their training and, in turn, the field as a whole. This court has considered carefully, but does not credit, these opinions.

141 Schwartz noted a movement in the fingerprint investigation field requiring fingerprint examiners to move away from basing one's conclusion on one's "many years in the field" and instead basing one's conclusion on "X, Y, and Z" and being able to articulate why "X, Y, and Z" are important.

IBIS is a digitized database of shell casings that enables an examiner to compare shell casings from anywhere in the United States. The reliability and effectiveness of this database was called into question in the 1997 article by Masson who wrote that as IBIS expands, shell casings fired from different firearms look increasingly more like shell casings fired from the same firearm, thereby making them difficult to distinguish. Masson, Exhibit 75, at 43. Masson viewed IBIS as a teaching tool and noted that to limit the database would be to limit the universe.

Schwartz, in turn, believes that Masson's article vindicates Biasotti's concern that as the universe expands, identifications are more difficult and less reliable. Biasotti was a toolmark examiner whose 1955 master's thesis at University of California at Berkeley was the first attempt to establish a statistical foundation for firearms identification. Biasotti published a summary of that master's thesis in 1959 in an article titled "A Statistical Study of the Individual Characteristics of Fired Bullets." Alfred A. Biasotti, "A Statistical Study of the Individual Characteristics of Fired Bullets," 4 J. of Forensic Sci. 34, 37 (January 1959) ("Biasotti 1959, Exhibit 82"). Biasotti was motivated to conduct this study because he was concerned that firearms examinations rested on examiners' "eye-balling" projectiles and concluding there was a match.

In this study, Biasotti used bullets fired from Smith Wesson revolvers, and he examined both known matches and known non-matches. He individualized each land-engraved area and found the overlap between and among striae was so significant that to differentiate between matches and non-matches was impossible. Biasotti discovered a match of fifteen to twenty percent of the striae in known non-matches. Schwartz points out two problems with Biasotti's study, acknowledged by Biasotti himself: it was not a blind study in that he knew which bullets were matches and which bullets were non-matches; and the counting of striae is subjective in and of itself.

Biasotti recognized that his study was just a beginning and he hoped for more research. In 1997, Biasotti and John Murdock ("Murdock"), a prominent firearms examiner, propounded the firearms examination method of CMS; the only published research they based this technique on was Biasotti's study set out in 1959. In 2000, Jerry Miller ("Miller") conducted a study in which he used IBIS to test whether Biasotti's theory, by then over four decades old, actually worked. Schwartz, supra, at 22 n. 88, citing Jerry Miller, "Criteria for Identification of Toolmarks Part II," 32(2) AFTE Journal, 116, 130 (2000). Miller determined that many false negatives resulted and that known matches did not satisfy Biasotti and Murdock's CMS requirements. Id. Miller proposed that, instead of conducting CMS on one land-engraved area to determine if there is a match, "examiners should conclude that the same gun fired both test and evidence bullets whenever the CMS criteria are satisfied by the sum of the consecutive matching striae on all land impressions[,]" thus conducting CMS on each land-engraved area and totaling the number of matches.Id. at 23 (emphases in original), citing Miller, supra.

Schwartz believes that the number of striations on a projectile is relevant to an examiner's determining whether there is a match. In her view, a claim of identification is by its nature a statistical claim. Unlike DNA and fingerprints, however, the number of striations a single firearm will leave on all of the projectiles it fires will never be exactly the same. The striations, therefore, cannot readily be set out in a database.¹⁴² This court finds that a database is neither practical nor necessary.

142 Schwartz' analogy to fingerprints is not persuasive. The evidence fingerprint is invariably a partial print and the recovered portion is often not as clear and sharp as the exemplar. At least in these respects, fingerprints are similar to toolmarks.

In his 1959 paper, Biasotti did not include a discussion of sub-class characteristics. Schwartz believes that the first mention of the concept of sub-class characteristics was in a 1974 article written by Murdock concerning the marks that staplers left on staples. The point of Murdock's article was to warn examiners to be aware of the existence of sub-class characteristics. The term "sub-class characteristic" became part of the AFTE Glossary in 1992. The existence of sub-class characteristics calls into question the issue of uniqueness of the marks on a firearm and, as a consequence, the marks on a projectile and shell casing. Sub-class characteristics, in Schwartz' view, make firearms identification scientifically problematic. Just a "rule of thumb" warning with respect to sub-class characteristics is not enough, in Schwartz' opinion. This court finds that examiners can reliably distinguish sub-class characteristics.

Another problem Schwartz sees in the field of firearms examinations is a lack of uniformity among police departments and crime labs. For example, some examiners take notes and photographs throughout the examination process rather than at the end; and some labs require more than one examiner to examine evidence and to take notes. Schwartz believes that there can be no genuine peer review in the absence of documentation simultaneous with the examination; the examiners should leave a "trail" with the intent of peer review. One examiner should be able to look at another examiner's report and then replicate the examination. Lydon's reports with respect to his examination of the evidence in Meeks and Warner include only Lydon's bare conclusions, thereby making them "impossible to evaluate." See Lydon Warner Report, Exhibit 8; Lydon Meeks Report, Exhibit 9. In Schwartz' opinion, Lydon's reports are "non-reports." Seeid. Although this court concludes that Lydon's work was reliably re-evaluated by both Doherty and Lydon himself, there is much force in these criticisms of Schwartz. As noted above, the Unit has begun to take measures to document, photograph, and peer review its work.

Schwartz believes that peer review, as it is currently exercised, is problematic. Generally, a firearms examination is only reviewed by another examiner when an identification has been made by the first examiner. Because the peer reviewer knows that the other examiner has identified the evidence as having been fired from the same firearm, "confirmation bias" results. Schwartz testified that this procedure is different from that followed in forensic DNA labs where the evidence is peer reviewed regardless of the conclusion reached. She believes that confirmation bias can also exist with the original examiner if he collects projectiles and/or shell casings and is subsequently presented by the police with a recovered firearm suspected to have fired the collected projectiles and/or shell casings. As noted below, this court concludes that peer review clearly enhances reliability. Ideally it would be "blind" and done with respect to every examination. Budgetary restraints may well preclude that approach. As noted above, the Unit now does a peer review of match opinions, and ultimately did so in these cases. The Unit's procedures reasonably assure reliability; the shortcomings of the Unit's peer review procedures can be readily explored before the jury on cross-examination.

Schwartz is familiar with the concept that photographs should be used as a roadmap to aid in peer review,¹⁴³ but she believes photographs have limits. First, an examiner only photographs the areas s/he believes to be the basis for the identification. Second, by taking a photograph of that "area of agreement," the examiner is asserting that s/he found sufficient agreement between the two pieces of evidence. Another examiner, however, cannot look at that photograph and examine any other areas. Schwartz also maintains that examiners should photograph inconclusives and exclusions in addition to identifications in order to aid peer review. Finally, she argues that the examiners in the Meeks and Warner cases took photographs after the examination in order to justify the identification. Schwartz views these photographs as being of minimal value; to be helpful, the photographs should be taken before the identification and throughout the examination process. Again, as noted below, this court concludes that photographs of matches, even after the initial examination as occurred here, provide reasonable assurance of reliability.

143 Schwartz attributed this theory to Bruce Moran.

Proficiency testing is also required to make the field reliable. Although such testing is administered now, only CTS is approved by ASCLD, and a disclaimer appears on CTS' tests that states that the tests are not intended to measure the proficiency of the field as a whole. Schwartz finds it troubling that the only measure of proficiency explicitly states that it cannot be used to measure proficiency. Schwartz acknowledged that others in the field have written that, while the CTS tests cannot test proficiency, the tests can be used to develop an error rate. She also points out that the tests are not representative of actual case work because with the tests, the test-takers examine large amounts of evidence in the absence of a firearm. Moreover, the testing is not blind because the examiners know they are being tested and may take special care in their examinations. The ideal method of testing would be blind testing, with the examiners receiving samples mixed in with their actual case work without knowing they are being tested. As noted below, although several of Schwartz' criticisms are serious, there exists credible evidence that the error rate for false positives is low. Further, testing appears to be improving, including at the Unit.

Schwartz is of the opinion that even if such measures were taken, and confirmation bias were not an issue, the premise of the field itself is inherently unreliable. For example, the composition of ammunition affects the way markings are left; no two pieces of ammunition can be exactly the same. Even consecutively fired bullets¹⁴⁴ show a difference in the markings. As noted below, this court concludes that these variables do not undermine the basic reliability of firearms examinations in general or in these cases.

144 Schwartz testified that she has only seen documentation concerning consecutively fired cartridge cases on one occasion, specifically a federal case in California where the Los Angeles Police Department acknowledged that while consecutively fired cartridge cases are difficult to match, the examiners still made an identification. Schwartz is involved in that case as an expert.

Schwartz considered the evidence in Meeks and Warner and believes that the examinations of the ballistics evidence in each case is unreliable and should be excluded, although she has not asserted that there have been misidentifications in this case. Several aspects of the two cases concern her. First, in both cases, confirmation bias in Lydon's initial examinations was present because Lydon had been told that the recovered firearms were possibly connected to the evidence that Lydon had already examined. Second, with respect to Warner, there was an indication in Lydon's report that Torres and the Brookline police officer waited to learn the result of Lydon's comparison of the test fire from the recovered Intratec pistol and the ballistics evidence he had already examined. Typically, a firearms examination takes approximately thirty minutes; as Lydon conducted four such examinations while Torres and the officer waited, Schwartz questions whether Lydon rushed through the examinations.¹⁴⁵ Third, there is no indication in Lydon's report that he took into consideration the fact that the Intratec's barrel might have changed over time.

145 As set forth above, Lydon's and Torres' testimonies established that the men did not wait for Lydon to conduct his examination, therefore Schwartz' concern is unfounded.

Additionally, with respect to Meeks, Schwartz was concerned that Lydon was unable to find in January 2005 the area of identification — between one of the bullets removed from Sanders' body and the test fired bullet from the recovered Smith Wesson revolver — that he had previously located in August 2002. Schwartz also pointed out that in his August 2002 report, Lydon wrote that he found one good land-engraved area; in his February 2005 memorandum to the Commonwealth, Lydon wrote that, after having conducted further investigation of the same pieces of evidence from his August 2002 examination, he found two land-engraved areas and one groove-engraved area. Lydon did not explain this change in his opinion in his February 2005 memorandum; Schwartz found this omission troubling. This last concern exemplifies her criticisms of the firearms investigation field: because an examiner relies upon his/her own subjective criteria of identification, each examiner's criteria changes over time and, in the absence of thorough documentation, an examiner is able to make a different analysis at a later time. As noted below, Schwartz' criticisms of Lydon's work have merit and certainly provide a basis for cross-examination as to the weight his testimony should receive. These criticisms do not, however, render his testimony so unreliable that it should be excluded.

Schwartz also had concerns with respect to the photographs Doherty took in January 2005, over two years after Lydon's original examination. Photographs, in her view, are meant to be used as a roadmap to aid in peer review, but knowing that these photographs were taken well after the August 2002 identification, another examiner viewing the photographs could not be confident that they were photographs of the same area leading to that August 2002 identification.

B. David Lamagna

As noted above, David Lamagna ("Lamagna") did not testify at the hearing on these motions, although he has testified as a firearms examiner in other cases. The court takes the following information from Lamagna's affidavit which the defendants filed with this court in support of their motions in limine, and the transcripts of his prior testimony submitted by the Commonwealth. This court does not extensively credit the testimony of Lamagna.

Lamagna is a forensic examiner and investigator with training in firearms examination. He has a bachelor's of science degree in plastics engineering with a minor in biology, and a master of science degree in materials science; he is pursuing his doctorate in materials science. His education and work have given him a background in the manufacturing process of firearms, surface metrology, toolmark analysis, and the interpretation and mathematical processing of measured data. Lamagna is a member of such professional organizations as the National Society of Professional Engineers, the Microscopy Society of America, the American College of Forensic Examiners, the American Society for Law Enforcement Training, and the Society of Manufacturing Engineers. He is not a member of AFTE. Lamagna has published articles on forensic science in the Journal of the Massachusetts Academy of Trial Attorneys and The Champion. He is a licensed private investigator in several states and a certified medical investigator and computer technician, and he is currently a principal¹⁴⁶ of American Forensic Technologies¹⁴⁷ with offices in Massachusetts and Florida.

146 Lamagna is the only full-time employee of American Forensic Technologies, although he does "work with other people" on a consulting basis. United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.), Lamagna Testimony, p. 3244, Exhibit D to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence.

147 According to its homepage, American Forensic Technologies "provide[s] forensic consulting, examination, investigative services, products, supplies, books and literature, for . . . [eighteen] forensic disciplines[,]" including arson investigations, questioned documents, drug testing and analysis, DNA profiling and serology, bloodstain pattern analysis, crime scene reconstruction, fish and game investigations, wrongful death investigations, computer forensics and security, product liability and failure analysis, and firearms examination and investigation. American Forensic Technologies, available at http://www.americanforensic.com, attached as Exhibit B to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence.

The Curriculum Vitae attached to his affidavit shows that ballistics, and firearms examination in particular, are only a small part of Lamagna's background and experience. Lamagna has taken a number of various certificate level training courses since 1994, including those on such topics as DNA profiling techniques; genome sequencing and analysis; blood stain pattern analysis; homicide investigations; crime scene investigations; physical evidence; forensic photography; palm print classification; accident reconstruction; fingerprint technology; footwear, toolmarks, and firearm impressions; arson and explosion investigations; trace evidence; forensic investigations into drugs and alcohol; document and voice examination; submachine guns; use of deadly force investigations; fraud investigations.

Lamagna also took certificate-level training courses in firearms-related topics between 1994 and 1999. These courses included several armorer's schools¹⁴⁸ with Smith Wesson, Ruger, and Glock; gunsmithing, and a course in firearms examination through the American Institute of Applied Science in North Carolina.¹⁴⁹ Lamagna never worked under an experienced toolmark examiner as an apprentice, unlike Lydon, Doherty, Mann, and Striupaitis. United States v. Prochilo, Crim. No. 96-10321-DPW (D. Mass. April 28, 2000) (Woodlock, J.), Lamagna Testimony, p. 46, Exhibit C to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence; see United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.), Lamagna Testimony, p. 3300, Exhibit D to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence (testifying that he did not "feel it was necessary to work under a patrol officer to learn to be a scientist").¹⁵⁰

148 According to Lamagna, the purpose of armorer's school is to teach the attendees how firearms are manufactured and how firearms function. United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.), Lamagna Testimony, p. 3287, Exhibit D to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence.

149 Lamagna took one "distance education course" in toolmark and firearm examinations through the American Institute of Applied Science in North Carolina. United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.), Lamagna Testimony, pp. 3281-3282, Exhibit D to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence. His recollection was that the course was approximately thirty-four hours long. Id. at p. 3283.

150 Additionally, Lamagna could only recall three specific occasions on which he testified in Massachusetts as a toolmark examiner. United States v. Jackson, Crim. No. 02-756 (S.D.N.Y. Dec. 21, 2005) (McKenna, J.), Lamagna Testimony, pp. 3267-3269, Exhibit D to Commonwealth's Proposed Findings of Fact and Conclusions of Law Concerning Firearms Examination Evidence.

Lamagna relied on his background in the chemical, plastics, and metal working industries to reach conclusions about the impressions toolmarks leave on firearms. First, changes in the methods of firearm manufacture result in "far fewer toolmarks on firearms. . . ." Lamagna Affidavit, ¶ 16. Specifically, by "the end of the twentieth century, newer manufacturing methods were being perfected and applied in the firearms manufacturing field . . . [including] investment casting, die-casting, metal injection molding, computer-controlled methods [such as CNC which Borgio testified about] . . .; as well as hammer forging, and electrochemical machining of gun barrels, etc." Id. Prior to those advancements,¹⁵¹ gunparts "still required some degree of hand finishing work such as hand-filing, etc.[,]" that would contribute to the uniqueness of the interior of each barrel. Id. ¹⁵² SAdditionally, "[d]ifferent brands of ammunition are manufactured using metals of different degrees of hardness, grain structure, alloy content, etc. The harder the ammunition, the fewer marks from particular gun parts . . . will be transferred onto the cartridge case head. . . . Other brands of ammunition are sometimes made of a softer metal and will take more impressions." Id. at ¶ 31.

151 Traditionally, the manufacture of firearms involved the "machining and fitting by hand . . . [of] parts that were machined from rough castings, forgings, and sheet metal stampings." Lamagna Affidavit, ¶ 15. In the late nineteenth and early twentieth centuries, the parts themselves were sized and fit by hand as well. Throughout the twentieth century, although there were improvements to the tools, "gunparts . . . were still manufactured from rough casting, forgings, and sheet metal stampings. These parts still required some degree of hand finish work such as hand-filing, etc." Id. at ¶ 16. By "the end of the twentieth century, newer manufacturing methods were being perfected and applied in the firearms manufacturing field."Id.

152 Lamagna states in his affidavit that "law enforcement firearms experts acknowledge, for example, that modern manufacturing methods make it impossible, using the comparison microscopes available in most forensics laboratories, to determine whether a particular Glock firearm fired a particular bullet." Lamagna Affidavit, ¶ 23. Doherty conducted a study, not blind, of consecutive serialed weapons by examining test fires from Glock firearms and compiling a database of two test fires from each firearm, test fire #1 and test fire # 850. Although her study involved the examination of shell casings rather than bullets, on which Lamagna's statement focuses, Doherty was able to match the markings on the casing of test fire #1 and test fire #850 for each Glock firearm in her study.

While Lamagna points out serious issues with respect to toolmarks on spent bullets and shell casings, these issues do not support a finding of unreliability. With respect to the manufacturing process in the late twentieth century, Borgio testified that the Smith Wesson revolver allegedly involved in the Meeks case was neither broached nor reamed with a CNC machine. Borgio also examined the Intratec firearm allegedly involved in the Warner case, and testified that it appeared that die-casting had been used for the frame, that the barrel had been reamed and broached, and that the breech face had been made with a rotary tool. The issues raised by Lamagna, including the type of ammunition used for the test fires, concern the weight of the evidence in the Meeks and Warner cases rather than the reliability of the methodology of the field of firearms examination. Therefore, exploration of these subjects is appropriate for cross-examination.

According to Lamagna,

"the real issue is not what type of manufacturing method was employed, but how to perform any firearm examination and identification, with a high degree of scientific certainty. The basic difference between an art form or technology and a real science is the ability to describe the result of any experiment, examination, measurement, or identification in mathematical terms. Every well-known philosopher/scientist since the time of Pythagoras (and before) has stated that if something cannot be described mathematically, it either does not exist, or we do not fully understand it."

Lamagna Affidavit, ¶ 19. There are several problems with this passage that contribute to this court's attributing little credibility to Lamagna's affidavit. First, as discussed further below in the Rulings of Law section, Daubert and its progeny govern a court's "preliminary assessment of whether the reasoning or methodology underlying the [expert scientific or technical] testimony is [technically or] scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue." 509 U.S. at 592-593 (emphases added). Therefore, Lamagna's attempt to require a "high degree of scientific certainty" from firearms examinations is not the relevant standard to apply to an assessment of the reliability of the testimony. See id. Second, Lamagna's attempt to distinguish between "an art form or technology and a real science" is inapplicable on these facts. Firearms examination need not be defined as a scientific field because "Lanigan applies to technical evidence as well as scientific evidence. . . ."Commonwealth v. Patterson, 445 Mass. 626, 642 (2005), citingCanavan's Case, 432 Mass. 304, 313 (2000). There are many aspects of medicine, psychiatry, psychology, the social sciences, and law which "cannot be described mathematically." Although no one may "fully understand" any of these fields, reliable expert testimony is regularly permitted to assist the jury. "[T]here are important differences between the quest for the truth in the courtroom and the quest for truth in the laboratory." Daubert, 509 U.S. at 596-597. Our legal procedures are "designed not for the exhaustive search for cosmic understanding but for the particularized resolution of legal disputes." Id. at 597.

Lamagna also found fault with the BPD's conclusions in theMeeks and Warner cases. First, the "bald" identifications in both cases were insufficient because there was no supporting documentation. The reports "do not describe or specify the quantity or quality of the markings on the spent ammunition components, which the BPD used to reach its conclusions." Lamagna Affidavit, ¶ 6. Further, "[t]he BPD's examinations in theWarner and Meeks cases did not provide sufficient scientific basis for their conclusions, nor did they rely on studies conducted in a scientifically acceptable and controlled manner that would validate the use of one test firing to attempt to identify the ammunition at issue." Lamagna Affidavit, ¶ 69. This court finds that Lydon's initial reports in the Meeks andWarner cases were lacking in specifics, and that they reflected the procedures in place in the Unit at that time. See LydonWarner Report, Exhibit 8; Lydon Meeks Report, Exhibit 9. Once Doherty implemented stricter procedures, including reviews of evidence and more documentation, the reports became more detailed. See Doherty Meeks Report, Exhibit 60; DohertyWarner Report, Exhibit 61. Moreover, the condition of the evidence did not change in the time between its recovery and the hearing in these cases. This court concludes that Lydon's and Doherty's most recent examinations — that resulted in the same conclusions as their initial examinations — are reliable.

Lamagna also stated that, although the photomicrograph in theMeeks case "showed some alignment of the coarser striae in a single land impression, not all the striae actually `match[,]' . . . [and the] comparison photomicrograph [did] not provide an accurate representation of the length, width, height and contour of the striae present on the land impression in question." Lamagna Affidavit, ¶ 5. Lamagna also opined that the BPD should have conducted more than one test firing, "preferably utilizing more than one test firearm of the same make and model. . . ."Id. at ¶ 71. With respect to the Warner case specifically, a spent .22 caliber cartridge case was recovered at the scene, but neither Lydon nor Doherty test fired that caliber of ammunition.

While this court agrees with Lamagna's position that photomicrographs are a necessary part of firearms examination because, "without the photomicrographs, the evidence is solely one [examiner's] subjective untested opinion[,]" id. at ¶ 34, this court disagrees that the photomicrographs in these cases were insufficient. Doherty testified that the photographs accurately — though not exactly — represented what she observed under the comparison microscope, and this court credits her testimony. Additionally, as noted above, the issues of the type of ammunition used in the test fires and the number of test fires taken are subjects appropriately reserved for cross-examination as they go to the weight of the conclusions rather than the reliability of the methodology underlying them.

Finally, as discussed in the Rulings of Law section, below, among the factors the Daubert Court set forth to assist in the consideration of whether the theory or technique the expert utilized to reach the conclusion is reliable, is whether that theory or technique is generally accepted in the relevant scientific or technical community. 509 U.S. at 594. "In the context of technical forensic evidence, the community must be sufficiently broad to permit the potential for dissent. . . . [i.e.,] to include `some practitioners who acknowledge flaws in the methodology' and tolerant enough to allow `some, albeit, limited room for dissent.'" Patterson, 445 Mass. at 643. Lamagna is arguably a firearms examination "practitioner" and he points out "flaws in the methodology" throughout his affidavit by challenging the firearms examination field. Moreover, also throughout his affidavit, Lamagna quotes from and cites to articles appearing in the AFTE Journal and articles used at AFTE training seminars as support for his testimony. See, e.g., Lamagna Affidavit, ¶¶ 24, 26, 35. The central principles of firearms examination are generally accepted in the field, and that community has been and continues to be sufficiently "tolerant" to consider criticisms and dissent.

RULINGS OF LAW

I. Standard of Review

In Daubert v. Merrell Dow Pharmaceuticals, Inc., the Supreme Court set forth five factors to aid the trial court in conducting its "preliminary assessment of whether the reasoning or methodology underlying the [expert scientific] testimony is sufficiently valid and of whether that reasoning or methodology properly can be applied to the facts in issue." 509 U.S. 579, 592-593 (1993); see Kumho Tire Co. v. Carmichael, 526 U.S. 137, 141, 149 (1999) (applying Daubert to testimony of "experts who are not scientists[,] . . . [including] testimony based on `technical' and `other specialized' knowledge"). These factors, which are not "a definitive checklist or test[,]" consider whether the theory or technique the expert utilized to reach the conclusion "can be (and has been) tested[;]. . . . whether the theory or technique has been subjected to peer review and publication[;]. . . . [what] the known or potential rate of error [is;] . . . [whether] standards controlling the technique's operation [exist and are maintained;]" and whether the theory or technique is generally accepted in the relevant scientific community. Daubert, 509 U.S. at 593-594. In making this determination as to the admissibility of the expert testimony, the court's focus "must be solely on principles and methodology, not on the conclusions that they generate." Id. at 595.¹⁵³

153 "[T]he gatekeeping inquiry must be `"tied to the facts"' of a particular `case.'" Kumho Tire Co., 526 U.S. at 150, quoting Daubert, 509 U.S. at 591. As a result, "a trial court . . . [must] make a reliability determination . . . where any proposed expert testimony's `factual basis, data, principles, methods, or their application are called sufficiently into question[.]" Risinger Saks, Exhibit 44, at 5. In making that reliability determination with respect to the expert testimony "`relevant to the task at hand[,]'" id. at 141, quotingDaubert, 509 U.S. at 597, "it would seem incumbent upon judges and lawyers to inform themselves concerning the status of knowledge bearing on such factors." Risinger Saks, Exhibit 44, at 5, quoting 526 U.S. at 149 (footnote omitted); see United States v. Mahone, 453 F.3d 68, 71, 72 (1st Cir. 2006) (quoting Fed.R.Evid. 702 for proposition that not only must "`the testimony [be] the product of reliable principles and methods[,]'" but also that "expert witness [must have] `applied the principles and methods reliably to the facts of the case'").
As Judge Gertner further explained,

"the Daubert Court noted the difference between information gleaned in a scientific setting and information presented in a courtroom. In the former, the decision makers are professionals; there is no need to come to a definitive conclusion; the decision making process comports with certain rules established by the professional scientific community. In the courtroom, the decision makers are lay, a jury; there is a need to come to a definitive conclusion; the decision making process has to satisfy norms of due process and fairness. In our tradition, for example, the adversary system and party examination and cross examination are central. The issue, then, is not only how objectively reliable the evidence is, but also the legitimacy of the process by which it is generated. It is not just how valid the data is, but how well the jury can understand it after direct and cross examination and legal instructions."

United States v. Hines, 55 F. Supp. 2d 62, 65 (D. Mass. 1999).

In Commonwealth v. Lanigan, the Supreme Judicial Court ("SJC") held that while it "accept[ed] the basic reasoning of theDaubert opinion[,] . . . general acceptance in the relevant scientific community will continue to be the significant, and often the only, issue." 419 Mass. 15, 26 (1994). That notwithstanding, "a proponent of scientific opinion evidence may demonstrate the reliability or validity of the underlying scientific theory or process by some other means, that is, [by applying the other factors set forth in Daubert] without establishing general acceptance." Id. Therefore, "Lanigan's progeny make clear that general acceptance in the relevant community of the theory and process on which an expert testimony is based, on its own, continues to be sufficient to establish the requisite reliability for admission in Massachusetts courts regardless of other Daubert factors." Commonwealth v.Patterson, 445 Mass. 626, 640 (2005); see Palandjian v.Foster, 446 Mass. 100, 112 (2006) (noting "broad discretion that Lanigan and its progeny provide trial judges in assessing the reliability of expert testimony"). Still, however, "[w]here general acceptance is not established by the party offering the expert testimony, a full Daubert analysis provides an alternate method of establishing reliability." Patterson, 445 Mass. at 641; see Canavan's Case, 432 Mass. 304, 313 (2000) (followingKumho Tire Co. and applying Daubert-Lanigan analysis to "expert testimony based on, inter alia, personal observations and clinical experience"). "A judge may also `look to his own common sense, as well as the depth and quality of the proffered expert's education, training, experience, and appearance in other courts' to determine reliability." Commonwealth v. Pasteur, 66 Mass. App. Ct. 812, 826, review denied, 447 Mass. 1109 (2006),¹⁵⁴ quoting Commonwealth v. Goodman, 54 Mass. App. Ct. 385, 391 (2002).

154 Pasteur appears to be the most recent Massachusetts appellate decision concerning the reliability of expert testimony on ballistics evidence. The Appeals Court affirmed the trial court's admission of such testimony, although the focus of the expert's testimony was the ricochet of a bullet. Pasteur, 66 Mass. App. Ct. at 827. The expert whose testimony the trial court admitted was a state police sergeant who "testified regarding his methodology, which essentially relied on examination of a bullet's shape and substances, if any, on its surface. That information would suggest with what kinds of materials the bullet had come into contact and, from that, judgments could be made with respect to the bullet's likely path of travel" Id. The court found it notable that "[h]e testified that the seven other qualified investigators in his unit used the same method, and that he had testified previously in Massachusetts regarding ricochet on the basis of that method." Id. Ultimately, however, the trial judge's admission of the expert's testimony as reliable "was to a great extent academic because [the expert] was unable to give an opinion on the bullet's trajectory that was meaningful to the case." Id.

In this case, the Commonwealth seeks to admit firearms examination testimony concerning the conclusion that the recovered ballistics evidence in both defendants' cases was identified as having been fired from the particular firearm recovered in each case. The defendants filed motions in limine to exclude this firearms identification evidence, contending that the Commonwealth failed to demonstrate that firearms identification evidence is reliable and, consequently, admissible. See Palandjian, 446 Mass. at 112 n. 17 ("[T]he burden is on the proponent of expert testimony to demonstrate its reliability, not on the opposing party to refute it."). As inPatterson, it is immaterial whether firearms examiners are classified as scientists or technicians "[g]iven that Lanigan applies to technical evidence as well as scientific evidence. . . ." 445 Mass. at 642, citing Canavan's Case, 432 Mass. at 313. Analysis under Daubert-Lanigan is therefore proper to resolve this dispute.

II. General Acceptance

As noted above, "general acceptance in the relevant community of the theory and process on which an expert's testimony is based, on its own, continues to be sufficient to establish the requisite reliability for admission in Massachusetts courts regardless of the other Daubert factors." Id. at 640 (emphases added). "In the context of technical forensic evidence, the community must be sufficiently broad to permit the potential for dissent. . . . [i.e.,] to include `some practitioners who acknowledge flaws in the methodology' and tolerant enough to allow `some, albeit, limited room for dissent.'" Id. at 643. For example, in Patterson, the SJC held that the "fingerprint examiner community consists primarily of fingerprint examiners from local, State, Federal, and foreign law enforcement agencies as well as independent or retired examiners. . . . Also included are scientists from other fields . . . who study the underlying premises of fingerprint examination." Id. at 641 n. 12. Additionally, "[t]he fingerprint community has formed a number of associations and professional groups better to share information and experience, and better to control the standards of their profession." Id.

The evidence in these cases suggests that the firearms examination community is similarly composed and it, too, has a voluntary professional group, the Association of Firearms and Toolmark Examiners, or AFTE. Striupaitis testified that the firearms examiners community numbers approximately 1,100 individuals, 900 of whom are AFTE members.¹⁵⁵ He further testified that the AFTE Theory of Identification requiring "sufficient agreement" among striations before an identification is found is generally accepted among the community of firearms and toolmark examiners. He knows of no examiner who disagrees with the AFTE Theory of Identification and the underlying principles of the field, although he acknowledged that firearms examiners engage in debates on a variety of issues concerning the field of firearms examinations. He based these assertions on the positions he has held in AFTE since 1981, including vice president and president, and on the conferences he has attended and the AFTE Journal articles he has read.

155 Lydon and Doherty are not "disqualified" as firearm examination experts based on the fact that they are not members of or certified by AFTE. See United States v. Mahone, 453 F.3d 68, 71 (1st Cir. 2006) (holding defendant's argument that forensic scientist's "qualifications are insufficient, simply because she is not qualified as a footwear examiner through the International Association for Identification . . . ha[d] no merit").

The hearing testimony of firearms examiners Lydon, Doherty, Mann, and Striupaitis demonstrate that the premise of firearms identification, that the firing of a firearm leaves markings on bullets and shell casings, is undisputed, and, accordingly, generally accepted. The defendants argue, however, that the methodology firearms examiners utilize in examining spent bullets and shell casings should not be deemed reliable merely because it has been traditionally considered to be so.

Both the hearing testimony and the hearing exhibits, however, not only establish that the methodology employed in firearms examinations is generally accepted by firearms examiners, but also support the underlying premise, that firing a firearm leaves marks that can identify or eliminate the spent shell casing and/or bullet as having been fired from a particular firearm. For example, at a panel discussion at the 2001 AFTE conference, firearms examiners discussed the topic of toolmark criteria for identification. See "AFTE 2001 Training Seminar Tool Mark Criteria for Identification Panel Discussion," 34 AFTE Journal 323, 323 (Summer 2002) ("AFTE Panel Discussion, Exhibit 84"). The discussion focused primarily on CMS, and an audience member commented that "CMS . . . as a pattern recognition system is powerful because we know that intuitively that these are random process[es] that all of a sudden become coincidences. . . . There's something going on here that's more than pure coincidence. . . . We have a coincidence and we have a quantity. This is a relationship of how we see something intuitively but also we are trying to get a quantity." Id. at 327.

In response, firearms examiner Bruce Moran ("Moran") raised Biasotti's 1959 publication. Id. In his article, "A Statistical Study of the Individual Characteristics of Fired Bullets," Biasotti concluded that "the chance occurrence of even a very small number of consecutive matching lines (e.g., more than 3 or 4) is for all practical purposes impossible except as the result of a common agent, e.g., same gun." Biasotti 1959, Exhibit 82, at 44 (emphasis added); see AFTE Panel Discussion, Exhibit 84, at 327 ("[Biasotti] observed that the coincidence of getting 3-4 CMS in a known non-match essentially stopped occurring."). Moran expanded on this reference, commenting that "in a known match large runs and many runs of highly consecutive matching striae (3 and 4 and 5 and 6 runs etc) occurred. So far we [meaning the firearms examination community] have been unable to disprove that." Id. For almost half a century, therefore, the premise that matching striations in known matches will exceed those in known non-matches has not been disproved.

Even though this underlying premise has yet to be refuted, disagreements among examiners arise. See, e.g., Bruce Moran, "Photo Documentation of Toolmark Identifications — An Argument in Support," 35 AFTE Journal 174, 183 (Spring 2003) ("Moran Photos, Exhibit 86") (responding to "side issue that erupted out of the AFTE 2001 Toolmark Criteria for Identification Panel Discussion . . . on the subject of whether toolmark identifications should be photographed. . . . [and noting that] [t]here is obviously some dissention [sic] among AFTE members regarding this topic"). At the panel discussion on toolmark criteria for identification at the 2001 AFTE conference, firearms examiners described their use of CMS in their examinations. AFTE Panel Discussion, Exhibit 84, at 324-333. Specifically, Moran stated that he

"use[s] CMS in conjunction with his ability to recognize patterns. . . . [If he is] looking at something that is a very close call where the amount of agreement is marginal [he] can fall back on a method of deciphering between an ID and non-ID that is scientifically tested both theoretically and empirically. [He] believe[s] that this [method] will be much more reliable when explaining [his] basis for his conclusions in a court of law than statements that there is `more agreement than the best known non-match that [he] can remember' or that `[he] recognize[s] it when [he] see[s] it'. CMS will have great utility in more reliably deciphering between those two close calls."

Id. at 324-325 (emphasis added). Those examiners who "work in [Moran's] laboratory don't consciously quantitate like [he] do[es]." Id. at 326.

This disagreement, however, does not pertain to the underlying premise of firearms examination, as Moran points out in a later article for the AFTE Journal. Bruce Moran, "Comments and Clarification of Responses from a Member of the AFTE 2001 Criteria for Identification of Toolmarks Discussion Panel," 35 AFTE Journal 55, 55 (Winter 2003) ("Moran, Exhibit 85"). He wrote that his

"approach to toolmark ID is no different than any other traditional toolmark examiner. [He] . . . approach[es] the interpretation of evidence by employing a combination of [his] cognitive ability to recognize agreement between patterns that in [his] `minds [sic] eye' potentially constitutes an ID or `match' between a questioned toolmark and toolmarks produced from known tools."

Id. (emphasis added). "All toolmark examiners require a certain amount (quantity) of agreement to make this determination [that there is a match]. Therefore, [they] are all making a quantitative assessment of the toolmarks [they] examine. In other words, [they] are all counting, subconsciously." Id. (bold in original) (emphasis added). Rather than describe this agreement in the same subjective way as traditional toolmark examiners do, however, Moran "shift[s] to a critical evaluation of the candidate areas by tabulating the numerical extent of consecutiveness within the striated toolmark pattern and referencing it to a conservative numerical threshold. . . ."Id. at 55-56. Moran, therefore, consciously counts the striations. Id. at 55.

While his "approach distances [him] from [the] traditional subjective `minds [sic] eye' decision making process in individualizing toolmarks. . . . [his] criteria is exactly the same as all toolmark examiners. . . . That is, when [examiners] see enough agreement that has been produced by a non-subclass bearing surface, [they] make a positive ID." Id. at 56. This disagreement therefore does not go to the underlying premise behind firearms identifications; rather, Moran's deviation from the traditional method of communicating the results of toolmark examinations is his response to his own concerns about the subjectivity involved in toolmark identification.

Another area of disagreement in the firearms examination community, which, again, does not concern the underlying premise of firearms examination, relates to the limitations of IBIS. As discussed above in the Findings section, Masson, in his 1997 article in the AFTE Journal titled "Confidence Level Variations in Firearms Identifications through Computerized Technology," acknowledges that IBIS has led him "to increased levels of identification confidence when comparing specimen candidates." Masson, Exhibit 75, at 42.¹⁵⁶ That notwithstanding, he further acknowledges the lists of possible matches that IBIS generates consist of "numerous non-identifications that have a considerable number of similarities and are highly ranked by the system." Id. Thus, as the database grows, there will be "a number of known non-matched testfires from different firearms that [will] com[e] up near the top of the candidate list." Id.

156 Masson's article deals solely with projectiles.

This recognition of IBIS' limitations, however, merely serves as further support for the reliability of the field of firearms examination. Prior to IBIS, "best examples of known non-matched agreement were collected from casework and thus, surfaced sporadically." Id. at 43. With IBIS, firearms examiners are able to "take advantage of this current expanded database to fully familiarize themselves with the extent of similarities found in many non-identifications [i.e., known non-matches] in order to hone their criteria for striae identification." Id. In fact, Masson expressly states that IBIS' ever-increasing database "opens a broad field for the underlying principles and methods of firearms identification." Id.

In his 2001 article, "Technical Evaluation: Feasibility of a Ballistics Imaging Database for All New Handgun Sales," Frederic A. Tulleners ("Tulleners"), the Laboratory Director of the Sacramento and Santa Rosa Criminalistics Laboratories, described the performance tests he conducted on an IBIS-like database comprised solely of cartridge cases. Tulleners Database, Exhibit 76, at 1-3. The tests "were designed to mimic what would happen when a database is substantially increased in size." Id. Among the results of Tulleners' tests was the finding that, as Tulleners increased the size of his database, the ranking of potential matches that, prior to the database's increase, had been ranked in the top ten, "would change (with one exception) to undetectable ranks." Id. at 1-4. Tulleners theorized that this change in rank accompanying the database growth could preclude an examiner from making identifications. See id. Manual review of "potential `hits' flagged for further inspection by computer correlation must [still] be confirmed by `hands on' microscopic examination by a qualified firearms examiner." Id. at 1-2. Tulleners therefore concluded that the "huge inventory of potential candidates . . . [that is] generated for manual review. . . . will be so large as to be impractical and will likely create logistic complications so great that they cannot be effectively addressed." Id. at 1-1.

Tulleners' conclusion is therefore essentially the same as Masson's: as a database, such as IBIS, grows, its usefulness is increasingly limited. Masson does, however, go one step further than Tulleners by considering the positive effect a large, growing database will have on the field of firearms examination as a whole. In pointing out these limitations, neither Tulleners nor Masson challenges the underlying premise of the field, nor do they place in doubt the conclusion that the methodology utilized in the field is generally accepted among the community of firearms examiners.

In her testimony, Schwartz did challenge the underlying premise of the field as well as the methodology utilized in firearms examinations. Schwartz is not a firearms examiner and has no training in the field of firearms examination. She is an academician who has read literature in the field and published her own articles criticizing the field. Schwartz believes that for the field of firearms examination to be reliable, examiners must have, in part, a database on which to rely. She also believes, however, that the number of striations on a spent projectile, for example, cannot be set out in a database because a single firearm will never leave exactly the same number of striations on all of the projectiles it fires.

The testimony of Lydon, Doherty, Mann, and Striupaitis overcomes Schwartz' challenge to the underlying premise of firearms examinations. Their testimony concerning the studies they conducted, albeit not blind, and their own training and experience demonstrate that examination of spent projectiles and cartridge cases under the comparison microscope enables them to view the striations and impressions, respectively, and to refer back to their own "internal database," as Striupaitis defined it, in order to reach a conclusion. Thus, the limitations of IBIS do not support Schwartz' view that a database is not feasible because IBIS is merely a step in the examination process; an examiner still must conduct a manual examination, thereby calling upon his internal database to reach a conclusion.

Lamagna's challenges to the field of firearms examination also do not preclude a finding of general acceptance. Although, unlike Schwartz, Lamagna has had some training and experience in firearms examination, this court does not consider him to be an experienced firearms examiner in the same fashion that Lydon, Doherty, and Striupaitis are and that Mann was. Lamagna maintains that the changes in the manufacturing processes of firearms from handtools to computer-controlled methods, such as CNC, result in more uniformity, meaning that the premise that no two firearms will leave the same marks on a fired shell casing and projectile is no longer valid. Borgio's testimony, however, renders this challenge moot with respect to the evidence in these cases. Borgio, an expert in the manufacturing processes of firearms, concluded that, based on his visual inspection of the firearms allegedly involved in the Meeks and Warner cases, the manufacturing issues Lamagna raises are not present on either firearm.

The ballistics evidence in this case is therefore not excluded as unreliable because "the reasoning or methodology underlying the [expert] testimony is" generally accepted and "properly can be applied to the facts in issue." Daubert, 509 U.S. at 592-593; see Patterson, 445 Mass. at 640 ("[G]eneral acceptance in the relevant community of the theory and process on which an expert's testimony is based, on its own, continues to be sufficient to establish the requisite reliability for admission in Massachusetts courts regardless of the other Daubert factors."). This decision does not foreclose defendants' challenges to the weight of the evidence at trial. As the Court in Daubert noted, "[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence." Id. at 596; see United States v. Mooney, 315 F.3d 54, 63 (1st Cir. 2002) ("[O]nce a trial judge determines the reliability of the proffered expert's methodology and the validity of his reasoning, the expert should be permitted to testify as to the inferences and conclusions he draws from it, and any flaws in his opinion may be exposed through cross-examination or competing expert testimony."); see also United States v. Hines, 55 F. Supp. 2d 62, 67 (D. Mass. 1999) ("[C]ross-examination and limiting instructions may be more effective in `technical fields' because they are more accessible to the jury than fields with the charisma of science.").¹⁵⁷

157 Cross-examination, contrary evidence and the burden of proof are critical at trial because there is some evidence that jurors unconsciously "overestimate the probative value of scientific evidence." Tom R. Tyler, "Viewing CSI and the Threshold of Guilt: Managing Truth and Justice in Reality and Fiction," 115 Yale L.J. 1050, 1068 (2006). Studies have revealed that there is "a consistent tendency to overestimate accuracy. . . . Jurors . . . overestimate . . . types of scientific evidence, ranging from the accuracy of lie detector evidence and the `repressed' memories of childhood abuse recalled by adults. This consistent pattern of overestimation suggests that people are not just bad at evaluating evidence but rather are motivated to see evidence as probative." Id. at 1070. "[I]t is also possible that the portrayal [by such television shows as "CSI"] of science as the ultimate crime-fighting tool actually encourages the already existing overbelief in the value of flawed scientific findings that jurors confront in actual trials." Id. at 1071.
Tyler concludes that "it is the credibility of science that is crucial, because jurors seek a form of justification that is plausible and compelling to bolster their own desire for certainty." Id. The Daubert-Lanigan analysis is the essential first step, and the crucible of trial, the second step.

III. Other Daubert Factors

Although the court's conclusion that general acceptance in the relevant scientific community supports admission of the Commonwealth's ballistics evidence is sufficient for purposes of this reliability analysis, see Patterson, 445 Mass. at 640, the other factors the Daubert Court set forth further support that admission. As noted above, the other four factors Daubert andLanigan set forth as guidance for trial judges include whether the theory or technique the expert utilized "can be (and has been) tested[;]. . . . whether the theory or technique has been subjected to peer review and publication[;]. . . . [what] the known or potential rate of error [is;] . . . [and whether] standards controlling the technique's operation [exist and are maintained.]" Daubert, 509 U.S. at 593-594.

A. Testing

"[A] key question to be answered in determining whether a theory or technique is scientific [or technical] knowledge that will assist the trier of fact will be whether it can be (and has been) tested. `Scientific methodology today is based on generating hypotheses and testing them to see if they can be falsified; indeed this methodology is what distinguishes science from other fields of human inquiry.'" Daubert, 509 U.S. at 592 (citation omitted). The hearing testimony of Lydon, Doherty, Mann, and Striupaitis establishes that firearms examiners test the premises underlying the field of firearms examinations on a daily basis in their training and case work by examining evidence under the comparison microscope. Their training establishes their "internal database" of best known non-matches and through their casework, they draw on that internal database to reach conclusions regarding the evidence before them.

Documentation of these examinations must be sufficient to permit the subsequent reproducibility of the examination. As demonstrated in the Meeks and Warner cases, Lydon and Doherty produced better documentation as the cases progressed. Although these more detailed notes and reports still contain omissions, they reflect an attempt to accurately describe the examination processes. That detail permits subsequent reproducibility and opinion testimony which is understandable to the jury and subject to meaningful cross-examination. Moreover, Doherty uses CMS as a "means of communication" after using pattern matching to reach her conclusions; her reports reflect this usage, thereby providing an additional method for a subsequent examiner to reproduce and test her examination. Contrast Patterson, 445 Mass. at 650 (noting concern that subjectivity involved in the fingerprint identification process "means the process itself defies easy testing").

There has also been testing of the methodology underlying the field of firearms examination beyond this in-lab case-specific testing that occurs on a daily basis. For example, Doherty conducted a test of consecutively serialed BPD-issued Glock firearms by test firing each at least 850 times, and she compiled a database that contained the cartridge cases for test fire #1 and test fire #850 for each firearm. She then compared each pair of test fires, finding that, as to each firearm, she was always able to match the markings on the cartridge case of test fire #1 to the markings on the cartridge case of test fire #850. Although this test was not blind, it tested the underlying theory of the field of firearms examination, that the markings impressed on shell casings upon firing are unique to that firearm such that they enable an examiner to identify a shell casing as having been fired by a particular firearm. Doherty's test also demonstrated that even though the firearms were consecutively serialed, and therefore manufactured close in time to each other, identification was still possible.

Striupaitis, too, testified about two studies he had conducted, the first of which was similar to Doherty's test of the Glock firearms. Striupaitis conducted a blind "validation study" for which he obtained six firearms with consecutively rifled barrels from two different manufacturers. He fired each of the twelve firearms two times, entering the twenty-four spent bullets into IBIS. He then distributed the test fires among the volunteers, a group of twenty-to-thirty examiners, giving each examiner six known bullets, numbered one through six, and six unknown bullets that they had to match to the known bullets. The examiners matched the unknown bullets to the known bullets without any misidentifications. This result not only confirms the underlying theory of firearms examination, but also demonstrates, as does Doherty's test above, that even where the barrels of firearms are consecutively rifled, examiners are still able to conduct a successful examination, identifying two bullets as having been fired from the same firearm.

The second study Striupaitis conducted was a "wear study" of consecutively fired weapons in order to determine if multiple firings would affect an examiner's ability to examine individual characteristics such that the examiner would be able to identify the bullets and/or shell casings as having been fired by a particular firearm. He fired a .25 caliber semi-automatic Raven 501 times, and examined each of the 501 spent bullets and each of the 501 spent shell casings. Although he found there were changes over the 501 firings, Striupaitis did not observe any discernible differences in the individual characteristics that would preclude an identification.¹⁵⁸ This test, although not blind, therefore confirmed for Striupaitis the underlying theory of firearms examination, that an examination of the individual characteristics enables an examiner to identify or eliminate a bullet or shell casing as having been fired from a specific firearm, regardless of the number of times a firearm has been fired.¹⁵⁹

158 Tulleners described a similar "Longevity Study" in which he test fired two firearms six hundred times each "to determine the effect of multiple test firings on the persistence of cartridge case impression signatures." Tulleners Database, Exhibit 76, at 1-5. He found that "[t]here [was] some indication of signature degradation as one compare[d] test #600 to test #1, but no definitive conclusions could be made." Id.

159 As noted in the Findings section, above, Striupaitis testified that while a degree of wear and/or damage could prevent identification, that situation would result in a false negative rather than a false positive.

The combination of the case-specific examinations that examiners conduct on a daily basis and consider to be a continuation of their training, with the tests about which Doherty and Striupaitis testified, all of which confirm the premise of firearms examination, weighs in favor of a finding of reliability of the ballistics evidence in these cases. The fact that the evidence in these cases exists and remains available for further testing contributes to this finding as well.¹⁶⁰

160 Although the facts in this case are distinguishable from those in Commonwealth v. Dinkins, 440 Mass. 715 (2004), the SJC's comments on the reliability of ballistics evidence are consistent with those of this court. In Dinkins, the defendant sought to suppress testimony and ballistics reports because the BPD had destroyed "the gun and some of the ballistics evidence" during the nine years the murder which the defendant was charged with and ultimately convicted of remained unsolved. Id. at 716, 717. The defendant argued "that the unavailability of the gun and the other ballistics evidence precluded him from conducting independent tests and thereby developing exculpatory evidence."Id. The defendant's motion was denied because "[i]t is not enough for a defendant merely to argue, as [the defendant] did . . ., that, with access to the evidence, his expert `could have made potentially exculpatory findings'. . . . [because] `could have' is merely an introduction to speculation and is not a substitute for `concrete evidence.'" Id. at 718.
The Court held, in part, that the potential prejudice to the defendant as a result of the destroyed gun and ballistics evidence was "low" because the defendant "was able to use the absence of the gun to discredit the ballistician's testimony, and in turn to create doubt about the Commonwealth's case." Id. at 719. Through cross examination, [the defendant's] attorney demonstrated that the ballistics expert who matched the gun to the shootings could not remember the particular characteristics on which he based the match, and could not prove that, as required by procedure, he secured a second opinion to verify his determination of the match." Id. The Court pointed out that

"[t]he opportunity to plant these seeds of doubt in the minds of the jury was undoubtedly a great deal more than [the defendant] would have accomplished had the evidence been available for further testing. Although it may be possible for two ballisticians to analyze the same evidence and come to different conclusions, . . . the recognized reliability of ballistics analysis and the Commonwealth's expert's unchallenged assertion of the continuity in ballistics analysis methods over time since the tests were performed make it highly likely that additional tests would have mirrored the results of those already done and thereby bolstered the Commonwealth's case."

Id. at 719-720 (internal citation omitted), citing Joseph L. Peterson Penelope N. Markham, "Crime Laboratory Proficiency Testing Results, 1978-1991, II: Resolving Questions of Common Origin," 40 J. Forensic Sci., 1009, 1018-1019, 1028 (1995).

B. Error Rate

In their chapter titled "Scientific Issues" contained within volume 4 of Modern Scientific Evidence: The Law and Science of Expert Testimony, by David L. Faigman, et al., Biasotti and Murdock declare that "the field [of firearms and toolmark examination] is in a poor position to calculate error rates." Biasotti Murdock, Exhibit 83A, at 414. But see Commonwealth v. Teixeira, 40 Mass. App. Ct. 236, 239 (1996) (noting that "infallibility would be remarkable in any laboratory"). Proficiency testing from Collaborative Testing Services, Inc., or CTS, is the only ASCLD-approved testing service for firearms examiners. As quoted in full above, however, the CTS tests expressly state that "the results compiled in the Summary Report [of the test results] are not intended to be an overview of the quality of work performed in the profession and cannot be interpreted as such. The Summary Comments . . . are not intended to reflect the general state of the art within the profession." 2003 CTS Test, Exhibit 79; 2005 CTS Test, Exhibit 80. The test-takers have the option of using the test for purposes of a "training exercise, known or blind proficiency testing, research and development of new techniques, etc." Id.

The BPD has used CTS proficiency testing since 2003. Striupaitis takes a CTS proficiency test each year even though he recognizes the limitations of this test: first, there is no quality control performed on the test specimens sent out to the test-takers;¹⁶¹ and second, inconclusive answers are automatically marked wrong when in fact they could be the correct response. Moreover, notwithstanding the express disclaimer on the CTS tests, "[i]t would be more instructive if the crime laboratories completing the proficiency tests . . . were required to indicate if normal laboratory procedures were followed, whether this included technical peer review, and supervisorial scrutiny, whether the test was used as a test for a trainee, and so on." Biasotti Murdock, Exhibit 83A, at 415. "With this additional information, more meaningful comments could be made about these proficiency test results." Id. at 415-416. Furthermore, the CTS tests themselves are not blind, therefore the tests would carry more weight if they were "submitted to examiners as if they were part of the regular caseload — that is, blind proficiency testing." Id. at 416.

161 Biasotti and Murdock note this circumstance as one of the "inherent difficulties associated with the production of toolmark proficiency tests. Due to the nature of this evidence, each sample is unique. Since there are dynamic forces involved in producing the toolmark samples, there are opportunities for variations between samples." Biasotti Murdock, Exhibit 83A, at 416.

Limited proficiency testing therefore does exist, and Lydon, Doherty, and Striupaitis take CTS tests despite their awareness of the tests' limitations. Lydon testified that since completing his training, no other BPD examiner who has re-examined his work has, to Lydon's knowledge, disagreed with his conclusion, and on the four occasions that non-BPD examiners re-examined his work, his conclusions were upheld. Similarly, Striupaitis testified that he knows of no false positives he has made, although he recalled one occasion when he changed his conclusion from inconclusive to identification after a quality assurance process was conducted on the firearm and bullet he examined. Additionally, Mann testified that on two occasions, she questioned whether she had found a match, and on both of those occasions, a more experienced examiner found a match.

As noted above, the field of firearms examination is self-contained in that laboratories are concerned with their own examinations rather than those conducted elsewhere. SeePatterson, 445 Mass. at 652 ("[T]he concern is solely with the rate of false positive. . . . [because] `rate of false negatives is immaterial to the Daubert admissibility of . . . [evidence] offered to prove positive identification.'" (quoting United States v. Mitchell, 365 F.3d 215, 239 (3d Cir. 2004))). For these reasons, the current state of proficiency testing and its use in the Unit, although not ideal, is sufficient for its purposes and does not preclude a finding of reliability of the methodology underlying the field of firearms examination. See, e.g., Teixeira, 40 Mass. App. Ct. at 240 (holding that "[w]eakness in the laboratory's proficiency testing [in the context of DNA analysis conducted by the FBI] went to the weight to be ascribed to the evidence of match, not to its admissibility"); see also Biasotti Murdock, Exhibit 83A, at 414 (recognizing laboratory errors can be minimized through "the concept of known non-match comparisons, the thorough understanding of the influence of sub-class characteristics, and in-laboratory peer review by skilled co-workers").

Expert testimony need not be "unassailable" to be admissible. In many, if not all, of the physical sciences and the social sciences, error-free analysis cannot be achieved. Cf.Commonwealth v. Ruiz, 442 Mass. 826, 833-834 (2004) (emergency medical technician, difficulty breathing caused by chest injury); Commonwealth v. Rice, 441 Mass. 291, 297-299 (2004) (state police forensic chemist, changes to sperm cells over time); Commonwealth v. Frangipane, 433 Mass. 527, 531-535 (2001) (social worker, symptoms of sexual abuse);Commonwealth v. Simmons, 419 Mass. 426, 432-433 (1995) (Department of Public Safety chemist, blood pattern analysis);Commonwealth v. Dockham, 405 Mass. 618, 628-630 (1989) (child psychiatrist, attributes of sexually abused children). Malcolm Gladwell ("Gladwell") has pointed out that mammography is not rooted in statistical and medical certainties. Malcolm Gladwell, "The Picture Problem: Mammography, Air Power, and the Limits of Looking," The New Yorker, Dec. 13, 2004. Gladwell asserts that "mammograms do not have to be infallible to save lives[,]" and that mammography "reduces the risk of dying from breast cancer by about ten percent. . . ." Id. He concludes that "[m]ammography isn't as good as we'd like it to be. But we are still better off than we would be without it." Id.

Mammography and toolmark identification examinations are most concerned about the risk of different types of errors: mammography with the risk of a false negative, and toolmark identification with the risk of a false positive. It is, of course, true that those who practice mammography are highly trained individuals. Nevertheless, it appears undeniable that mammography relies upon subjective judgment to a considerable degree. See Joann G. Elmore, et al., "Variability in Radiologists' Interpretations of Mammograms," 331 New Eng. J. Med. 1493, 1499 (Dec. 1, 1994). This court concludes that based on all of the evidence in these cases, and despite the legitimate concerns as to a false positive, the ballistics evidence at issue is sufficiently reliable to come before the jury.

C. Objective Standards

Another factor Daubert suggests a court should consider "in the case of a particular scientific [or technical] technique . . . [is] the existence and maintenance of standards controlling the technique's operation." 509 U.S. at 594. "A wide variety of experts whose testimony is generally admitted at trial use their education, training, and knowledge to opine on matters about which there does not exist an objective standard." Patterson, 445 Mass. at 653. "In such instances, `the expert is operating within a vocational framework that may have numerous objective components, but the expert's ultimate opining is likely to depend in some measure on experiential factors that transcend precise measurement and quantification.'" Id. (emphasis added), quoting United States v. Llera Plaza, 188 F. Supp. 2d 549, 571 (E.D. Pa. 2002).

The hearing testimony of the firearms examiners in these cases uniformly indicates that throughout their training period, they examine known non-matches and known matches under the comparison microscope. Their testimony is further uniform with respect to the techniques they employ in examining shell casings and bullets. Without variation, Lydon, Doherty, and Mann testified that, for example with bullets, they first attempt to find an elimination using class characteristics; second, they move on to individual characteristics, when the class characteristics match, by engaging in pattern matching. Pattern matching involves looking at the quality or appearance of the striae, such as their contour, on areas of significance. Striupaitis further testified that these objective observations lead to a subjective final conclusion. See AFTE Theory, Exhibit 28, at 86 ("[T]he interpretation of the . . . identification is subjective in nature, founded on scientific principles and based on the examiner's training and experience."); Biasotti Murdock, Exhibit 83A, § 36:9, at 413 ("[T]he interpretation that forms the basis for these conclusions is subjective.").

In an attempt to answer "the question of how much agreement is needed" to find an identification, the AFTE Theory of Identification "gives a nonquantitative answer. . . ." Biasotti Murdock, Exhibit 83A, § 39:9, at 411. There must be "sufficient agreement" for an examiner to find an examination; this term "is related to the significant duplication of random toolmarks as evidenced by the correspondence of a pattern or combination of patterns of surface contours." AFTE Theory, Exhibit 28, at 86. While the AFTE Theory of Identification does not set forth entirely "objective standards" such that, as Schwartz believes, the examiner will know a match when he/she sees it, it does provide a standard for examiners to follow. While that standard is clearly not a straightforward litmus test, it is nonetheless one with a "`restricted compass'" that is "bounded by objective factors." Patterson, 445 Mass. at 653, quoting in part Llera Plaza, 188 F. Supp. at 570, citing Maryland Cas. Co. v.Therm-O-Disc, 137 F.3d 780 (4th Cir. 1998). Further, the examiners' individual empirical studies conducted during their training period, combined with their post-training case work, contribute to their "internal database" of the best known non-matches.

The considerable subjective judgment inherent in the AFTE standard is more troublesome, and correspondingly less helpful to the jury, when the opinion testimony given on direct examination is little more than a fiat. In the experience of this court, ballistics evidence is often a bare opinion with minimal explanation and no documentation. The jury is asked, in effect, to trust the expert and his "scientific" conclusions. Such testimony may have "the aura of mathematical certainty or infallibility and thus the potential to overawe jurors." Ned Miltenberg, "Out of the Fire and into the Fryeing Pan or Back to the Future," 37 Trial 18 (March 2001), citing Michael H. Graham, "The Expert Witness Predicament," 54 U. Miami L.Rev. 317, 331 (2000). The California Supreme Court has expressed concern about techniques which appear to provide "some definitive truth which the expert need only accurately recognize and relay to the jury. The most obvious examples are machines or procedures which analyze physical data. Lay minds might easily, but erroneously, assume that such procedures are objective and infallible."People v. Stoll, 783 P.2d 698, 710 (Cal. 1989); see alsoCanavan's Case, 432 Mass. at 317 (Greaney, J., concurring) (noting that appellate courts will examine carefully "new hard science").

Although this court concludes that firearms identification evidence is based upon reliable principles which have been reliably applied in these cases, some restriction on the presentation of that evidence at trial is appropriate. InDepartment of Youth Servs. v. A Juvenile, 398 Mass. 516 (1986), the Supreme Judicial Court adopted Proposed Rule of Evidence 705. That rule permits an expert to testify as to an opinion and the reasons therefor "`without prior disclosure of the underlying facts or data, unless the court requires otherwise.'" Id. at 532, quoting Proposed Mass. R. Evid. 705. The Court emphasized that "`it is within the court's discretion to require an initial disclosure of underlying facts or data.'"Id., quoting The Advisory Committee's Note on Proposed Rule 705.

At trial, this court will require the Commonwealth's firearms identification witnesses to provide, on direct examination, detailed reasons for their opinions, as well as appropriate documentation such as sketches and photographs.¹⁶² In these cases, that information has already been provided to the defendants in discovery and as a result of this Lanigan hearing. Such discovery is essential to an effective cross-examination and therefore to a fair trial.

162 In his article "Photo Documentation of Toolmark Identifications — An Argument in Support," Moran writes,

"It is the experts' obligation to provide a thorough interpretation of the meaning of the agreement viewed by court and jury in the photograph(s) being offered, using the training and experience of the examiner. Pointing out areas of significant repetitive agreement supporting the identification and explaining the numerous reasons for expecting areas of disagreement in a match comparison should be routine for any well trained toolmark examiner. That is why [they] are experts. A good photograph goes a long way to instilling jury confidence in the expert's opinion."

Moran Photos, Exhibit 86, at 183.

This court has considered the thoughtful opinion of Judge Gertner in United States v. Green, Crim. No. 02-10301-NG (D. Mass. Dec. 20, 2005). In that case, Judge Gertner permitted the firearms examiner to testify as to the similarities and dissimilarities between the test fires and the evidence, but not as to the examiner's opinion as to whether there is a match. This court does not follow that approach because it concludes that ultimately the distinction is artificial and unworkable. A jury, like a party involved with or affected by the real world of a scientific or technical field, is entitled to the judgment and opinion of an expert in that field as to the critical questions: do you have an opinion, and if so, what is that opinion. The examinations of a witness by capable counsel will in all events move inexorably to those questions. Further, the fundamental rationale of expert opinion assistance to the trier of fact — would not be served fully if the expert were precluded from responding to those questions.

This objective standards factor, and its application to the facts in these cases, weighs in favor of a finding of reliability of the ballistics evidence.

D. Peer Review and Publication

Publication "is but one element of peer review. . . ."Daubert, 509 U.S. at 594. "[S]ubmission to the scrutiny of the scientific community [in a publication] is a component of `good science,' in part because it increases the likelihood that substantive flaws in the methodology will be detected." Id. "The fact of publication . . . in a peer reviewed journal[, however,] . . . [is] a relevant, though not dispositive consideration in assessing the scientific validity of a particular technique or methodology on which an opinion is premised." Id. at 594.

AFTE publishes a seasonal peer-reviewed journal, AFTE Journal, that contains articles concerning the field of firearms and toolmark examination. As discussed above, AFTE holds conferences at which panel discussions take place; these discussions may thereafter provide content for articles responding to or further explaining the discussions. See AFTE Panel Discussion, Exhibit 84; Moran, Exhibit 85, at 55. Lydon, Doherty, and Striupaitis testified that they regularly read articles in the AFTE Journal, and Doherty testified that she requires the firearms examiners in the Unit to "sign-off" after they have read the AFTE Journal articles she randomly chooses for them to read.

As noted above, the peer review in the field of firearms examination occurs on a lab-specific basis. Lydon, Doherty, Mann, and Striupaitis all testified to utilizing similar methods of peer review with respect to their examinations. Specifically, they testified that they seek, or, in Mann's case, sought, verification of their conclusion from another firearms examiner after finding a match. This process raises the possibility of confirmation bias, but Lydon and Doherty testified that they do not take into consideration extraneous factors (e.g., the knowledge that the previous examiner found a match) when examining evidence under the comparison microscope, they merely look at the evidence. Striupaitis testified that he takes this verification process one step further, seeking verification when he has found the two pieces of evidence have the same class characteristics but reaches the conclusion of elimination.

In the context of footprint analysis, the court in United States v. Mahone held that the methodology utilized was reliable even where the verification was not blind; verification that is not "blinded . . . still constitutes `peer review' that favors admission of the method[.]" 453 F.3d 68, 72 (1st Cir. 2006), citing United States v. Mitchell, 365 F.3d 215, 239 (3d Cir. 2004). Moreover, as noted above with respect to proficiency testing, "`the logistics of constructing fully blind proficiency tests are formidable'" and unrealistic.Commonwealth v. Vao Sok, 425 Mass. 787, 795 n. 13 (1997). Striupaitis' additional measure approaches blind testing in that the verifying examiner does not know if the examiner found a match or elimination. While this method is preferable to that the Unit currently employs, the absence of this method does not negate the effectiveness of the peer review procedure already in place.

Peer review and publications exist in the field of firearms examination and were applied in the Unit, specifically with respect to these cases, thereby weighing in favor of a finding of reliability. See Biasotti Murdock, Exhibit 83A, at 414 (recognizing "in-laboratory peer review by skilled co-workers" as one method to hold errors "to an absolute minimum").

CONCLUSION

The theory and process of firearms identification are generally accepted and reliable, and the process has been reliably applied in these cases. Accordingly, the firearms identification evidence, including opinions as to matches, may be presented to the juries for their consideration, but only if that evidence includes a detailed statement of the reasons for those opinions together with appropriate documentation.