Ex Parte Nempont et alDownload PDFPatent Trial and Appeal BoardNov 8, 201714438229 (P.T.A.B. Nov. 8, 2017) Copy Citation United States Patent and Trademark Office UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O.Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 14/438,229 04/24/2015 Olivier Pierre Nempont 2012P01086WOUS 5692 24737 7590 11/13/2017 PTTTT TPS TNTFT T FfTTTAT PROPFRTY fr STANDARDS EXAMINER 465 Columbus Avenue CHOI, TIMOTHY WING HO Suite 340 Valhalla, NY 10595 ART UNIT PAPER NUMBER 2665 NOTIFICATION DATE DELIVERY MODE 11/13/2017 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): patti. demichele @ Philips, com marianne. fox @ philips, com katelyn.mulroy @philips .com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte OLIVIER PIERRE NEMPONT, PASCAL YVES FRANCOIS CATHIER, and RAOUL FLORENT Appeal 2017-005914 Application 14/438,229 Technology Center 2600 Before JAMES R. HUGHES, ERIC S. FRAHM, and STEVEN M. AMUNDSON, Administrative Patent Judges. AMUNDSON, Administrative Patent Judge. DECISION ON APPEAL Appellants1 seek our review under 35 U.S.C. § 134(a) from a final rejection of claims 1,2, 4—6, 8, 9, and 11—17, i.e., all pending claims. We have jurisdiction under 35 U.S.C. § 6(b). We affirm. 1 Appellants identify the real party in interest as Koninklijke Philips N.V. App. Br. 2. Appeal 2017-005914 Application 14/438,229 STATEMENT OF THE CASE The Invention According to the Specification, the invention “relates to an image processing device for enhancing ultrasound images, to a medical imaging system, and to a method for enhancing ultrasound images, as well as to a computer program element and to a computer-readable medium.” Spec. 1:7—9.2 The Specification explains that (1) an “image data input unit is configured to provide an ultrasound image of a region of interest of an object, and to provide an X-ray image of the region of interest of the object”; (2) a “central processing unit is configured to select a predetermined image area in the X-ray image, to register the ultrasound image and the X-ray image, to detect the predetermined area in the ultrasound image based on the registered selected predetermined image area, and to highlight at least a part of the detected area in the ultrasound image to generate a boosted ultrasound image”; and (3) a “display unit is configured to provide the boosted ultrasound image as guiding information on a display area.” Abstract. Exemplary Claim Independent claim 1 exemplifies the claims at issue and reads as follows: 1. An image processing device for enhancing ultrasound images, comprising: an image data input unit; 2 This decision uses the following abbreviations: “Spec.” for the Specification, filed April 24, 2015; “Final Act.” for the Final Office Action, mailed April 22, 2016; “Adv. Act.” for the Advisory Action, mailed July 8, 2016; “App. Br.” for the Appeal Brief, filed September 7, 2016; “Ans.” for the Examiner’s Answer, mailed December 30, 2016; and “Reply Br.” for the Reply Brief, filed February 21, 2017. 2 Appeal 2017-005914 Application 14/438,229 a central processing unit; and a display unit; wherein the image data input unit is configured to provide an ultrasound image of a region of interest of an object; and to provide an X-ray image of the region of interest of the object; wherein the central processing unit is configured to select at least one target object in the X-ray image; to register the ultrasound image and the X-ray image; to detect the at least one target object in the ultrasound image based on the registration; and to generate a boosted ultrasound image in which at least an area of the detected target object in the ultrasound image is highlighted; and wherein the display unit is configured to provide the boosted ultrasound image as guiding information on a display area, and wherein the x-ray image is generated by an x-ray imaging device, the ultrasound image is generated by an ultrasound imaging device, and the X-ray imaging device and the ultrasound imaging device are configured to acquire the images simultaneously. App. Br. 13 (Claims App.). The Prior Art Supporting the Rejections on Appeal As evidence of unpatentability, the Examiner relies on the following prior art: Thornton US 6,549,802 B2 Apr. 15,2003 Gerard et al. (“Gerard”) US 2007/0276243 A1 Nov. 29, 2007 Pencilla Lang et al., US—Fluoroscopy Registration for Transcatheter Aortic Valve Implantation, 59 IEEE Transactions on Biomedical Engineering 1444—53 (May 2012) (“Lang”) 3 Appeal 2017-005914 Application 14/438,229 The Rejections on Appeal Claims 1, 2, 4, 5, 8, 9, and 11—16 stand rejected under 35 U.S.C. § 103(a) as unpatentable over Thornton and Gerard. Final Act. 8—14. Claims 6 and 17 stand rejected under 35 U.S.C. § 103(a) as unpatentable over Thornton, Gerard, and Lang. Final Act. 14—16. ANALYSIS We have reviewed the rejections of claims 1, 2, 4—6, 8, 9, and 11—17 in light of Appellants’ arguments that the Examiner erred. For the reasons explained below, we disagree with Appellants’ assertions regarding Examiner error. We adopt the Examiner’s findings and reasoning in the Final Office Action (Final Act. 4—5, 8—16), Advisory Action (Adv. Act. 2), and Answer (Ans. 3—7). We add the following to address and emphasize specific findings and arguments. The § 103(a) Rejection of Claims 1, 2, 4, 5, 8, 9, and 11—16 Simultaneously Acquiring X-Ray Images and Ultrasound Images Appellants argue that the Examiner erred in rejecting independent claims 1, 9, 14, and 15 because the references do not disclose or suggest the following limitation in claim 1 and similar limitations in the other independent claims: “the X-ray imaging device and the ultrasound imaging device are configured to acquire the images simultaneously.” See App. Br. 9—10; Reply Br. 5—7. Appellants state that the Examiner finds that Gerard discloses simultaneous image acquisition “because Gerard generates an ultrasound image and an x-ray image in real time.” App. Br. 9; see Reply Br. 5. Appellants assert, however, that “just because each image is processed in real time (i.e. as the data is acquired) does not mean that they 4 Appeal 2017-005914 Application 14/438,229 are acquired simultaneously.” App. Br. 9. According to Appellants, “Gerard only discloses that neither the ultrasound data not [sic] the x-ray data is acquired from [a] previous recording, such as preoperative x-rays, for example.” Reply Br. 5. Citing Gerard paragraph 55, Appellants assert that in a first embodiment, “a viewing angle is selected in the ultrasound image, then the viewing angle is applied to the C-arm system (x-ray device) prior to generating an x-ray image.” App. Br. 9 (citing Gerard 55); Reply Br. 6. Based on paragraph 55, Appellants contend that ultrasound imaging occurs before X-ray imaging. App. Br. 9; Reply Br. 6. Citing Gerard paragraph 51, Appellants assert that in a second embodiment, “there is a priori no need to displace the ultrasound probe during the clinical procedure, because ultrasound data are intended to be used once the medical instrument has been guided in a cavity of the patient body to be investigated.” App. Br. 9—10 (quoting Gerard 151); Reply Br. 6. Based on paragraph 51, Appellants contend that X-ray imaging occurs before ultrasound imaging. App. Br. 10; Reply Br. 6. Appellants’ arguments do not persuade us of Examiner error because the Examiner finds that: (1) Gerard discloses acquiring X-ray images and ultrasound images in “real time”; and (2) “real time” acquisition teaches or suggests “to one of ordinary skill that the ultrasound and x-ray images are acquired simultaneously to be registered in real-time.” Final Act. 10 (citing Gerard 128); see Adv. Act. 2 (citing Gerard 128). The Examiner reasons that “[o]ne of ordinary skill. . . would understand that performing an action in ‘real-time’ to be performing that action virtually instantly and perceptually simultaneous.” Ans. 3. 5 Appeal 2017-005914 Application 14/438,229 Further, the Examiner finds that Gerard Figure 11 depicts X-ray imaging steps and ultrasound imaging steps “performed in parallel.” Ans. 4 (citing Gerard Tflf 77—83, Fig. 11); see Adv. Act. 2 (citing Gerard Tflf 77—83, Fig. 11). Those steps include: (1) localizing an ultrasound probe with respect to the X-ray acquisition system; (2) selecting a region of interest within an ultrasound image; (3) converting a localization relating to the region of interest within the ultrasound image to a localization with respect to the X-ray acquisition system; and (4) generating and displaying a bimodal representation of a medical instrument based on both X-ray images and ultrasound images. Gerard 80—83, Fig. 11. Those steps contemplate coordination between the two systems, e.g., localizing an ultrasound probe with respect to the X-ray acquisition system, and concurrent data collection. See id. Tffl 77—83, Fig. 11. Consequently, those steps teach or suggest simultaneously acquiring X-ray images and ultrasound images. See Adv. Act. 2; Ans. 4. Consistent with that, Gerard discloses an embodiment employing the “live” acquisition of X-ray images and ultrasound images during a procedure, e.g., to enable the “live” visualization of a medical instrument when guided inside a patient’s body and to compensate for patient movement. Gerard H 45, 73. For example, Gerard explains that patient movement may change the ultrasound probe’s position with respect to the X-ray acquisition system “and therefore induce errors when mapping the 2D X-Ray data with the 3D ultrasound data.” Id. ^ 73. To compensate for patient movement, ultrasound probe localization updates occur in “real time” based on manual user input or “automatically at regular time intervals.” Id. The repeated “live” acquisition of X-ray images and ultrasound images and 6 Appeal 2017-005914 Application 14/438,229 the subsequent processing to eliminate mapping errors during a procedure teaches or suggests simultaneous image acquisition. Also, the Examiner explains that paragraphs 51 and 55 do not conflict with simultaneously acquiring X-ray images and ultrasound images. Ans. 4—5; see Adv. Act. 2. The Examiner notes that paragraph 51 instructs that no need exists to move an ultrasound probe during a procedure because the probe can acquire ultrasound images “in a stationary position.” Ans. 4. Contrary to Appellants’ contention, paragraph 51 does not require X-ray imaging to always occur before ultrasound imaging, and it does not preclude simultaneous image acquisition. See Gerard 151. In addition, the Examiner notes that paragraph 55 addresses “processing to apply a preferred viewing angle” after already acquiring X-ray images and ultrasound images. Ans. 3^4; see Adv. Act. 2. Contrary to Appellants’ contention, paragraph 55 does not require ultrasound imaging to always occur before X-ray imaging, and it does not preclude simultaneous image acquisition. See Gerard 1 55. Detecting a “Target Object in the Ultrasound Image Based on the Registration” Appellants argue that the Examiner erred in rejecting independent claims 1, 9, 14, and 15 because the references do not disclose or suggest the following limitation in claim 1 and similar limitations in the other independent claims: “detect[ing] the at least one target object in the ultrasound image based on the registration.” See App. Br. 10; Reply Br. 7— 8. In particular, Appellants contend that “Thornton calculates a seed position using [a] maximization algorithm, then projects or displays the seeds onto the ultrasound image at the calculated positions.” App. Br. 10 7 Appeal 2017-005914 Application 14/438,229 (citing Thornton 7:8—36). In addition, Appellants contend that “mapping a seed location (point) from a fluoroscopic image space to an ultrasound image space” as disclosed in Thornton “is not detecting an object in an ultrasound image.” Reply Br. 8 (citing Thornton 7:32—36). Appellants’ arguments do not persuade us of Examiner error because, as the Examiner notes, claims are given their broadest reasonable interpretation consistent with the specification during examination proceedings. See Final Act. 4; Ans. 5; see also In re Hyatt, 211 F.3d 1367, 1372 (Fed. Cir. 2000). Here, Thornton discloses a system for positioning radiotherapy seeds relative to targeted tissue that includes a fluoroscopy (X-ray) imaging device and an ultrasound imaging device. Thornton 2:33— 37, 4:29-48, 4:63—66, 5:55—59, 8:17—21, Abstract, Fig. 1, Fig. 3; see Adv. Act. 2. Thornton Figure 4 depicts steps in a method for determining seed position relative to targeted tissue. Thornton 1:56—57, 4:49—60, 5:35—6:31, 6:52—7:40, Fig. 4; see Final Act. 9—10 (citing Thornton 5:35—6:30, 6:50—65, 7:5 40). Those steps include: (1) locating the seeds in fluoroscope images; (2) determining the seed positions in a 3D fluoroscope coordinate system; (3) mapping the seed positions in the 3D fluoroscope coordinate system to their respective positions in a 3D ultrasound coordinate system, e.g., using a mathematical transformation; (4) determining or calculating the seed positions in a 3D ultrasound image based on the mapping; and (5) displaying the seed positons in the 3D ultrasound image, e.g., “as transparent, colored cylinders.” Thornton 6:52—7:36, Fig. 4. The Examiner finds that mapping the seed positions from the 3D fluoroscope coordinate system to the 3D ultrasound coordinate system equates to “registering the ultrasound image and the fluoroscopic x-ray 8 Appeal 2017-005914 Application 14/438,229 images.” Ans. 5. That finding comports with the Specification’s discussion of registering. See, e.g., Spec. 2:7—10, 2:15—16, 3:19—21, 5:22—26, 6:4—6, 8:18—20. The Specification explains that “[t]he term ‘register’ refers to providing a positional or spatial relation between the X-ray image and the ultrasound image.” Spec. 2:15—16. The Specification similarly states that “[s]tep d) [in Figure 3] comprises registering at least one detected target object in the ultrasound image,” and “[t]hus, the target position information achieved from the X-ray image is transferred by a registration to the ultrasound image.” Id. at 8:18—20. Consistent with the Examiner’s finding, Lang describes a registration method employing a mathematical transformation. Lang 1445—46, Fig. 3. Further, the Examiner finds that determining or calculating the seed positions in a 3D ultrasound image based on Thornton’s mapping equates to “detecting] the at least one target object in the ultrasound image based on the registration.” Ans. 5. That finding comports with the Specification’s discussion of detecting. See, e.g., Spec. 2:7—10, 3:13—18, 4:1—8, 4:16—29, 5:22—26, 5:31—34, 7:34—8:4, 8:18—21. For example, the Specification explains that (1) X-ray image data is “used to detect in the ultrasound images target objects . . . whose detection could be very challenging otherwise” and (2) detecting interventional tools in ultrasound images “is facilitated by the position and shape priors [sic] that arise from the X-Ray detection and that are based on the registration of the two image modalities.” Id. at 4:22—27. Because the Examiner relies on Thornton’s mapping for registering images rather than detecting, Appellants’ contention that Thornton’s mapping “is not detecting” does not respond to the rejection. See Ans. 5; Reply Br. 8. 9 Appeal 2017-005914 Application 14/438,229 Appellants assert that “using registration to facilitate detection is not necessary” in Thornton because “Thornton specifically states that the seed[s] are visible in the ultrasound image.” Reply Br. 8 (citing Thornton 5:52—54). But Thornton’s system enhances seed appearance when displayed on a monitor viewed by a user, e.g., by showing the seeds “as transparent, colored cylinders.” Thornton 7:32—39. Along similar lines, the Specification explains that (1) the term “boosting” according to the invention refers to improving target-object visibility for a user and (2) “boosting may be obtained by enhancing, for example colorizing, target objects . . . .” Spec. 2:11-13,4:5-7. Summary for Independent Claims 1,9,14, and 15 For the reasons discussed above, Appellants’ arguments have not persuaded us that the Examiner erred in rejecting claims 1,9, 14, and 15 for obviousness based on Thornton and Gerard. Hence, we sustain the § 103(a) rejection of claims 1,9, 14, and 15. Dependent Claims 2,4,5,8,12, and 13 Claims 2, 4, 5, and 8 depend from claim 1, while claims 12 and 13 depend from claim 9. App. Br. 13—16 (Claims App.). Appellants do not argue patentability separately for dependent claims 2, 4, 5, 8, 12, and 13. App. Br. 9—11; Reply Br. 5—8. Because Appellants do not argue the claims separately, we sustain the § 103(a) rejection of these dependent claims for the same reasons as the related independent claims. See 37 C.F.R. § 41.37(c)(l)(iv). Dependent Claims 11 and 16 Claim 11 depends from claim 9 and specifies that the “target object is an interventional tool, which is detected and tracked automatically.” App. 10 Appeal 2017-005914 Application 14/438,229 Br. 15 (Claims App.). Claim 16 depends from claim 1 and similarly specifies that the “target object corresponds to” an “interventional tool or device.” Id. at 17. Appellants argue that the Examiner erred in rejecting claims 11 and 16 because: (1) “[t]he term ‘interventional tools’ as generally understood in the art, and as used in the specification means catheters, needles, probes, and the like”; and (2) Thornton’s radiotherapy seeds “are implanted devices not tools.” App. Br. 11. Appellants’ arguments do not persuade us of Examiner error because the Examiner determines that the term “interventional tools” encompasses “devices used to carry out a surgical interventional function.” Adv. Act. 2; Ans. 6; see Final Act. 5. Based on that broad but reasonable claim interpretation, the Examiner finds that Thornton’s radiotherapy seeds constitute “interventional tools” and correspond to the claimed “target object.” Ans. 6; see Final Act. 5, 9, 13—14; Adv. Act. 2. Consistent with that, Thornton discusses using the radiotherapy seeds to treat cancer by surgically implanting them near the affected area. Thornton 1:13—21,2:33— 37, 3:29-48, 6:52—65, 7:32—39, Abstract. Accordingly, we discern no error in the Examiner’s claim interpretation or finding. In the Reply Brief, Appellants do not address claims 11 and 16. See Reply Br. 5—8. We note that Gerard discloses “guiding a catheter within the heart cavities in order to diagnose and cure heart diseases,” e.g., during an “interventional procedure.” Gerard H 1,43. Because Appellants’ arguments have not persuaded us that the Examiner erred in rejecting claims 11 and 16 for obviousness based on 11 Appeal 2017-005914 Application 14/438,229 Thornton and Gerard, we sustain the § 103(a) rejection of these dependent claims. The § 103(a) Rejection of Claims 6 and 17 Claim 6 depends indirectly from claim 1 and specifies that the “target object” is a “probe that is used for acquiring the ultrasound image.” App. Br. 14 (Claims App.). Claim 17 depends from claim 16 and similarly specifies that “the target object is the ultrasound probe.” Id. at 17. Appellants assert that the Examiner erred in rejecting claims 6 and 17 because Thornton’s radiotherapy seeds “are not an ultrasound probe.” App. Br. 11. Appellants’ assertion does not persuade us of Examiner error because the Examiner relies on Lang rather than Thornton for teaching or suggesting an “ultrasound probe” as a “target object.” Final Act. 15—16; Ans. 7. The Examiner finds that Lang discloses: (1) a “transesophageal echo (TEE) transducer” corresponding to an “ultrasound probe”; (2) “tracking the TEE transducer” during a procedure; and (3) “aligning] a fluoroscopy model image of the TEE probe” using an “intensity based registration.” Ans. 7; see Lang 1445—46, Fig. 4; see also Final Act. 15. In the Appeal Brief and the Reply Brief, Appellants do not address Lang’s disclosure. See App. Br. 9—11; Reply Br. 5—8. And in the Reply Brief, Appellants do not address claims 6 and 17. See Reply Br. 5—8. Because Appellants’ assertion has not persuaded us that the Examiner erred in rejecting claims 6 and 17 for obviousness based on Thornton, Gerard, and Lang, we sustain the § 103(a) rejection of these dependent claims. 12 Appeal 2017-005914 Application 14/438,229 DECISION We affirm the Examiner’s decision rejecting claims 1, 2, 4—6, 8, 9, and 11—17. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv). See 37 C.F.R. § 41.50(f). AFFIRMED 13 Copy with citationCopy as parenthetical citation