Ex Parte Itkowitz et al

Patent Trial and Appeal Board

Mar 28, 2017

12780557 (P.T.A.B. Mar. 28, 2017)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
12/780,557 05/14/2010 Brandon D. Itkowitz ISRG 02640/US 1935
51947 7590 03/30/2017
Patent Dept. - Intuitive Surgical Operations, Inc.
1020 Kifer Road
Sunnyvale, CA 94086
EXAMINER
BRIER, JEFFERY A
ART UNIT PAPER NUMBER
2613
NOTIFICATION DATE DELIVERY MODE
03/30/2017 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
dana.mbenstein@intusurg.com
patent.group@intusurg.com
natalie.hood @ intusurg.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte BRANDON D. ITKOWITZ, SIMON P. DIMAIO,
and TAO ZHAO
Appeal 2014-003145
Application 12/780,5571
Technology Center 2600
Before HUNG H. BUI, JON M. JURGOVAN, and
KEVIN C. TROCK, Administrative Patent Judges.
TROCK, Administrative Patent Judge.
DECISION ON APPEAL
Introduction
Appellants seek review under 35 U.S.C. § 134(a) from the Examiner’s
Final Rejection of claims 1—10, 12—14, and 16—20.2 We have jurisdiction
under 35 U.S.C. § 6(b).
We AFFIRM-IN-PART.
1 Appellants indicate the Real Party in Interest is Intuitive Surgical
Operations, Inc. App. Br. 2.
2 Claims 11 and 15 are objected to as being dependent upon a rejected base
claim, but would be allowable if rewritten in independent form, including all
of the limitations of the base claim and any intervening claims. Final Act.
27.
Appeal 2014-003145
Application 12/780,557
Invention
The disclosure relates to endoscopic imaging, and more particularly to
overlaying an image on a stereoscopic real-time video display for a surgeon.
Spec. 11.
Exemplary Claim 1 is reproduced below:
1. A method comprising:
receiving an acquired video image at a controller,
wherein the acquired video image comprises a hand pose image
of a hand gesture;
segmenting, by the controller, the hand pose image from
the acquired video image to obtain a segmented hand pose
image, the segmenting comprising a depth threshold process
and a flood fill process, the depth threshold process and the
flood fill process each using depth data for pixels in the
acquired video image, the depth threshold process eliminating
pixels having a depth greater than a maximum depth threshold
from the acquired video image to obtain a first modified data
frame, and the flood fill process processing the first modified
data frame to obtain a hand pose mask;
combining, in real time by the controller, the segmented
hand pose image with an image of a surgical site to obtain a
combined image, wherein the combining comprises using an
alpha mask; and
sending the combined image from the controller to a
display device.
Exemplary Claim
References
Suzuki
Hasser
Mathe
US 2005/0231532 A1
US 2007/0167702 A1
US 2011/0102438 A1
Oct. 20, 2005
July 19, 2007
May 5, 2011
2
Appeal 2014-003145
Application 12/780,557
Jonathan Finger & Oliver Wang, Video Matting from Depth Maps
University of California Santa Cruz (2007).
Examiner’s Rejections
Claims 1—4, 7—10, and 16 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over Applicants’ Admitted Prior Art (AAPA), Finger,
and Mathe.
Claims 5 and 6 stand rejected under 35 U.S.C. § 103(a) as being
unpatentable over AAPA, Finger, Mathe, and Suzuki.
Claims 12—14 and 17—20 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over AAPA, Finger, Mathe, and Hasser.
ANALYSIS
We have reviewed the Examiner’s rejections and the evidence of
record in light of Appellants’ arguments that the Examiner has erred. Except
with respect to claims 12—14 and 17, we disagree with Appellants’
arguments and conclusions and we adopt as our own: (1) the findings and
reasons set forth by the Examiner in the Office Action from which this
appeal is taken and (2) the findings and reasons set forth in the Examiner’s
Answer. We further highlight specific findings and argument for emphasis
as follows.
Independent Claims 1, 18
Appellants contend the Examiner erred in combining Finger, Mathe
and AAPA to reject independent claims 1 and 18. App. Br. 7—18; Reply Br.
2—9. Appellants argue the Examiner’s proposed combination of Finger and
Mathe is improper because the Examiner essentially ignores the pre-
3
Appeal 2014-003145
Application 12/780,557
processing steps of the segmentation and flood fill methods taught by Finger
and Mathe, respectively. Appellants argue that the Examiner’s proposed
combination does not consider the references as a whole, because when so
considered, Finger and Mathe teach a “completely different” process (App.
Br. 12), or “teach away” (App. Br. 13), or would not “produce an acceptable
result” (App. Br. 14), or is not “likely to work” (App. Br. 15). Appellants
also argue the Examiner has improperly used Appellants’ Specification as
motivation to combine the prior art. App Br. 15—17.
In identifying a reason that would have prompted a person of ordinary
skill in the relevant field to combine the prior art teachings, an Examiner
must show some articulated reasoning with some rational underpinning to
support the legal conclusion of obviousness. KSR Int’l Co. v. Teleflex Inc.,
550 U.S. 398, 418 (2007). A reason to combine teachings from the prior art
“may be found in explicit or implicit teachings within the references
themselves, from the ordinary knowledge of those skilled in the art, or from
the nature of the problem to be solved.” IVMS Gaming Inc. v. Int 7 Game
Tech., 184 F.3d 1339, 1355 (Fed. Cir. 1999) (citing In reRouffet, 149 F.3d
1350, 1357 (Fed. Cir. 1998)). “Under the correct [obviousness] analysis,
any need or problem known in the field of endeavor at the time of invention
and addressed by the patent can provide a reason for combining the elements
in the manner claimed.” KSR, 550 U.S. at 420.
Here, AAPA acknowledges that “[s]ome work has been reported that
appears to use a constant color background to segment a video of a hand of a
proctor and to display the segmented image in the field of view of a surgeon
during normal open surgery.” Final Act. 5 (citing Spec. 1 5). The Examiner
finds that segmenting comprising a depth threshold process and a flood fill
4
Appeal 2014-003145
Application 12/780,557
process using depth data for pixels recited in claim 1 is not disclosed in
AAPA. Final Act. 6.
The Examiner finds, however, and we agree, Finger teaches
segmenting objects from received video and inserting the segmented objects
into video using depth information. Id. (see Finger Abstract) (“We look at
an existing technique for foreground-background separation called
Bayesian matting and improve upon it by adding depth information acquired
by a time-of-flight range scanner.'1'’). The Examiner further finds, and we
agree, Mathe describes segmenting a human target from a captured image
captured by a 3-D camera or RGB camera by using a depth process and
flood fill process. Id. (citing Mathe ^fl[ 88, 99, 93—101) (“The target
recognition, analysis, and tracking system may then flood-fill such points in
each frame [. . . and . . . ] may further remove or discard the pixels
associated with the environment, based on the depth history or history map,
by assigning them, for example, an invalid depth value.”).
The Examiner concludes, and we agree,
In view of Finger it would have been obvious to one of ordinary
skill in the art at the time of applicants!”] invention to substitute the
color based system of AAPA with the alpha mask generated using the
hand pose image since applicants] [have] selected from art recognized
equivalents for segmenting the proctor’s hand from received video for
overlaying segmented proctor’s hand into video of surgical site since
the result of the substitution would be predictable in view of being art
recognized equivalents.
Final Act. 7—8.
We find the Examiner’s proffered combination of familiar prior art
elements according to their established functions would have conveyed a
reasonable expectation of success to a person of ordinary skill at the time of
5
Appeal 2014-003145
Application 12/780,557
the invention. We are not persuaded by Appellants’ argument that
combining the respective elements of Finger, Mathe and AAPA was
“uniquely challenging or difficult for one of ordinary skill in the art.”
Leapfrog Enters., Inc. v. Fisher-Price, Inc., 485 F.3d 1157, 1162 (Fed. Cir.
2007) (citing KSR, 550 U.S. at 418). Appellants fail to provide sufficient
evidence that the combination yields anything other than predictable results.
See KSR at 416 (“The combination of familiar elements according to known
methods is likely to be obvious when it does no more than yield predictable
results.”). Arguments of counsel cannot take the place of factually
supported objective evidence. See, e.g., In re Huang, 100 F.3d 135, 139-40
(Fed. Cir. 1996); In re De Blauwe, 736 F.2d 699, 705 (Fed. Cir. 1984).
Appellants’ argument that Finger and Mathe teach some pre­
processing steps does not diminish what the references teach to one of
ordinary skill in the art. See App. Br. 13—15; Reply Br. 5—8. Nor does the
teaching of pre-processing steps preclude one of ordinary skill in the art
from combining the references in the manner proffered by the Examiner.
The Examiner properly notes, and we agree, claim 1 does not exclude a prior
art reference from having pre-processing with regard to the claimed
segmenting. Ans. 29.
Moreover, the test for obviousness is not whether the features of one
reference may be bodily incorporated into the structure of another reference;
nor is it that the claimed invention must be expressly suggested in any one or
all of the references. Rather, the test is what the combined teachings of the
references would have suggested to those of ordinary skill in the art. In re
Keller, 642 F.2d 413, 425 (CCPA 1981). Thus, Appellants’ argument that
the output data from the prior art’s depth thresholding step (Finger) would
6
Appeal 2014-003145
Application 12/780,557
be different, and therefore incompatible, with the input data to a prior art
flood fill step (Mathe), is not persuasive. App. Br. 13—15; Reply Br. 5—8.
Such an alleged incompatibility, even if arguably true, would be readily
resolved by a person of ordinary skill in this field of endeavor. “A person of
ordinary skill is also a person of ordinary creativity, not an automaton.”
KSR at 421.
Finally, we are also not persuaded by Appellants’ argument that Finger
and Mathe “teach away” from the claimed invention. See App. Br. 13;
Reply Br. 5—7. To teach away, a reference must actually “criticize, discredit,
or otherwise discourage” investigation into the claimed solution. In re
Fulton, 391 F.3d 1195, 1201 (Fed. Cir. 2004). Appellants’ argument that
Finger and Mathe teach away “because there is no suggestion of passing
data from a depth thresholding process to a flood fill process without the
additional pre-processing of the data taught by Mathe” (Reply Br. 6—7) does
not criticize, discredit or otherwise discourage investigation into the claimed
solution.
Accordingly, we are not persuaded the Examiner erred in finding the
combination of AAPA, Finger, and Mathe teaches or suggests the disputed
elements of independent claims 1 and 18. Therefore, we sustain the
Examiner’s rejection of these claims.
Dependent Claim 5
Claim 5 depends from claim 1, and adds the limitation “transforming a
perspective of the segmented hand pose image prior to the combining, the
transforming being by the controller.” Appellants contend the Examiner
erred in rejecting claim 5 because the combination of Finger, Mathe, and
Suzuki does not teach or suggest “transforming a perspective of the
7
Appeal 2014-003145
Application 12/780,557
segmented hand pose image prior to the combining.” App. Br. 20—21; Reply
Br. 10.
The Examiner finds, however, and we agree, Finger teaches
transforming the image after the segmenting performed by thresholding and
prior to combining. Final Act. 12—13 (citing Finger 3.1.1, 3.1.2, 3.1.3).
Accordingly, we are not persuaded by Appellants’ argument that the
Examiner erred in finding the combination of Finger, Mathe, and Suzuki
teaches or suggests “transforming a perspective of the segmented hand pose
image prior to the combining,” as recited in dependent claim 5. Therefore,
we sustain the Examiner’s rejection of this claim.
Dependent Claim 6
Appellants contend the Examiner erred in rejecting claim 6 because the
combination of Finger, Mathe, and Suzuki does not teach or suggest
“transforming the segmented hand pose image to achieve an apparent scale
and depth consistent with a slave surgical instrument or anatomy viewed in
the surgical site image.” App. Br. 21; Reply Br. 10. The Examiner finds,
however, and we agree, Finger teaches transforming the image after the
segmenting performed by thresholding and prior to combining while
achieving an apparent scale and depth consistent with the background. Final
Act. 13—14 (citing Finger 3.1.1, 3.1.2, 3.1.3).
Accordingly, we are not persuaded by Appellants’ argument that the
Examiner erred in finding the combination of Finger, Mathe, and Suzuki
teaches or suggests “transforming the segmented hand pose image to achieve
an apparent scale and depth consistent with a slave surgical instrument or
anatomy viewed in the surgical site image,” as recited in claim 6. Therefore,
we sustain the Examiner’s rejection of this claim.
8
Appeal 2014-003145
Application 12/780,557
Dependent Claim 8
Appellants contend the Examiner erred in rejecting claim 8 because the
combination of Finger and Mathe does not teach or suggest “eliminating
pixels not representing the hand gesture . . . based at least in part on the
depth data.” App. Br. 19; Reply Br. 9. The Examiner finds, however, and
we agree, Finger teaches removing background around a subject based on
depth information and Mathe describes segmenting a human target by using
a depth process and flood fill process to eliminate pixels having a depth
greater than a maximum depth threshold. Final Act. 10 (citing Finger 3.1.2;
Mathe H 88, 99, 93-101; Fig. 7).
Accordingly, we are not persuaded by Appellants’ argument that the
Examiner erred in finding the combination of Finger and Mathe teaches or
suggests “eliminating pixels not representing the hand gesture . . . based at
least in part on the depth data,” as recited in claim 8. Therefore, we sustain
the Examiner’s rejection of this claim.
Dependent Claims 12—14
Claim 12 depends from claim 1, and adds the limitation “wherein the
segmenting the hand pose image further comprises the controller using
information characterizing the master manipulator in the segmenting,
wherein the information is different from depth data.”
The Examiner finds Finger teaches more than depth information is
used to characterize the background, such as the background having a master
manipulator. Final Act. 15—16 (citing Finger 3.1.1—3.1.3).
Appellants argue that the Examiner erred in rejecting claim 12 because
the Examiner “has failed to cite any suggestion of any information in
addition to the acquired pixel data and the depth data being used in the
9
Appeal 2014-003145
Application 12/780,557
segmentation process of Finger.” App. Br. 22.
We agree with Appellants. We do not discern any discussion of a
background having a master manipulator in the sections of Finger cited by
the Examiner. Absent a specific explanation from the Examiner as to how
the cited limitation is taught or suggested by the relied upon references, we
are left to speculate how the reference is being applied. We decline to
engage in such speculation. As such, we conclude the rejection lacks the
requisite specificity required to establish a prima facie case of obviousness.
Accordingly, we are persuaded by Appellants’ argument that the
Examiner erred in finding the combination of Finger, Mathe, and Hasser
teaches or suggests “using information characterizing the master
manipulator in the segmenting, wherein the information is different from
depth data,” as recited in dependent claim 12. Therefore, we do not sustain
the Examiner’s rejection of claim 12. Claims 13 and 14 depend from claim
12. Because we do not sustain the Examiner’s rejection of claim 12, we also
do not sustain the Examiner’s rejection of claims 13 and 14.
Dependent Claim 17
Claim 17 depends from claim 1, and adds the limitation “parking the
master manipulator to provide an unobstructed volume in which to make the
hand gesture.” The Examiner finds that Hasser is silent with regard to
parking the master manipulator, but finds that Hasser’s Figure 2 illustrates
master inputs 203 and 204 parked such that a hand may be gestured above
the master inputs. Final Act. 17—18. The Examiner also finds that Hasser’s
Figure 1 illustrates a surgeon’s hands within the console and Figure 2
illustrates master manipulator input devices 203 and 204 within the console.
10
Appeal 2014-003145
Application 12/780,557
Ans. 33. The Examiner concludes it would have been obvious to one of
ordinary skill in the art at the time of the invention to park a master
manipulator because this would have the predictable result of a surgeon
being able to work at an ergonomic site. Final Act. 18.
Appellants argue that whether Hasser’s surgeon console provides an
ergonomic site for minimally invasive surgery neither teaches nor suggests
anything concerning the relationship of the master manipulators and the
surgeon making a hand gesture. App. Br. 24; Reply Br. 12.
We agree with Appellants. The Examiner does not explain in
sufficient detail how the position of the master inputs in Hasser teaches or
suggests providing an unobstructed volume in which to make a hand gesture,
as recited in claim 17. Absent a specific explanation from the Examiner as
to how the limitation is taught or suggested by the reference, we are left to
speculate how the reference is being applied. We decline to engage in such
speculation. As such, we conclude the rejection lacks the requisite
specificity required for establishment of a prima facie case of obviousness.
Accordingly, we are persuaded by Appellants’ argument that the
Examiner erred in finding the combination of prior art teaches or suggests
“parking the master manipulator to provide an unobstructed volume in
which to make the hand gesture,” as recited in dependent claim 17.
Therefore, we do not sustain the Examiner’s rejection of this claim.
Remaining Claims 2—4, 7, 9—10, 16, 19, and 20
Appellants have not presented separate, substantive, or persuasive
arguments with respect to dependent claims 2-4, 7, 9-10, 16, 19, and 20.
See App. Br. 18, 25. Appellants merely assert the same arguments made for
other claims or assert patentability because of dependency. Such assertions
11
Appeal 2014-003145
Application 12/780,557
fail to sufficiently constitute a separate issue of patentability. Therefore, we
are not persuaded the Examiner erred in rejecting these claims. See
37 C.F.R. § 41.37(c)(l)(iv) (2014); In re Lovin, 652 F.3d 1349, 1356 (Fed.
Cir. 2011) (“We conclude that the Board has reasonably interpreted Rule
41.37 to require applicants to articulate more substantive arguments if they
wish for individual claims to be treated separately.”). Accordingly, we
sustain the Examiner’s rejection of claims 2-4, 7, 9-10, 16, 19, and 20.
DECISION
We AFFIRM the Examiner’s rejection of claims 1—10, 16, and 18—20.
We REVERSE the Examiner’s rejection of claims 12—14 and 17.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv).
AFFIRMED-IN-PART
12