Nissanka Arachchige Bodhi. Priyantha et al.

Patent Trials and Appeals Board

Sep 12, 2019

13923098 - (D) (P.T.A.B. Sep. 12, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/923,098 06/20/2013 Nissanka Arachchige Bodhi Priyantha 338878.01 9903
39254 7590 09/12/2019
Barta, Jones & Foley, P.C.
(Patent Group - Microsoft Corporation)
2805 Dallas Parkway
Suite 222
Plano, TX 75093
EXAMINER
BOYLAN, JAMES T
ART UNIT PAPER NUMBER
2486
NOTIFICATION DATE DELIVERY MODE
09/12/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docket@bjfip.com
usdocket@microsoft.com
uspto@dockettrak.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte NISSANKA ARACHCHIGE BODHI PRIYANTHA,
MATTHAI PHILIPOSE, and STEPHEN EDWARD HODGES
____________

Appeal 2018-009012
Application 13/923,0981
Technology Center 2400
____________

Before JOSEPH P. LENTIVECH, DAVID J. CUTITTA II, and
SCOTT RAEVSKY, Administrative Patent Judges.

CUTITTA, Administrative Patent Judge.

DECISION ON APPEAL

This is an appeal under 35 U.S.C. § 134(a) from the Examiner’s final
rejection of claims 1–20, all the pending claims in the present application.
We have jurisdiction over the appeal under 35 U.S.C. § 6(b).
We AFFIRM.

1 Microsoft Technology Licensing, LLC (“Appellant”) is the Applicant, as
provided for under 37 C.F.R. § 1.46, and is also identified in the Appeal
Brief as the real party in interest. See Appeal Br. 1.
Appeal 2018-009012
Application 13/923,098

2
STATEMENT OF THE CASE
Invention
Appellant’s invention relates to continuous vision systems having
“energy-efficient region of interest detection and capture . . . using
multimodal sensors, in which lower power sensors are used to ‘gate’ higher
power sensors.” Spec. ¶ 20; see ¶ 1.2

Exemplary Claim
Claims 1, 10, and 18 are independent claims. Claim 1 is exemplary
and is reproduced below with limitations at issue emphasized.
1. A method comprising:
processing first data of a first set of pixels corresponding
to an evaluation image captured by a gating sensor, the gating
sensor being a narrow depth-of-field sensor having a first
resolution;
determining to activate a gated sensor based on the
processing of the first data indicating that a region of interest is
captured within the evaluation image;
based on the determination, activating the gated sensor
to obtain an image set comprising one or more images that
include the region of interest, wherein the gated sensor captures
images that have a second resolution that is greater than the first
resolution;
based on the image set comprising one or more images
obtained by the gated sensor, determining to activate a second
gated sensor, the second gated sensor capturing images that

2 This Decision refers to: (1) Appellant’s Specification (“Spec.”) filed June
20, 2013; (2) the Final Office Action (“Final Act.”) mailed November 2,
2017; (3) the Appeal Brief (“Appeal Br.”) filed June 25, 2018; (4) the
Examiner’s Answer (“Ans.”) mailed July 19, 2018; and (5) the Reply Brief
(“Reply Br.”) filed September 19, 2018.
Appeal 2018-009012
Application 13/923,098

3
have a third resolution that is greater than the second resolution;
and
obtaining, from the second gated sensor, an image that
includes the region of interest captured by the one or more
images obtained from the gated sensor.
Appeal Br. A-1 (Claims Appendix).

REFERENCES
The Examiner relies upon the following prior art3 in rejecting the
claims on appeal:
McGowan US 2006/0139470 Al June 29, 2006
KONDO (“Kondo”) US 2007/0236497 A1 Oct. 11, 2007
Nguyen US 2011/0115886 A1 May 19, 2011
Ruedin US 7,961,906 B2 June 14, 2011
Hao US 2011/0216210 Al Sept. 8, 2011
Högasten US 2011/0221599 Al Sept. 15, 2011
Baker US 2012/0327218 A1 Dec. 27, 2012
Swaminathan US 2014/0168056 A1 June 19, 2014

REJECTIONS
Claims 1–3, 5, 7, 8, 10, 13, 16, and 18 stand rejected under 35 U.S.C.
§ 103 as being unpatentable over the combination of Kondo, Baker, and
Nguyen. See Final Act. 4–16.
Claim 4 stands rejected under 35 U.S.C. § 103 as being unpatentable
over the combination of Kondo, Baker, Nguyen, and Hao. See Final
Act. 16–18.

3 All citations to the references use the first-named inventor only.
Appeal 2018-009012
Application 13/923,098

4
Claim 6 stands rejected under 35 U.S.C. § 103 as being unpatentable
over the combination of Kondo, Baker, Nguyen, and Ruedin. See Final
Act. 18–19.
Claims 9 and 11 stand rejected under 35 U.S.C. § 103 as being
unpatentable over the combination of Kondo, Baker, Nguyen, and
McGowan. See Final Act. 19–21.
Claims 12, 14, 19, and 20 stand rejected under 35 U.S.C. § 103 as
being unpatentable over the combination of Kondo, Baker, Nguyen, and
Högasten. See Final Act. 21–25.
Claim 15 stands rejected under 35 U.S.C. § 103 as being unpatentable
over the combination of Kondo, Baker, Nguyen, Högasten, and McGowan.
See Final Act. 25–26.
Claim 17 stands rejected under 35 U.S.C. § 103 as being unpatentable
over the combination of Kondo, Baker, Nguyen, and Swaminathan. See
Final Act. 26–27.
Our review in this appeal is limited to the above rejections and the
issues raised by Appellant. Arguments not made in the Briefs are waived.
See 37 C.F.R. § 41.37(c)(1)(iv) (2016).

ANALYSIS
Issue 1: Does the Examiner err in finding the combination of Kondo,
Baker, and Nguyen teaches or suggests “determining to activate a gated
sensor based on the processing of the first data indicating that a region of
interest is captured within the evaluation image,” as recited in independent
claim 1?
Appeal 2018-009012
Application 13/923,098

5
The Examiner relies, inter alia, on Kondo’s Figure 23 to teach the
first limitation at issue. See Final Act. 5; Ans. 4–5. Kondo’s Figure 23 and
corresponding text disclose “object detection processing performed by [a]
security camera” in which an infrared sensor 424 acts as a gating sensor to a
microwave sensor 425, and the microwave sensor acts as a gating sensor to
the security camera 422. Kondo ¶ 89; see Kondo ¶¶ 223–238. The
Examiner finds “Kondo is being relied upon for ‘determining to activate a
gated sensor’” because Kondo discloses “[d]etermining to turn on a
microwave sensor based on the processing of infrared image data.” Ans. 5;
Final Act. 2–3 (emphasis omitted) (citing Kondo s201–s203 of Fig. 23).
More specifically, the Examiner finds Kondo discloses “the object detection
processing of the infrared sensor’s output and where the sensor receives
infrared rays emitted from an object including a human and outputs a
corresponding detection signal. This shows a gated sensor is activated based
on a determination of a ROI (human in the captured infrared image).” Final
Act. 2 (emphasis and bracketing omitted) (citing Kondo ¶¶ 223, 224). The
Examiner states “Kondo is silent in regards to the gated sensor capturing
images” but finds Kondo’s “sensors are not limited to the ones described,”
i.e., the microwave sensor. Final Act. 3 (citing Kondo ¶ 150). The
Examiner further finds Kondo discloses a “video camera [that] captures
images” of a ROI and acts as a gated sensor. Final Act. 3 (emphasis
omitted). Kondo’s video camera may be used as a gated sensor in place of
the microwave sensor “by a simple substitution of sensor.” Ans. 5–6.
Appeal 2018-009012
Application 13/923,098

6
The Examiner also relies on Baker to teach the limitation at issue,
finding:
Baker is being relied upon for “utilizing a second camera based
on ROI determination from a previous sensor” [See Baker
[Fig. 1, 0032 and 0043] After ROI determination is performed
based on thermal information, other sensors, including one or
more of the following: an RGB sensor, a depth sensor, are
focused on capturing and processing data corresponding to the
region of interest.
Ans. 5–6 (emphasis and some bracketing omitted). The Examiner
determines it would have been obvious to combine the teachings of Kondo
with Baker because the combination “is based upon a simple substitution of
known elements to perform power saving in the field of camera technology.”
Ans. 5.
Appellant does not challenge the Examiner’s conclusion of
obviousness but instead disputes the Examiner’s factual findings regarding
Kondo. Appellant argues “Kondo does not describe or suggest a gated
sensor that captures images” so “Kondo cannot describe or suggest
‘determining to activate a gated sensor based on the processing of the first
data indicating that a region of interest is captured within the evaluation
image,’ as the recited gated sensor is defined as a gated sensor that captures
images.” Appeal Br. 11.
We do not find Appellant’s arguments persuasive. Rather, we find the
Examiner has provided a comprehensive response to Appellant’s arguments,
and we adopt the Examiner’s findings and explanations. See Final Act. 2–8;
see also Ans. 4–6. Appellant’s argument that “the gated sensor in Kondo
. . . is a microwave sensor that transmits microwaves” and “does not
describe or suggest a gated sensor that captures images,” (Reply Br. 2), fails
Appeal 2018-009012
Application 13/923,098

7
to persuasively address the Examiner’s determination that (1) Kondo’s gated
sensor is not limited to the microwave sensor, (2) Kondo also discloses a
gated sensor that is a video camera capturing images, and (3) substituting the
video camera in Kondo for the microwave sensor would have been a simple
substitution of known elements. See Final Act. 3; Ans. 4–5.
Nor does Appellant address the Examiner’s conclusion that it would
have been obvious to substitute any of Baker’s “thermal sensor, a depth
sensor, and a RGB sensor, where each of these sensors capture the ROI,”
with Kondo’s sensor. Final Act. 3 (emphasis omitted) (citing Baker ¶ 32,
Fig. 1). In conclusion, Appellant’s argument fails to address why Kondo’s
determining to activate a gated sensor, such as a video camera substituted for
a microwave sensor, based on an output reading of an IR sensor (gating) in
combination with Baker’s ROI determination, fails to teach or suggest
“determining to activate a gated sensor,” as recited in claim 1.

Issue 2: Does the Examiner err in finding the combination of Kondo
and Baker teaches or suggests “based on the determination, activating the
gated sensor to obtain an image set comprising one or more images that
include the region of interest,” as recited in independent claim 1?
The Examiner finds Kondo teaches “activat[ing] a gated sensor” that
captures images, as claimed, because Kondo discloses “[d]etermining to turn
on a microwave sensor based on the infrared image data.” Final Act. 5
(emphasis omitted) (citing Kondo s201–s203 in Fig. 23). The Examiner also
finds Kondo discloses “[t]urning on power source of video camera based on
the output reading of the microwave sensor . . . [and] the sensors are not
limited to the ones described within . . . therefore, Kondo discloses the main
Appeal 2018-009012
Application 13/923,098

8
concept of processing sensor data to activate a following sensor for
capturing of data.” Final Act. 3; Ans. 5 (emphasis omitted) (citing Kondo
¶¶ 222–224, 150 and Fig. 3). The Examiner further finds Baker discloses “a
thermal sensor, a depth sensor, and a RGB sensor, where each of these
sensors capture the ROI.” Final Act. 3 (emphasis omitted) (citing Baker
¶ 32, Fig. 1. The Examiner concludes “the combination of Kondo and Baker
disclose the processing of each sensor data (i.e. three in total) in sequential
steps to activate the following sensor (i.e. first one is on, then second one is
turned on, then third one is turned on) for capturing image data of the
determined ROI.” Final Act. 3.
Appellant first argues “Baker does not describe a gated sensor,
merely a depth sensor that captures images, and the depth sensor in [B]aker
is not used as a gated sensor.” Appeal Br. 11; see Reply 3. We find this
argument unpersuasive because rather than Baker, the Examiner relies on
Kondo to teach the gated sensor. See Final Act. 5 (citing Kondo Fig. 23).
Appellant next argues “Kondo describes a gated sensor that cannot
capture images.” Appeal Br. 11. We find this argument unpersuasive for
the same reasons discussed above with respect to issue one.
In summary, we conclude Appellant’s arguments do not address the
actual reasoning of the Examiner’s rejection. Appellant, instead, attacks the
references singly for lacking teachings that the Examiner relies on a
combination of references to show. It is well established one cannot show
non-obviousness by attacking references individually where the rejections
are based on combinations of references. See In re Keller, 642 F.2d 413,
426 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 1097 (Fed. Cir.
1986). Our reviewing court requires that references must be read, not in
Appeal 2018-009012
Application 13/923,098

9
isolation, but for what they fairly teach in combination with the prior art as a
whole. Merck, 800 F.2d at 1097. Accordingly, we agree with the Examiner
the combination of Kondo and Baker teaches or suggests “based on the
determination, activating the gated sensor,” as recited in claim 1.

Issue 3: Does the Examiner provide erroneous reasoning in
combining the teachings of Kondo and Baker?
Appellant argues
the rejection is impermissibly based on a hindsight
reconstruction” because “to replace the microwave sensor
(alleged gated sensor) that transmits microwaves . . . with a
sensor from Baker that captures images, would render the
system in Kondo inoperable for its intended purpose as the
system in Kondo requires a sensor that transmits two detection
signals (microwaves) and . . . [a] captured image cannot provide
microwaves or detection signals.
Appeal Br. 10, 12 (citing In re Gordon, 733 F.2d 900 (Fed. Cir. 1984)).
Appellant’s argument is premised on a “physical” or “bodily”
incorporation of limitations of one reference into the other. However, this is
not the standard. “It is well-established that a determination of obviousness
based on teachings from multiple references does not require an actual,
physical substitution of elements.” In re Mouttet, 686 F.3d 1322, 1332 (Fed.
Cir. 2012) (citing In re Etter, 756 F.2d 852, 859 (Fed. Cir. 1985) (en banc));
see also In re Sneed, 710 F.2d 1544, 1550 (Fed. Cir. 1983) (“[I]t is not
necessary that the inventions of the references be physically combinable to
render obvious the invention under review.”). Rather than express
obviousness as the physical incorporation of a structure from one reference
into the structure of another reference, in an obviousness analysis, the prior
Appeal 2018-009012
Application 13/923,098

10
art should be viewed as a combination of teachings from different sources,
and the use of those teachings by one of ordinary skill in the art. See Keller,
642 F.2d at 425.
We find Appellant’s argument unpersuasive because the rejection
does not require physically incorporating any of Baker’s thermal sensor,
depth sensor, or RGB sensor within Kondo’s microwave sensor. Instead, the
Examiner rejects the claim as unpatentable over the combined teachings of
the references.
Accordingly, we determine the Examiner has provided sufficient
motivation for modifying Kondo based on the teachings of Baker, without
resorting to impermissible hindsight reasoning.
For these reasons, we sustain the Examiner’s 35 U.S.C. § 103
rejection of claim 1, and similarly, independent claims 10, and 18, which are
not argued separately from claim 1. See Appeal Br. 13. Dependent claims
2–9, 11–17, 19, and 20 are either not argued separately or are merely
nominally argued separately and so we also sustain the Examiner’s 35
U.S.C. § 103 rejection of these claims. See id. at 12–14.

DECISION
We affirm the Examiner’s decision to reject claims 1–20 under 35
U.S.C. § 103.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED