Intel Corporation

Patent Trials and Appeals Board

Jun 1, 2020

2019002328 (P.T.A.B. Jun. 1, 2020)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/976,489 12/21/2015 Jim S. Baca P90318 (884.T68US1) 9934
45457 7590 06/01/2020
SCHWEGMAN LUNDBERG & WOESSNER/Intel
P.O. Box 2938
MINNEAPOLIS, MN 55402
EXAMINER
HUANG, FRANK F
ART UNIT PAPER NUMBER
2485
NOTIFICATION DATE DELIVERY MODE
06/01/2020 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
SLW@blackhillsip.com
uspto@slwip.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte JIM S. BACA, DAVID STANASOLOVICH,
NEAL PATRICK SMITH, and YOGESHWARA KRISHNAN
____________

Appeal 2019-002328
Application 14/976,489
Technology Center 2400
____________

Before ELENI MANTIS MERCADER, NORMAN H. BEAMER, and
GARTH D. BAER, Administrative Patent Judges.

BEAMER, Administrative Patent Judge.

DECISION ON APPEAL
Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
Final Rejection of claims 1–24. We have jurisdiction over the pending
rejected claims under 35 U.S.C. § 6(b).
We AFFIRM.

1 We use the word “Appellant” to refer to “applicant” as defined in
37 C.F.R. § 1.42. Appellant identifies Intel Corporation as the real party in
interest. (Appeal Br. 3.)
Appeal 2019-002328
Application 14/976,489

2
THE INVENTION
Appellant’s disclosed and claimed invention is directed to enhanced
imaging (Spec. ¶ 1) that provides contextual and object information in the
captured image itself. (Spec. ¶ 9.)
Independent claim 1, reproduced below, is illustrative of the subject
matter on appeal:
1. At least one non-transitory machine readable medium
including instructions for enhanced imaging, the instructions,
when executed by a machine, cause the machine to perform
operations comprising:
sampling light from the environment to create an image;
sampling reflected energy from the environment
contemporaneously to the sampling of the light to create a
depth image of the environment;
applying a classifier to the image and the depth image to
provide a set of object properties of an object in the
environment; and
constructing a composite image that includes a portion of
the image in which the object is represented, a corresponding
portion of the depth image, and the set of object properties,
including:
encoding the depth image as a channel of the image; and
including a geometric representation of the object in meta
data in the composite image, the geometric representation
registered to the image.
Appeal Br. 14. (Claims Appendix.)

REJECTIONS
The Examiner rejected claims 1–24 under 35 U.S.C. § 103 as being
unpatentable over Yakubovich et al (US 2014/0079314 A1, pub. Mar. 20,
2014) (hereinafter “Yakubovich”), Reisner-Kollmann et al (US
Appeal 2019-002328
Application 14/976,489

3
2015/0062120 A1, pub. Mar. 5, 2015) (hereinafter “Reisner-Kollmann”),
and Official Notice. (Final Act. 3; Ans. 16.)2
ISSUE ON APPEAL
Appellant’s arguments in the Appeal and Reply Briefs present the
following issue:3
Whether the Examiner erred in finding the combination of
Yakubovich and Reisner-Kollmann teaches or suggests the limitations of
sampling reflected energy from the environment
contemporaneously to the sampling of the light to create a
depth image of the environment,
and
constructing a composite image that includes a portion of
the image in which the object is represented, a corresponding
portion of the depth image, and the set of object properties,
including:
encoding the depth image as a channel of the image; and
including a geometric representation of the object in meta
data in the composite image, the geometric representation
registered to the image,
as recited in independent claim 1, and the commensurate limitations recited
in independent claims 9 and 17. (Appeal Br. 10–12; Reply Br. 2–4.)

2 The Examiner additionally took Official Notice in the Answer that “depth
map images as a channel image is well known in the art.” See Ans. 16.
3 Rather than reiterate the arguments of Appellant and the positions of the
Examiner, we refer to the Appeal Brief (filed July 2, 2018); the Reply Brief
(filed Jan. 21, 2019); the Final Office Action (mailed Feb. 2, 2018); and the
Examiner’s Answer (mailed Nov. 20, 2018) for the respective details.
Appeal 2019-002328
Application 14/976,489

4
ANALYSIS
We have reviewed the Examiner’s rejections in light of Appellant’s
arguments. Arguments Appellant could have made but chose not to make
are deemed to be waived. See 37 C.F.R. § 41.37(c)(1)(iv).
In finding that the combination of Yakubovich, Reisner-Kollmann,
and Official Notice teaches or suggests the independent claim 1 limitations
at issue, the Examiner relies on Yakubovich’s disclosure of: (1) a working
image library that includes object characteristics and parameters
corresponding to each object characteristic, scene characteristics and
parameters corresponding to each scene characteristic, camera
characteristics, and parameter domains for the object characteristics;
RFID system in a business with customers waiting in lines to check out; and
(2) a camera for capturing an image and obtaining metadata. (Final Act. 4–
5, Ans. 13–15; Yakubovich Figs. 10, 4, Claims 10–11).
The Examiner further relies on the Reisner-Kollmann’s disclosure of:
(1) techniques for three-dimensional mapping, including (a) a simultaneous
localization and mapping process, (b) use of depth cameras including a
depth sensor; and (c) image capture that includes depth information using
time-of-flight analysis or a stereoscopic camera system; and
(2) image classification of point cloud data. (Final Act. 5–7, Ans. 14–16;
Reisner-Kollmann ¶¶ 83–84, 60, 94, Fig. 9.)
Appellant argues that the references form “a rag-tag group of
materials” in which “[n]one of the references are concerned with capturing
the triumvirate of depth data, image data, and classification data from a
single device into a single composite image format as claimed.” (Reply Br.
2.) Appellant contends that
Appeal 2019-002328
Application 14/976,489

5
a person of ordinary skill in the art would not seek these
references because they are not in the same field and they are
not pertinent to the particular problem with which the present
application is concerned — which is to provide “efficient
transformation of raw sensor data into actionable depth and
classification data for down-stream use.”
(Reply Br. 2, citing Yakubovich Abstract, Reisner-Kollmann Abstract,
McDowall4 Title, Abstract; Spec. ¶¶ 9–13, 24–26)
We are not persuaded by Appellant’s arguments. Regarding
Appellant’s characterization of Yakubovich and Reisner-Kollmann as being
from a “rag-tag group,” we disagree that a person of ordinary skill in the art
would not seek these references. As stated in In re Kahn, 441 F.3d 977, 987
(Fed. Cir. 2006), “[t]he analogous-art test requires that the Board show that a
reference is either in the field of the applicant’s endeavor or is reasonably
pertinent to the problem with which the inventor was concerned in order to
rely on that reference as a basis for rejection.” Id. at 986–87 (citing In re
Oetiker, 977 F.2d 1443 at 1447).
Here, Yakubovich “relates to object detection or object recognition
and more specifically to training an object detector” (Yakubovich ¶ 1), one
skilled in the art would look to Yakubovich’s teachings as providing a
benchmark for obtaining accurate results, and the manner in which such
results are communicated via datasets is pertinent. Further, Yakubovich’s
description of the contents of a “working image library” provides one skilled
in the art with an extensive list of “object properties” important to providing
context to the collected image data. See Yakubovich Claim 10. Similarly,

4 McDowall (US 2013/0041221 A1; Feb. 14, 2013) was cited in the Final
Rejection as teaching a system “known to provide [a] depth image as a
channel of the image.” (Final Act. 8, Ans. 6, citing McDowall ¶ 37.)
Appeal 2019-002328
Application 14/976,489

6
Reisner-Kollmann extends these teachings through “machine-based
recognition of real-world surroundings” (Reisner-Kollmann ¶ 2) that
includes “constructing a digital representation of a physical scene by
obtaining information about the physical scene.” (Reisner-Kollmann ¶ 6.)
One skilled in the art would find these references pertinent to developing an
“image capture system” that would provide “context with regard to objects
(e.g., things, planes, shapes, etc.) that are detectable in the captured images.”
(Spec. ¶ 9.)
Appellant also argues that “the Examiner alleges that Yakubovich
discusses obtaining information on objects, asserting that this is somehow a
composite image when Yakubovich never discusses any single image format
in this context.” (Reply Br. 3, citing Ans. 13.) Appellant contends that in
Yakubovich, “a perspective 3D image does not use a depth sensor, but rather
multiple cameras from which depth information may be extracted by
comparing the different perspectives.” (Reply Br. 3.)
We are not persuaded by Appellant’s critique of Yakubovich, as
Appellant attacks Yakubovich individually regarding the lack of a depth
sensor, whereas the Examiner finds, and we agree, that Reisner-Kollmann
teaches “the images may be taken with depth cameras — e.g., cameras
which include a depth sensor” (Ans. 15, citing Reisner-Kollmann ¶ 83) in
which “a depth sensor provides a range image where each pixel
represents the distance to the camera — i.e. depth image. The pixels can
be easily converted to a three dimensional (3-D) point cloud.” (Ans. 15,
citing Reisner-Kollmann ¶ 84.) Appellant’s arguments fail to consider the
combined teachings of the references.
Appeal 2019-002328
Application 14/976,489

7
Appellant finally argues that regarding the Wikipedia article, “as the
article discusses multiple depths maps of a scene, and depth maps as images,
it is not immediately apparent that one depth map is a channel in a depth
image of several depth maps.” (Reply Br. 4.)
We are not persuaded by Appellant’s argument. In the application of
Official Notice, the Examiner finds, and we agree, that “a depth map is an
image or image channel that contains information relating to the distance
of the surfaces of scene objects from a viewpoint” (Ans. 16, citing
https://web.archive.org/web/20150223035926/https://en.wikipedia.org/wiki/
Depth_map (emphasis added)), and Appellant fails to persuasively challenge
the statement that “a depth map is an image or image channel.” Further,
claim 1 does not define “channel of the image,” and this term is used in the
disclosure without further clarification, other than the statement that “the
depth information is treated akin to a color channel in the composite
image.” See Spec. ¶ 22; see also Spec. ¶¶ 42, 66, 79, 92, 107.
Accordingly, we sustain the Examiner’s obviousness rejection of
independent claim 1, as well as independent claims 9 and 17 commensurate
in scope, and all dependent claims not argued separately. See Appeal Br.
11–12.
For the reasons stated above, we affirm the Examiner’s obviousness
rejection of claims 1–24.

Appeal 2019-002328
Application 14/976,489

8
DECISION SUMMARY
In summary:
Claims
Rejected
35 U.S.C. § Reference(s)/Basis Affirmed
1–24 103 Yakubovich, Reisner-
Kollmann, Official Notice
1–24

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED