Google LLC

Patent Trials and Appeals Board

May 17, 2021

2020001365 (P.T.A.B. May. 17, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
16/009,696 06/15/2018 Andrew Ian Russell 0059-610001 2748
79318 7590 05/17/2021
BRAKE HUGHES BELLERMANN LLP
P.O. Box 1077
Middletown, MD 21769
EXAMINER
LIU, ZHENGXI
ART UNIT PAPER NUMBER
2611
NOTIFICATION DATE DELIVERY MODE
05/17/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docketing@brakehughes.com
uspto@brakehughes.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
_______________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
_______________

Ex parte ANDREW IAN RUSSELL
_______________

Appeal 2020-001365
Application 16/009,696
Technology Center 2600
_______________

Before ERIC S. FRAHM, JOHN A. EVANS, and
BETH Z. SHAW, Administrative Patent Judges.

FRAHM, Administrative Patent Judge.

DECISION ON APPEAL
Appellant1 appeals under 35 U.S.C. § 134(a) from the
Examiner’s final rejection of claims 1–21. We have jurisdiction under
35 U.S.C. § 6(b).
We affirm in part.
DISCLOSED AND CLAIMED INVENTION
Appellant’s disclosed invention “relates, generally, to shading images
in a three-dimensional system” (Spec. ¶ 1), such as communication systems
used for video conferencing, “to fulfill the wish for efficient and natural

1 We use the word “Appellant” to refer to “applicant” as defined in
37 C.F.R. § 1.42. “The word ‘applicant’ when used in this title refers to the
inventor or all of the joint inventors, or to the person applying for a patent as
provided in §§ 1.43, 1.45, or 1.46.” 37 C.F.R. § 1.42(a) (2017). According
to Appellant, Google LLC is the real party in interest (Appeal Br. 1).
Appeal 2020-001365
Application 16/009,696

2
long-distance communication” including “a natural person-to-person
interaction between two or more people” that is lifelike (Spec. ¶ 2).
Appellant achieves these purposes by using depth data (common in three-
dimensional image processing) to differentiate and shade different image
content and/or portions of objects in an image scene with different shading
processes (see Spec. ¶¶ 4, 5; Fig. 5 (first portion 500 shaded differently than
second portion 504); claims 1 and 10). Appellant claims a method (claim 1)
and system (claim 19) for making a three-dimensional presentation by using
depth data to identify first and second image content and to apply different
shading thereto (see claims 1, 10, 19).
Independent claims 1 and 10 are illustrative of the invention and are
reproduced below.
1. A method comprising:
receiving three-dimensional (3D) information generated
by a first 3D system, the 3D information including images of a
scene and depth data about the scene, the images generated by
cameras capturing respective views of the scene;
identifying, using the depth data, first image content in the
images associated with a depth value that satisfies a criterion;
generating modified 3D information by applying first
shading regarding the identified first image content; and
making, using the modified 3D information, a 3D
presentation that includes at least a portion of the respective
views of the scene.
Appeal Br. 7, Claims Appendix (emphasis added).
10. The method of claim 1, wherein the scene contains an
object in the images and a first portion of the object has a greater
depth value in the depth data than a second portion of the object,
and wherein generating the modified 3D information further
comprises applying second shading regarding a portion of the
images where second image content corresponding to the second
portion is located.
Appeal 2020-001365
Application 16/009,696

3
Appeal Br. 8, Claims Appendix (emphases added).

EXAMINER’S REJECTIONS
(1) The Examiner rejected claims 1–5, 14, 18, and 19 under 35 U.S.C.
§ 103 as being unpatentable over Nguyen et al. (US 2012/0051631 A1;
published March 1, 2012) (hereinafter, “Nguyen”) and Dore et al. (US
2017/0347055 A1; published Nov. 30, 2017) (hereinafter, “Dore”). Final
Act. 6–13.
(2) The Examiner rejected claims 6–10, 13, 20, and 21 under
35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, and Chuang et al.
(US 2018/0012407 A1; published Jan. 11, 2018) (hereinafter, “Chuang”).
Final Act. 13–17.
(3) The Examiner rejected claims 11 and 12 under 35 U.S.C. § 103 as
being unpatentable over Nguyen, Dore, Chuang, and Kubota et al. (US
2011/0292018 A1; published Dec. 1, 2011) (hereinafter, “Kubota”). Final
Act. 17–18.
(4) The Examiner rejected claim 15 under 35 U.S.C. § 103 as being
unpatentable over Nguyen, Dore, and Seto (JP 401188176 A; published July
27, 1989). Final Act. 19.
(5) The Examiner rejected claim 16 under 35 U.S.C. § 103 as being
unpatentable over Nguyen, Dore, and Altberg et al. (US 2008/0263458 A1;
published Oct. 23, 2008) (hereinafter, “Altberg”). Final Act. 19–20.
(6) The Examiner rejected claim 17 under 35 U.S.C. § 103 as being
unpatentable over Nguyen, Dore, Altber, and Lux et al. (US 2016/0140689
A1; published May 19, 2016) (hereinafter, “Lux”). Final Act. 20–21.

Appeal 2020-001365
Application 16/009,696

4
ISSUES
Because Appellant does not present any separate arguments as to the
obviousness rejections of (i) claims 1–5, 14, 18, 19, and 21 over the
combination of Nguyen and Dore; (ii) claims 6–9 and 21 over the
combination taken with Chuang; and (iii) claims 15–17 over the
combination taken with various other tertiary prior art references, Appellant
has not shown that the Examiner erred in rejecting claims 1–9, 14–19, and
21, and we sustain the obviousness rejections of claims 1–9, 14–19, and 21
pro forma. Ex parte Frye, 94 USPQ2d 1072, 1076 (BPAI 2010)
(precedential) (“[T]he Board will generally not reach the merits of any issues
not contested by an appellant.”); 37 C.F.R. § 41.37(c)(1)(iv) (2017) (Each
ground of rejection must be treated under a separate heading.). Other than
including this prior art rejection of claims 1–9, 14–19, and 21 (see supra
Rejections 1, 2, and 4–6) in our conclusion, we will not discuss these
rejections further.
Based on Appellant’s arguments in the Appeal Brief (Appeal Br. 2–6)
and the Reply Brief (Reply Br. 1–5), the following three principal issues are
presented on appeal:
(1) Did the Examiner err in rejecting claims 10–12 under 35 U.S.C.
§ 103 as being unpatentable over the combination of Nguyen, Dore, and
Chuang, because Chuang, and thus the combination of Nguyen, Dore, and
Chuang, fails to teach or suggest a scene containing an object having first
and second portions, where the different portions are shaded differently, as
recited in claim 10?
(2) Has Appellant shown the Examiner erred in rejecting claim 13
under 35 U.S.C. § 103 as being unpatentable over the combination of
Appeal 2020-001365
Application 16/009,696

5
Nguyen, Dore, and Chuang, because Nguyen, and thus the combination of
Nguyen, Dore, and Chuang, fails to teach or suggest identifying a hole, as
recited in claim 13?
(3) Has Appellant shown the Examiner erred in rejecting claim 20
under 35 U.S.C. § 103 as being unpatentable over the combination of
Nguyen, Dore, and Chuang, because Chuang, and thus the combination of
Nguyen, Dore, and Chuang, fails to teach or suggest determining a surface
normal and applying shading, as recited in claim 20?

ANALYSIS
We have reviewed Appellant’s arguments in the Appeal Brief (Appeal
Br. 2–6) and the Reply Brief (Reply Br. 1–5), the Examiner’s rejections
(Final Act. 6–21), and the Examiner’s response to Appellant’s arguments
(Ans. 4–11). With regard to claim 20, we agree with and adopt as our own
the Examiner’s findings of facts and conclusions as set forth in the Final
Rejection (Final Act. 1, 16) and Answer (Ans. 11). Appellant’s arguments
with respect to claim 20 (see Appeal Br. 5–6; Reply Br. 4–5) regarding
Chuang’s failure to teach or suggest applying shading of second content
based on the determined surface normal are not persuasive. However,
Appellant’s arguments have persuaded us of error in the Examiner’s
rejection of claims 10 and 13 under 35 U.S.C. § 103 based on Chuang’s
failure to teach or suggest applying different shading to first and second
portions of an object in a scene as recited in claim 10 (see Appeal Br. 3–4;
Reply Br. 1–3), and Nguyen’s failure to teach or suggesting identifying a
hole as recited in claim 13 (see Appeal Br. 4–5; Reply Br. 3–4).

Appeal 2020-001365
Application 16/009,696

6
Issue (1): Claim 10

Claim 10 stands rejected over Nguyen, Dore, and Chuang, and is
reproduced below:
10. The method of claim 1, wherein the scene contains an
object in the images and a first portion of the object has a greater
depth value in the depth data than a second portion of the object,
and wherein generating the modified 3D information further
comprises applying second shading regarding a portion of the
images where second image content corresponding to the second
portion is located.
Appeal Br. 8, Claims Appendix (emphases added).
In rejecting claim 10, the Examiner relies on Figures 25 and 26 of
Nguyen as teaching the limitation “wherein the scene contains an object in
the images and a first portion of the object has a greater depth value in the
depth data than a second portion of the object,” set forth in claim 10 (see
Final Act. 6, 7, 15; Ans. 5–7). Nguyen’s Figure 25 shows a scene,
containing an object in the foreground (the man with the glasses and stripe
shirt on), and a background containing several objects. Nguyen’s Figure 26
shows the object in the foreground, and also shows the background being
shaded out in its entirety. Nguyen’s Figures 25 and 26 thus show shading
different objects differently, and shading all the objects in the background
differently than the object in the foreground. However, Nguyen’s Figures 25
and 26 do not show a scene containing an object, where the object has
different portions of that object shaded differently as required by Appellant’s
claims 1 and 10, and as shown in Appellant’s Figure 5 showing an object
(person 500) in the foreground having a first portion (rest of torso portion
Appeal 2020-001365
Application 16/009,696

7
500) and a second portion 504 shaded differently (see Spec. ¶¶ 63–66)
(describing Figure 5).
The Examiner maps the limitations of claims 1 and 10 as follows:
• “first image content” is mapped to the background content of a
scene
• “first shading” is mapped to the image processing that involves
removing the background objects and coloring the background
in gray or a lighter color (see Nguyen, fig. 26)
• “a first portion of the object” (Claim 10) is mapped to the
background portion of a scene object
• “second image content” (Claim 10) is mapped to the foreground
content of the scene
• “a second portion of the object” (Claim 10) is mapped to the
foreground portion of the scene object
• “second shading” (Claim 10) is mapped to Chuang's shading
method.
Ans. 6. Thus, the Examiner maps Nguyen’s (i) background content shown
in Figure 25 to the “first image content” of claim 1 and the “first portion of
the object” of claim 10; and (ii) foreground content (the man with the glasses
and stripe shirt on) shown in Figure 25 to the “second image content” and
“second portion of the object” of claim 10. This mapping is unreasonable,
because Nguyen’s Figure 25 shows a scene or image containing multiple
objects, and shades all objects in the background in one manner, and the
foreground (which contains only one object) in another. Thus, Nguyen’s
Figure 25 fails to show two different portions of a single object in a scene
that are being shaded differently.
Appellant argues, and we agree, that “the Examiner’s reasoning
regarding Nguyen fails to address whether a person of ordinary skill in the
art would consider the ceiling, walls, a table, a standing person, and a seated
person, on the one hand, and the person in a striped shirt, on the other hand,
Appeal 2020-001365
Application 16/009,696

8
to be portions of the same ‘object’ as required by claim 10” (Reply Br. 3).
Appellant contends, and we agree, that Nguyen “does not describe these
separate features as all being part of some scene object” (Reply Br. 3).
Further, Appellant contends, and we agree, that “the Examiner presents no
argument, and makes no evidence of record, that a skilled artisan would
have considered these disparate aspects of Nguyen’s FIG. 25 to be portions
of the same object” (Reply Br. 3).
Appellant’s arguments have persuaded us of error in the Examiner’s
rejection of claim 10 over the combination of Nguyen, Dore, and Chuang,
and the claims which depend thereupon (i.e., claims 11 and 12 rejected over
the same combination of Nguyen, Dore, and Chuang, further in view of
Kubota). Accordingly, we do not sustain the Examiner’s rejections of claim
10, or dependent claims 11 and 12, which rely on the same combination. At
best, the Examiner’s proposed combination is speculative and not based on
record evidence. We will not resort to speculation or assumptions to cure
the deficiencies in the Examiner’s fact finding. See In re Warner, 379 F.2d
1011, 1017 (CCPA 1967); Ex parte Braeken, 54 USPQ2d 1110, 1112 (BPAI
1999) (unpublished) (“The review authorized by 35 U.S.C. [§] 134 is not a
process whereby the examiner . . . invite[s] the [B]oard to examine the
application and resolve patentability in the first instance.”).
Appellant’s arguments as to claim 10 are persuasive. Based on the
record before us, we do not sustain the Examiner’s obviousness rejection of
clam 10.
Issue (2): Claim 13
Claim 13 stands rejected over Nguyen, Dore, and Chuang, and is
Appeal 2020-001365
Application 16/009,696

9
reproduced below:
13. The method of claim 1, further comprising identifying
a hole in at least one of the images, wherein generating the
modified 3D information comprises applying second shading
regarding the hole.

Appeal Br. 8, Claims Appendix (emphasis added).
In rejecting claim 13, the Examiner relies on paragraph 70 of
Nguyen as teaching the limitations set forth in claim 13. Nguyen’s
paragraph 70 describes dilating boundaries of regions of foreground
pixels and eroding holes in foreground regions (see Nguyen ¶ 70).
The Examiner reasons that pixels within identified foreground pixel
areas that are not identified as foreground pixels are considered hole
pixels (see Ans. 9–10).
Appellant argues that Nguyen fails to teach or suggest identifying a
hole (see Appeal Br. 4–5; Reply Br. 3–4). We agree. Even if we agreed with
the Examiner that Nguyen teaches a hole in an image, we find no support in
the record that Nguyen teaches or suggests identifying a hole in an image,
and then applying second shading different than the first shading applied to
the first image content, as set forth in claim 1, from which claim 13 depends.
Based on the record before us, Appellant’s arguments as to claim 13
are persuasive. Accordingly, we do not sustain the Examiner’s obviousness
rejection of clam 13.
Issue (3): Claim 20
Claim 20 stands rejected over Nguyen, Dore, and Chuang, and is
reproduced below:
20. The system of claim 19, wherein the scene contains an
object in the images, and wherein generating the modified 3D
Appeal 2020-001365
Application 16/009,696

10
information further comprises determining a surface normal for
second image content of the object, and applying second shading
regarding the second image content based on the determined
surface normal.

Appeal Br. 9, Claims Appendix (emphasis added).
In rejecting claim 20, the Examiner relies on paragraph 143 of Chuang
as teaching applying second shading based on a determined surface normal
(see Final Act. 13, 16; Ans. 11). The Examiner reasons that “Chuang’s
shading method is ‘based on the determined surface normal’ because
paragraph 143 of Chuang recites ‘Contributions from non-occluded images
can be weighted by respective dot products of camera orientations and
surface normal.’” (Ans. 11). This finding is reasonable and supported by
paragraph 143 of Chuang, because applying shading for a surface normal
first requires determining the surface normal.
To the extent Appellant argues that Chuang’s weighting of dot
products of contributions from non-occluded images is not the same as
applying shading to image content (see Appeal Br. 6), this argument is
conclusory and not persuasive, because Appellant has presented no evidence
to support such a proposition. It is well settled that mere lawyer’s arguments
and conclusory statements, which are unsupported by factual evidence, are
entitled to little probative value. In re Geisler, 116 F.3d 1465, 1470 (Fed.
Cir. 1997); In re De Blauwe, 736 F.2d 699, 705 (Fed. Cir. 1984); In re
Schulze, 346 F.2d 600, 602 (CCPA 1965) (It is well established that the
arguments of counsel cannot take the place of evidence in the record); In re
Pearson, 494 F.2d 1399, 1405 (CCPA 1974) (Attorney argument is not
evidence); see also In re Rijckaert, 9 F.3d 1531, 1532 (Fed. Cir. 1993)
Appeal 2020-001365
Application 16/009,696

11
(When an examiner establishes a prima facie case of obviousness, the
burden shifts to the applicant to show non-obviousness).
In this light, Appellant’s arguments with respect to claim 20 (see
Appeal Br. 5–6; Reply Br. 4–5) regarding Chuang’s failure to teach or
suggest applying shading of second content based on the determined surface
normal are not persuasive. Accordingly, we sustain the Examiner’s rejection
of claim 20.

CONCLUSIONS
(1) Appellant has not presented any arguments as to claims 1–9, 14–
19, and 21, therefore Appellant has not shown the Examiner erred in
rejecting claims 1–9, 14–19, and 21 over the combination of Nguyen and
Dore, or the combination in view of various tertiary prior art references.
Accordingly, we summarily sustain the Examiner’s obviousness rejections
of claims 1–9, 14–19, and 21.
(2) Appellant has shown the obviousness rejections of claim 10, as
well as claims 11 and 12 depending therefrom and claim 13 separately
argued, over the combination of Nguyen, Dore, and Chuang to be in error.
Accordingly, we do not sustain the Examiner’s obviousness rejections of
claims 10–13.
(3) Appellant has not shown the obviousness rejection of claim 20
over the combination of Nguyen, Dore, and Chuang to be in error.
Accordingly, we sustain the Examiner’s obviousness rejection of claim 20.
In view of the foregoing, we (i) affirm the Examiner’s obviousness
rejections of claims 1–9 and 14–21 over the combination of Nguyen and
Appeal 2020-001365
Application 16/009,696

12
Dore; and (ii) reverse the Examiner’s obviousness rejection of claims 10–13
over Nguyen, Dore, and Chuang, under 35 U.S.C. § 103.
In summary:
Claims
Rejected
35 U.S.C.
§
Reference(s)/Basis Affirmed Reversed
1–5, 14, 18, 19 103 Nguyen, Dore 1–5, 14,
18, 19

6–10, 13, 20,
21
103 Nguyen, Dore,
Chuang
6–9, 20,
21
10, 13
11, 12 103 Nguyen, Dore,
Chuang, Kubota
11, 12
15 103 Nguyen, Dore, Seto 15
16 103 Nguyen, Dore,
Altberg
16
17 103 Nguyen, Dore,
Altberg, Lux
17
Overall
Outcome
1–9, 14–
21
10–13

No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED IN PART