Babak Robert. Shakib et al.

Patent Trials and Appeals Board

Mar 2, 2020

14231644 - (D) (P.T.A.B. Mar. 2, 2020)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/231,644 03/31/2014 Babak Robert Shakib 561246US 1148
149118 7590 03/02/2020
Colby Nipper / Google
291 East Shore Drive
Suite 200
Eagle, ID 83616
EXAMINER
YOON, ERIC
ART UNIT PAPER NUMBER
2143
NOTIFICATION DATE DELIVERY MODE
03/02/2020 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docket@colbynipper.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________________
Ex parte BABAK ROBERT SHAKIB,
YURIY MUSATENKO, and ANDRII GUSHCHYK
____________________

Appeal 2018-008975
Application 14/231,644
Technology Center 2100
____________________

Before JAMES R. HUGHES, JOHN A. EVANS, and
JUSTIN BUSCH, Administrative Patent Judges.

BUSCH, Administrative Patent Judge.

DECISION ON APPEAL
Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the
Examiner’s decision to reject claims 1–4, 7, 10–15, 17, 18, 20, 23–27, 30,
and 31, which constitute all the claims pending in this application. We have
jurisdiction over the pending claims under 35 U.S.C. § 6(b).
We AFFIRM IN PART.

1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R.
§ 1.42. Appellant identifies the real party in interest as Google Technology
Holdings LLC. Appeal Br. 3.
Appeal 2018-008975
Application 14/231,644

2
CLAIMED SUBJECT MATTER
Appellant’s disclosure generally relates to “[t]echniques and
apparatuses for highlighting media through weighting of people or contexts”
to generate highlight reels based on the weightings. Spec. ¶ 5, Abstract.
Claims 1, 11, and 17 are independent. Claim 1 is reproduced below:
1. A method comprising:
determining, at a computing device, persons in source
media;
presenting, at the computing device, a user interface
enabling selection to weight the persons determined to be in the
source media, the user interface comprising a visual
representation of each of the persons determined to be in the
source media, the visual representation of each of the persons
comprising a facial image of each of the persons and being drawn
from a frame of a video or a still image of the source media, a
size of the visual representation of each of the persons
corresponding to a first weighting of the persons;
receiving, at the computing device, selection altering the
first weighting of the persons, the selection comprising receiving
a user gesture to alter the size of the visual representation of one
of the persons, the alteration of the size of the visual
representation of the one of the persons altering a corresponding
weighting of the one of the persons and producing a second
weighting of the persons; and
creating, at the computing device, a highlight reel based
on the source media and the second weighting of the persons.
REJECTIONS
Claim 31 stands rejected under 35 U.S.C. § 112(b) as indefinite. Final
Act. 2–3.
Claims 1–4, 7, 10, 11, 14, and 23–25 stand rejected under 35 U.S.C.
§ 103 as obvious in view of Morrison (US 2012/0189284 A1; July 26,
2012), Boiman (US 9,189,137 B2; Nov. 17, 2015), Kelly (US 8,484,149 B1;
Appeal 2018-008975
Application 14/231,644

3
July 9, 2013), and Wada (US 2009/0087037 A1; Apr. 2, 2009). Final
Act. 3–19.
Claims 12 and 13 stand rejected under 35 U.S.C. § 103 as obvious in
view of Morrison, Boiman, Kelly, Wada, and Mundy (US 2011/0183651
A1; July 28, 2011). Final Act. 19–20.
Claims 17, 18, 26, 27, 30, and 31 stand rejected under 35 U.S.C. § 103
as obvious in view of Morrison and Kelly. Final Act. 20–26.
Claim 20 stands rejected under 35 U.S.C. § 103 as obvious in view of
Morrison, Boiman, and Kelly. Final Act. 26–28.
ANALYSIS
We have reviewed the Examiner’s rejections in light of Appellant’s
arguments that the Examiner erred. In reaching this decision, we have
considered all evidence presented and all arguments Appellant made.
Arguments Appellant could have made, but chose not to make in the Briefs,
are deemed waived. See 37 C.F.R. § 41.37(c)(1)(iv).
THE REJECTION UNDER 35 U.S.C. § 112(b)
The Examiner rejects claim 31 under 35 U.S.C. § 112(b) as indefinite
for failing to particularly point out and distinctly claim the subject matter
Appellant regards as the invention. Final Act. 2–3. Claim 31 depends from
claim 30, which depends from independent claim 17. The Examiner finds
claims 17 and 30 recite “multiple contexts” and claim 31 recites “outdoor,
sunny, windy or people-eating contexts,” making it unclear whether claim
31’s recited contexts are the same contexts as the “multiple contexts” recited
in claims 17 and 30. The Examiner also finds claim 31 indefinite because it
is unclear whether the contexts in claim 31’s clause “the sample image
identifies one of the contexts” refers to the recited “multiple contexts”
Appeal 2018-008975
Application 14/231,644

4
recited in claims 17 and 30 or the “outdoor, sunny, windy or people-eating
contexts” recited in claim 31. Final Act. 3.
Appellant attempted to amend claim 31 on September 11, 2017, after
the Examiner issued the Final Rejection. See Appeal Br. 14. The Examiner
did not enter Appellant’s attempted amendment. Advisory Act. 1 (mailed
Sept. 27, 2017). Appellant argues the rejection of claim 31 under 35 U.S.C.
§ 112(b) should be reversed based on Appellant’s amendment submitted on
September 11, 2017. Appeal Br. 14. The Examiner explained that “[t]he
after-final amendment was a substantive amendment that modifies the scope
and meaning of the claims,” whereas only after-final amendments that
cancel claims, adopt Examiner’s suggestions, remove issues for appeal, or
require only cursory Examiner review are entered as a matter of right.
Ans. 25–26 (citing MANUAL OF PATENT EXAMINING PROCEDURE (MPEP)
§ 714.13(II)). Appellant refers to the MPEP’s statement that “amendments
that adopt Office suggestions should be entered” and argues “Appellant
adopted the Office’s suggestions as contained on pages 2 and 3 of the June
16, 2017, Final Office Action.” Reply Br. 2 (citing MPEP 714.13).
If an applicant believes an examiner improperly refused to enter an
amendment, the applicant may file a petition to the Director. 37 C.F.R.
§ 1.127. We do not decide such issues on appeal. Moreover, contrary to
Appellant’s argument, the cited portions of the Final Action merely state the
basis for the rejection of claim 31 under 35 U.S.C. § 112 and explain how
the Examiner is interpreting the claims for purposes of analyzing the claim
with respect to prior art. Final Act. 2–3. The Final Action does not provide
an Examiner-suggested amendment. Final Act. 2–3.
Appeal 2018-008975
Application 14/231,644

5
Appellant argues the merits of the indefiniteness rejection only with
respect to claim 31 as recited in Appellant’s amendment submitted on
September 11, 2017. Appellant does not argue the merits of claim 31 as
pending before us. For these reasons, we affirm the Examiner’s rejection of
claim 31 under 35 U.S.C. § 112(b) as indefinite.
THE REJECTIONS UNDER 35 U.S.C. § 103
The Examiner finds the combination of Morrison, Boiman, Kelly, and
Wada teaches or suggests every limitation recited in claims 1–4, 7, 10, 11,
14, and 23–25. Final Act. 3–19. The Examiner finds the combination of
Morrison and Kelly teaches or suggests every limitation recited in claims 17,
18, 26, 27, 30, and 31. Final Act. 20–26. The Examiner finds the
combination of Morrison, Boiman, Kelly, Wada, and Mundy teaches or
suggests every limitation recited in claims 12 and 13. Final Act. 19–20. The
Examiner finds the combination of Morrison, Boiman, and Kelly teaches or
suggests every limitation recited in claim 20. Final Act. 26–28.
Of particular relevance to this Appeal, the Examiner finds Kelly
teaches or suggests (1) “the visual representation of each of the persons . . .
being drawn from a frame of a video or a still image of the source media, a
size of the visual representation of each of the persons corresponding to a
first weighting of the persons” recited in claim 1, (2) “the visual
representation of each of the two or more persons . . . being drawn from a
frame of a video or a still image of the audiovisual media, a size of the visual
representation of each of the two or more persons corresponding to a first
weighting of the two or more persons” recited in claim 11, and (3) “the
visual representation of each of the multiple contexts . . . being drawn from a
frame of a video or a still image of the source media, a size of the visual
Appeal 2018-008975
Application 14/231,644

6
representation of each of the multiple contexts corresponding to a first
weighting of the multiple contexts” recited in claim 17 (collectively the
“visual representation drawn from the source media” limitations). Final
Act. 6 (citing Kelly 11:47–63, Fig. 7), 14, 21–22. For the reasons discussed
below, we disagree with the Examiner’s finding that Kelly teaches or
suggests the visual representation drawn from the source media limitations
recited in claims 1, 11, and 17 when these limitations are properly construed.
When rejecting claims for obviousness, the Examiner must show that
each and every limitation of the claim is described or suggested by the prior
art or would have been obvious based on the knowledge of those of ordinary
skill in the art or the inferences and creative steps a person of ordinary skill
in the art would have employed. In re Fine, 837 F.2d 1071, 1074 (Fed. Cir.
1988); KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 417 (2007).
As an initial matter, when construing claim terminology during
prosecution before the Office, claims are to be given their broadest
reasonable interpretation consistent with the Specification, reading claim
language in light of the Specification as it would be interpreted by one of
ordinary skill in the art. In re Am. Acad. of Sci. Tech Ctr., 367 F.3d 1359,
1364 (Fed. Cir. 2004). However, the broadest reasonable interpretation
differs from the broadest possible interpretation. In re Smith Int’l, Inc., 871
F.3d 1375, 1383 (Fed. Cir. 2017). The correct inquiry in giving a claim term
its broadest reasonable interpretation in light of the specification is “an
interpretation that corresponds with what and how the inventor describes his
invention in the specification, i.e., an interpretation that is ‘consistent with
the specification.’” Smith, 871 F.3d at 1382–83 (quoting In re Morris, 127
F.3d 1048, 1054 (Fed. Cir. 1997)).
Appeal 2018-008975
Application 14/231,644

7
The Examiner finds “the phrase ‘drawn from’ is interpreted broadly to
refer to almost any kind of connection or relationship” and, in the context of
Morrison, Kelly’s visual representation (i.e., bar graphs) “can relate to,
describe and/or represent[]” people or contexts “and aspects of source media
in” Morrison. Final Act. 6, 14, 22. As discussed below, we find the
Examiner’s construction of “drawn from” to be unreasonably broad and
inconsistent with how one of ordinary skill in the art would understand the
phrase in light of the Specification.
We look to the Specification to interpret the phrase “drawn from.”
The Specification describes exemplary methods and interfaces for weighting
people and contexts identified in source media. Spec. ¶¶ 91–105, Figs. 13–
16. The Specification discloses exemplary graphical user interfaces
including images of the people identified in the source media that can be
used to select a weighting for each person. See Spec. ¶¶ 93, 95, Fig. 14.
The Specification discloses that, although the people are “shown in line
drawing, these images 1406 for each person can be thumbnails showing each
person’s face pulled from source media (even the source media to be
highlighted), avatars, or other identifiers.” Spec. ¶ 95 (emphasis added); see
Spec. ¶ 95, 103 (“Thus a slider or size having an image of a sun and trees
can be used to enable selection to alter weightings for outdoor scenes and so
forth.”), Figs. 14–16. The Specification also provides an exemplary “size-
based weighting interface” that “presents an image for each of the persons in
the highlight reel or preview, each of the images representing each person’s
face and having a size corresponding to the person’s weighting in the
highlight reel.” Spec. ¶ 99, Figs. 15–16.
Appeal 2018-008975
Application 14/231,644

8
Although each of the disclosed embodiments merely provide
examples, they are instructive when evaluating how a person of ordinary
skill would interpret terms recited in the claims. We agree with the
Examiner’s underlying point that the broad claim language and open-ended
exemplary nature of the Specification leads to a broad construction of much
of the claim. In particular, a “visual representation of each of the persons” is
broad enough to encompass any image in a graphical user interface that
stands in for (i.e., represents) that person in the interface.
The claim language, however, more specifically recites that the
representation comprises “a facial image” and is “drawn from a frame of a
video or a still image of the source media.” As noted above, the
Specification discloses that the images representing the people in the
interface are shown in line drawing, but “can be thumbnails showing each
person’s face pulled from source media . . ., avatars, or other identifiers”
provides additional context for the claimed “visual representation . . .
comprising a facial image of each of the persons and being drawn from a
frame of a video or a still image of the source media.” In light of this
disclosure, we find a person of ordinary skill would have understood that the
visual representation “being drawn from a frame of a video or a still image
of the source media” means that the recited visual representation is an image
excerpted (i.e., “pulled”) from a video or still image, as opposed to merely a
graphical element that represents that person or context.
Kelly discloses, in relevant part, a graphical user interface with
rectangles depicting the weight of various factors, where the weight is the
product of an importance value and a rating value. Kelly 11:47–63, Fig. 7.
Thus, the two sides of the rectangle represent the importance and rating
Appeal 2018-008975
Application 14/231,644

9
values of a particular factor and the area within the rectangle provides a
visual depiction of the weighting of that factor. Kelly 11:47–63, Fig. 7.
Dragging the corner of a rectangle allows a user to modify the importance
and/or rating values and see a visualization of how that affects the weighting
(i.e., area) of the factors. Kelly 11:47–12:10.
The Examiner’s broad construction of “drawn from” encompasses any
connection or relationship. Given this construction, the Examiner finds
Kelly teaches or suggests the visual representations drawn from source
media limitations because Kelly discloses graphical user interfaces having
bar graphs representing the different factors a user can weight. Final Act. 6
(citing Kelly 11:47–63, Fig. 7); Ans. 27 (citing Kelly 11:58–12:3, Fig. 7).
However, given our interpretation of the claims, the Examiner’s findings are
problematic. In particular, the Examiner’s rejection is based on a discussion
of representations of factors that are not taken from images or videos and,
therefore, is different than the claimed “visual representation . . . being
drawn from a frame of a video or a still image of the source media.”
Appellant argues “Kelly does not describe or show actual persons nor
does it disclose or teach photographic-like representations of those users as
described in and consistent with the application disclosure.” Appeal Br. 29.
Although Appellant’s argument imprecisely describes what the claims
require, we ultimately agree with the issue Appellant raises. In fact, because
the Examiner construed these limitations differently than we did, the
Examiner does not make a finding that these limitations are taught by Kelly.
See Final Act. 7; Ans. 27–28. Therefore, in light of our construction, we
agree with Appellant that Kelly fails to teach or suggest the visual
representation drawn from the source media limitations.
Appeal 2018-008975
Application 14/231,644

10
Given the Examiner’s construction and finding that Kelly teaches
these limitations under the Examiner’s broad construction, the Examiner
made no findings that any of the additionally cited references teach or
suggest these limitations. Because we find (i) the Examiner’s construction
of “drawn from” to be unreasonably broad and inconsistent with the
Specification, (ii) the Examiner’s finding that Kelly teaches these limitations
in light of our construction to be in error, and (iii) the Examiner’s lack of
findings that any other cited disclosure teaches or suggests these limitations,
we are persuaded the Examiner erred. Because this error is dispositive of
the rejection under 35 U.S.C. § 103, we need not address other issues raised
by Appellant’s arguments.
Thus, constrained by the record before us, we do not sustain the
Examiner’s rejection of: (1) independent claims 1 and 11 and claims 2–4, 7,
10, 14, and 23–25, which depend therefrom and incorporate the respective
limitations, as obvious in view of Morrison, Boiman, Kelly, and Wada;
(2) claims 12 and 13, which depend from claim 11 and incorporate the
respective limitation, as obvious in view of Morrison, Boiman, Kelly, Wada,
and Mundy; (3) independent claim 17 and claims 18, 26, 27, 30, and 31,
which depend from claim 17 and incorporate the respective limitation, as
obvious in view of Morrison and Kelly; and (4) claim 20, which depends
from claim 17 and incorporates the respective limitation, as obvious in view
of Morrison, Kelly, and Boiman.
Notably, our decision is based only on the rejection before us. The
Examiner’s rejection was based on a construction with which we do not
agree. Accordingly, the Examiner has not had an opportunity to make
findings with respect to how additional portions of the cited references may
Appeal 2018-008975
Application 14/231,644

11
teach or suggest the visual representation drawn from the source media
limitations when construed consistently with the Specification. The Patent
Trial and Appeal Board is a review body, rather than a place of initial
examination. Therefore, we do not determine here whether the claims are
unpatentable over the cited prior art, but merely with respect to the specific
rejections on appeal. We leave it to the Examiner to ascertain the
appropriateness of any further rejections based on these or other references.2
Although we are authorized to enter a new ground of rejection, our decision
not to enter a new ground of rejection should not be considered an indication
regarding the appropriateness of further rejection or allowance of the non-
rejected claims. See MPEP § 1213.02 (9th ed. Rev. 08.2017, Jan. 2018).
CONCLUSION
Claims
Rejected
35
U.S.C. §
References/Basis Affirmed Reversed
31 112(b) Indefinite 31
1–4, 7, 10,
11, 14, 23–25
103 Morrison, Boiman,
Kelly, Wada
1–4, 7, 10, 11,
14, 23–25
12, 13 103 Morrison, Boiman,
Kelly, Wada, Mundy
12, 13
17, 18, 26,
27, 30, 31
103 Morrison, Kelly 17, 18, 26, 27,
30, 31
20 103 Morrison, Boiman, Kelly 20
Overall
Outcome
31 1–4, 7, 10–15,
17, 18, 20, 23–
27, 30, 31

2 For example, because the Examiner relied on Kelly to teach the visual
representations drawn from source media limitations, the Examiner has not
explained whether aspects of Wada may teach these limitations and, if so,
why and how a person of ordinary skill in the art would have combined such
teachings with the other cited teachings to arrive at the claimed invention.
Appeal 2018-008975
Application 14/231,644

12
TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED IN PART