Disney Enterprises, Inc.

Patent Trials and Appeals Board

Dec 4, 2020

2019004849 (P.T.A.B. Dec. 4, 2020)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
15/202,471 07/05/2016 Atousa Torabi 0260487 8296
63649 7590 12/04/2020
DISNEY ENTERPRISES, INC.
C/O FARJAMI & FARJAMI LLP
26522 LA ALAMEDA AVENUE, SUITE 360
MISSION VIEJO, CA 92691
EXAMINER
HASAN, SYED Y
ART UNIT PAPER NUMBER
2484
NOTIFICATION DATE DELIVERY MODE
12/04/2020 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docketing@farjami.com
farjamidocketing@yahoo.com
ffarjami@farjami.com
PTOL-90A (Rev. 04/07)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte ATOUSA TORABI and LEONID SIGAL
____________

Appeal 2019-004849
Application 15/202,471
Technology Center 2400
____________

Before JOSEPH L. DIXON, MAHSHID D. SAADAT, and
DONNA M. PRAISS, Administrative Patent Judges.

SAADAT, Administrative Patent Judge.

DECISION ON APPEAL

Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the
Examiner’s decision to reject claims 1–20, which are all the claims pending
in this application. We have jurisdiction under 35 U.S.C. § 6(b).
We REVERSE.

1 We use the word “Appellant” to refer to “applicant” as defined in
37 C.F.R. § 1.42(a). Appellant identifies the real party in interest as Disney
Enterprises, Inc., a wholly owned subsidiary of The Walt Disney Company.
Appeal Br. 2.
Appeal 2019-004849
Application 15/202,471

2
STATEMENT OF THE CASE
Introduction
Appellant’s disclosure is directed to methods of annotating video
frames by “determining actions depicted in media contents based on
attention weights of media content frames.” See Spec. 2:2–5.
Claim 1 is illustrative of the invention and reads as follows:
1. A system comprising:
a label database including a plurality of labels;
a non-transitory memory storing an executable code; and
a hardware processor executing the executable code to:
receive a media content including a plurality of
segments, each segment of the plurality of segments
including a plurality of frames;
extract segment data from a segment of the
plurality of segments;
extract frame data from each frame of the plurality
of frames of the segment;
determine an object attention weight, a scene
attention weight, and an action attention weight for each
frame of the plurality of frames of the segment based on
the segment data extracted from the segment and the
frame data extracted from each frame of the plurality of
frames of the segment;
determine that the segment depicts a label of the
plurality of labels in the label database based on the
segment data, the frame data, the object attention weight,
the scene attention weight, and the action attention
weight of each frame of the plurality of frames of the
segment;
tag the segment with the label;
receive a user input from a user device; and
Appeal 2019-004849
Application 15/202,471

3
perform an act on the media content based on the
user input and the label.
Appeal Br. 15 (Claims Appendix).
The Examiner’s Rejections
Claims 1–8 and 11–18 stand rejected under 35 U.S.C. § 102(a)(1) as
being anticipated by Jain (US 2017/0262996 Al; pub. Sept. 14, 2017)2.
Final Act. 8–11.
Claims 9 and 19 stand rejected under 35 U.S.C. § 103 as being
unpatentable over Jain and Toderice (US 2012/0123978 Al; pub. May 17,
2012). Final Act. 12.
Claims 10 and 20 stand rejected under 35 U.S.C. § 103 as being
unpatentable over Jain and Melton (US 2003/0133022 Al; pub. July 17,
2003). Final Act. 13.

ANALYSIS
With respect to claim 1, Appellant contends the Examiner erred in
finding Jain anticipates the claimed subject matter because Jain is not a prior
art to Appellant’s application. Appeal Br. 7. According to Appellant,
The filing date of Jain is August 29, 2016, which is after the filing
date of the present application, July 5, 2016. Appellant
respectfully notes that Jain claims priority to provisional
application serial number 62/307,359, filed March 11, 2016 (the
“Jain Provisional.”) But a review of the Jain Provisional reveals
that paragraphs [0058] and [0076] of Jain do not appear in the
Jain Provisional. For example, there is no mention of “[e]ach
attention feature map of each frame indicates one or more
potential actions, moving objects, and/or events” in the Jain
Provisional. (See paragraph [0076] of Jain.) Also, FIG. 5A of

2 Jain claims priority to the Provisional application No. 62/307,359, filed on
Mar. 11, 2016 (Jain Provisional).
Appeal 2019-004849
Application 15/202,471

4
the Jain Provisional is different than FIG. 5A of Jain. In addition,
the Jain Provisional does not include paragraph [0058] of Jain,
which describes its FIG. 5A and is relied upon by the Office
Action.
Appeal Br. 7–8. Appellant argues that Jain Provisional cannot be used as
prior art because Jain’s paragraphs 58 and 76 do not appear in Jain
Provisional. Appeal Br. 8. Additionally, Appellant argues that the
Examiner has erroneously concluded that the missing disclosure related to
the recited “for each frame of a segment, an object attention weight, a
scene attention weight, and an action attention weight are determined based
on segment data extracted from the segment and the frame data extracted
from the frame” are “implied” in Jain Provisional based on Jain’s disclosure.
Appeal Br. 11.
In response, the Examiner reiterates Jain’s disclosure in paragraphs 58
and 76 to show that Jain discloses “attention weights for object, scene and
action for each frame from the segment.” See Ans. 13–15. With respect to
whether Jain Provisional includes any disclosure that corresponds to Jain’s
disclosure in paragraphs 58 and 76, the Examiner asserts that the paragraph
numbering in Jain and Jain Provisional are different because they are
different documents. Ans. 18. To explain the teachings in Jain Provisional,
the Examiner compares Figure 5A and paragraphs 57, 60, 83, and 85 of Jain
with Figure 5A and paragraphs 49, 52, 82, 83, and 87 of Jain Provisional.
Ans. 19–22.
Based on the Examiner’s mapping of the disputed claim limitation to
Jain’s disclosure and a review of Jain, we observe that Jain’s paragraph 76 is
the most relevant part of the reference as shown below:
In contrast to conventional systems, aspects of the present
disclosure are directed to an attention recurrent neural network
Appeal 2019-004849
Application 15/202,471

5
(RNN) that generates attention feature maps for each frame of a
frame sequence. Each attention feature map of each frame
indicates one or more potential actions, moving objects, and/or
events. The potential actions, moving objects, and/or events may
be referred to as an action proposal. In one configuration, the
attention recurrent neural network is trained on action class
labels provided for an input frame sequence. That is, the attention
recurrent neural network is trained to classify frames based on
an identified action. Furthermore, in one configuration, the
bounding boxes are generated from an attention map that
captures the action. Although described generally with respect
to recurrent neural networks, the present disclosure can employ
a particular type of recurrent neural network, such as a[] long
short-term memory (LSTM) network.
Jain ¶ 76 (emphases added). Therefore, Jain explains that an attention
recurrent neural network (RNN) generates attention feature maps for each
frame that “indicates one or more potential actions, moving objects, and/or
events,” which “may be referred to as an action proposal.” Jain ¶ 76.
However, the Examiner has not sufficiently established that the above-
mentioned teaching is supported by the disclosure in Jain Provisional.
Similar discussion of RNN appears in Jain Provisional’s paragraph 78,
which reads as follows:
In contrast to conventional systems, aspects of the present
disclosure are directed to generating action proposals, such as
bounding-boxes, from an attention recurrent neural network
(RNN) that is trained on action class labels provided for an input
frame sequence. That is, the attention RNN is not trained to
generate action proposals (e.g., draw boxes around specific
action.) Rather, the attention RNN is trained to classify frames
based on identified action and the bounding-boxes are generated
from an attention map that captures the action. Although
Appeal 2019-004849
Application 15/202,471

6
described generally with respect to RNNs, the present disclosure
can employ a particular type of RNN, such as an LSTM network.
Jain Provisional ¶ 78 (emphasis added). Comparing the emphasized
portions of these two paragraphs in Jain with similar description in Jain
Provisional shows the missing disclosure in Jain Provisional (i.e., absence of
any reference to “[e]ach attention feature map of each frame indicates one or
more potential actions, moving objects, and/or events”).
A claim for priority under 35 U.S.C. § 119(e), requires the patent
application claiming priority be “disclosed in the manner provided by
section 112(a) (other than the requirement to disclose the best mode) in [the]
provisional application” to which priority is claimed. “[F]iling a provisional
application in the USPTO that meets the statutory requirements, in effect,
provides a self-authenticating[] instrument establishing a date of disclosure
for the subject matter contained within the” provisional application. Ex
parte Yamaguchi, 88 USPQ2d 1606, 1611 (BPAI 2008) (precedential).
Thus, “[l]ike a regular utility application, the provisional application is
considered prior art for all that it teaches.” Id. at 1612. That is, the filing
date of the provisional application can only be relied on if, given
hypothetical claims encompassing the pertinent subject matter, the
provisional application meets the requirements of 35 U.S.C. §112(a) for the
pertinent subject matter. Cf. In re Wertheim, 646 F.2d 527, 537 (CCPA
1981) (“if a patent could not theoretically have issued the day the application
was filed, it is not entitled to be used against another as ‘secret prior art’”);
see also Dynamic Drinkware, LLC v. Nat’l Graphics, Inc., 800 F.3d 1375,
1381 (Fed. Cir. 2015) (“A reference patent is only entitled to claim the
benefit of the filing date of its provisional application if the disclosure of the
Appeal 2019-004849
Application 15/202,471

7
provisional application provides support for the claims in the reference
patent.”).
Here, for Jain to be considered as having been effectively filed on the
date Jain Provisional was filed—for purposes of teaching or suggesting the
disputed recitations of claim 1—Jain Provisional must describe (providing
written description and enablement support for) the subject matter of the
disputed recitations. MPEP § 2151 (9th Ed., Rev. 10.2019, June 2020) (“a
U.S. patent document is effective as prior art as of the filing date of the
earliest application to which benefit or priority is claimed and which
describes the subject matter relied upon, regardless of whether the earliest
such application is a U.S. provisional or nonprovisional application”).
Therefore, we agree with Appellant that the Examiner’s rejection of claim 1
is in error, because Jain was filed after the present application and, although
Jain claims priority to Jain Provisional, no mapping of the highlighted
portions of Jain’s paragraph 76, or paragraph 58, to the disclosure of its
underlying provisional application appears in the Office Action.
In other words, the Examiner’s findings fail to show that the subject
matter of Jain pertinent to the disputed recitations has written description
and enablement support in Jain Provisional. As such, Jain Provisional does
not provide § 112 written description support for paragraph 76 of the Jain
reference. Additionally, the Examiner’s determination that Jain is
effectively filed as of the filing date of an earlier provisional application
because the missing disclosure is “implied,” is based on speculation and
insufficient to establish that Jain “is entitled” to claim priority to that
provisional application. Thus, the Examiner has not shown that Jain is
properly considered prior art to the instant application. Accordingly, we are
Appeal 2019-004849
Application 15/202,471

8
persuaded the Examiner reversibly erred in rejecting claim 1, independent
claim 11, which recites similar limitations, and claims 2–8 and 12–18
dependent therefrom, which are similarly rejected as anticipated by Jain.
Conclusion
For the above reasons, we do not sustain the rejection of claims 1–8
and 11–18 under 35 U.S.C. § 102(a)(1) as being anticipated by Jain. We
also do not sustain the 35 U.S.C. § 103 rejections of claims 9, 10, 19, and 20
as being unpatentable over Jain in view of Toderice and Melton because the
Examiner has not identified any teachings in the other applied prior art
references to cure the above-identified deficiency of Jain.
DECISION SUMMARY
In summary:
Claims
Rejected
35 U.S.C. § Reference(s)/Basis Affirmed Reversed
1–8, 11–18 102 Jain 1–8, 11–18
9, 19 103 Jain, Toderice 9, 19
10, 20 103 Jain, Melton 10, 20
Overall
Outcome
1–20

REVERSED