MasterCard International Incorporated

Patent Trials and Appeals Board

May 13, 2021

2020000969 (P.T.A.B. May. 13, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/540,446 11/13/2014 Debashis Ghosh 37537.0047U1- 7546
126944 7590 05/13/2021
MasterCard
c/o Ballard Spahr LLP
999 Peachtree Street, Suite 1600
Atlanta, GA 30309-4421
EXAMINER
LI, GRACE Q
ART UNIT PAPER NUMBER
2611
NOTIFICATION DATE DELIVERY MODE
05/13/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
uspatentmail@ballardspahr.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte DEBASHIS GHOSH and RANDALL SHUKEN
____________

Appeal 2020-000969
Application 14/540,446
Technology Center 2600
____________

Before JOHN A. JEFFERY, ELENI MANTIS MERCADER, and
JOHN P. PINKERTON, Administrative Patent Judges.

JEFFERY, Administrative Patent Judge.

DECISION ON APPEAL
Under 35 U.S.C. § 134(a), Appellant1 appeals from the Examiner’s
decision to reject claims 6–8, 10, 13–19, and 22–32. We have jurisdiction
under 35 U.S.C. § 6(b).
We AFFIRM.

1 We use the word “Appellant” to refer to “applicant” as defined in 37
C.F.R. § 1.42. Appellant identifies the real party in interest as Mastercard
International, Incorporated. Appeal Br. 1.
Appeal 2020-000969
Application 14/540,446

2
STATEMENT OF THE CASE
Appellant’s invention collects and aggregates images captured by an
augmented reality device that tags captured images with various metadata,
such as an image creation timestamp and location. See Spec. ¶¶ 6, 65.
Aggregated videos or images that match a search request based on location,
date, and timeframe are retrieved and transmitted to a user. See Spec. ¶¶ 68–
70. The user can reward the person that wore the augmented reality device
that captured the image matched to the search request and transmitted to the
user. See Spec. ¶ 71.
Claim 13 is illustrative:
13. A method comprising:
receiving, at a collection server and via a network interface,
images from a plurality of augmented reality devices, wherein each of
the images is associated with metadata comprising a timestamp
indicating a date and time at which the image was created, a location
at which the image was created, an orientation of the image, and an
identifier of a wearer of one of the plurality of augmented reality
devices;
aggregating, at a collection server, the received images based
on the timestamp and the location of each of the received images;
subsequent to the aggregation, receiving, via the network
interface, a request indicating a requested date, a requested timeframe,
and a requested location;
searching, with a processor, an index of the aggregated images
for one or more images that match the request based on the metadata,
the requested date, the requested timeframe, and the requested
location, resulting in matched images;
retrieving the matched images with the processor;
presenting the matched images to a viewer; and
providing a reward to one or more of the wearers of the
plurality of augmented reality devices based on the identifiers
associated with the one or more matched images.

Appeal 2020-000969
Application 14/540,446

3
THE REJECTIONS
The Examiner rejected claims 13, 15–19, 22, and 24–26 under 35
U.S.C. § 103 as unpatentable over Lee (US 2015/0348131 A1; published
Dec. 3, 2015), Adhikari (US 2012/0113274 A1; published May 10, 2012),
Reiley (US 2013/0326406 A1; published Dec. 5, 2013), and Hunt (US
2015/0327068 A1; published Nov. 12, 2015). Final Act. 2–9.2
The Examiner rejected claims 6–8, 10, 14, and 23 under 35 U.S.C.
§ 103 as unpatentable over Lee, Adhikari, Reiley, Hunt, and Tarumoto (US
2006/0193524 A1; published Aug. 31, 2006). Final Act. 9–12.
The Examiner rejected claim 27 under 35 U.S.C. § 103 as
unpatentable over Lee, Adhikari, Reiley, Hunt, Tarumoto, and Sinha (US
2015/0085159 A1; published Mar. 26, 2015). Final Act. 12–13.
The Examiner rejected claims 29 and 31 under 35 U.S.C. § 103 as
unpatentable over Lee, Adhikari, Reiley, Hunt, and Sinha. Final Act. 13–14.
The Examiner rejected claim 28 under 35 U.S.C. § 103 as
unpatentable over Lee, Adhikari, Reiley, Hunt, Tarumoto, Sinha, and
Rolston (US 2015/0039616 A1; published Feb. 5, 2015). Final Act. 14.
The Examiner rejected claims 30 and 32 under 35 U.S.C. § 103 as
unpatentable over Lee, Adhikari, Reiley, Hunt, Sinha, and Rolston. Final
Act. 15.

2 Throughout this opinion, we refer to (1) the Final Rejection mailed January
17, 2019 (“Final Act.”); (2) the Appeal Brief filed July 17, 2019 (“Appeal
Br.”); (3) the Examiner’s Answer mailed September 19, 2019 (“Ans.”); and
(4) the Reply Brief filed November 19, 2019 (“Reply Br.”).
Appeal 2020-000969
Application 14/540,446

4
THE OBVIOUSNESS REJECTION OVER LEE, ADHIKARI, REILEY,
AND HUNT

Regarding independent claim 13, the Examiner finds that Lee
discloses (1) receiving images from plural devices; (2) associating the
images with metadata comprising a timestamp and location at which the
image was created; (3) aggregating the images based on the timestamp and
location; (4) receiving a request indicating a timeframe and location; (5)
searching the aggregated images for those that match the request based on
the metadata; (6) retrieving the matched images; (7) presenting the matched
images to a user; and (8) providing a reward to a wearer of the device that
captured the matched images based on identifiers associated with the
matched images. Final Act. 2–4. Although the Examiner acknowledges that
Lee’s device is not an augmented reality device, and Lee’s metadata lacks
image orientation information, the Examiner cites Adhikari as teaching these
features. Final Act. 4–5. The Examiner further acknowledges that (1) Lee’s
timestamp metadata is not a date and time at which the image was created at
a location where the image was created, and (2) Lee’s searching of
aggregated images does not use an index, but cites Reiley as teaching these
features. Final Act. 5–6. Finally, the Examiner acknowledges that (1) Lee’s
metadata lacks an identifier of a wearer of the device, and (2) providing an
award based on the identifier, but cites Hunt as teaching these features, in
concluding that the claim would have been obvious. Final Act. 6–7.
Appellant argues that Lee fails to teach aggregating images based on
timestamp and location metadata associated with the images. Appeal Br. 6–
7; Reply Br. 3–4. Appellant adds that the Examiner’s combination of Lee,
Adhikari, Reiley, and Hunt is improper because it (1) changes the
Appeal 2020-000969
Application 14/540,446

5
fundamental operation of Lee (Appeal Br. 7–9; Reply Br. 4–6); (2) is based
on impermissible hindsight reconstruction (Appeal Br. 9–10; Reply Br. 6–7);
and (3) is based on an obviousness rationale that not only fails to consider
the claims as a whole, but is also conclusory and lacks rational underpinning
(Appeal Br. 10–12; Reply Br. 6–7).

ISSUES
I. Under § 103, has the Examiner erred in rejecting claim 13 by
finding that Lee, Adhikari, Reiley, and Hunt collectively would have taught
or suggested aggregating received images based on the timestamp and the
location of each of the received images?
II. Is the Examiner’s proposed combination of the cited references
supported by articulated reasoning with some rational underpinning to
justify the Examiner’s obviousness conclusion? This issue turns on whether
the Examiner’s proposed combination (1) changes Lee’s principle of
operation; (2) is premised on impermissible hindsight reconstruction as it
fails to consider the claims as a whole; and (3) is conclusory and lacks
rational underpinning.

ANALYSIS
We begin by noting that the Examiner’s reliance on Lee to teach (1)
receiving images associated with metadata comprising a timestamp and
location at which the image was created; (2) receiving a request indicating a
timeframe and location; (3) searching the aggregated images for matches to
the request based on the metadata; (4) retrieving the matched images,
presenting the matched images to a user; and (5) providing a reward to a
Appeal 2020-000969
Application 14/540,446

6
wearer of the device from the matched images, is undisputed. See Final Act.
2–4 (citing Lee ¶¶ 10, 55–57, 68, 71, 73, 75, 77, 85; Figs. 2–3)). Nor does
Appellant dispute the Examiner’s reliance on (1) Adhikari for teaching an
augmented reality device (see Final Act. 4–5 (citing Adhikari ¶ 41;
Abstract)); (2) Reiley for teaching date and time metadata and searching an
index of the aggregated images (see Final Act. 5–6 (citing Reiley ¶¶ 11, 47,
62)); and (3) Hunt for teaching an identifier for a wearer of a device and
providing an award based on the identifier (see Final Act. 6–7 (citing Hunt
¶¶ 4, 21)).
Rather, as noted above, this dispute turns solely on (1) the Examiner’s
reliance on Lee for teaching the recited aggregating the received images
based on the timestamp and the location of each of the received images, and
(2) the cited references’ combinability.
On this record, we find no error in the Examiner’s reliance on Lee’s
storage and classifying of uploaded images using space, time, and weather
metadata tagged to the images. See Final Act. 3 (citing Lee ¶¶ 68, 75; Fig.
3)). Appellant’s contention that the claimed aggregating is limited to using
only certain pieces of metadata (see Appeal Br. 6) is not commensurate with
the scope of the claim. The claim merely requires that aggregating is based
on the received images’ timestamp and location, not that aggregating uses
only the timestamp and location. The Examiner determines, and we agree,
that claim 13 does not exclude other metadata from being part of the basis
for aggregating. See Ans. 17. That claim 13’s preamble includes the term
“comprising,” which does not preclude additional unrecited steps,3 and the

3 “‘Comprising’ is a term of art used in claim language which means that the
named elements are essential, but other elements may be added and still
Appeal 2020-000969
Application 14/540,446

7
fact that the claim does not explicitly prohibit using other metadata for the
aggregating, further bolsters the reasonableness of the Examiner’s
construction.
Nor are we persuaded of error in the Examiner’s proposed
combination and rationale to combine the cited references’ teachings in
concluding that the claims would have been obvious. We find unavailing
Appellant’s contention that using Reiley’s indexed content with Lee’s
providing images responsive to an image request using metadata, would
change its principle of operation. See Appeal Br. 7–8; Reply Br. 4–6.
According to Appellant, Reiley teaches methods of predicting which media
items may be of interest to a user and choosing such media items not on the
basis of parameters associated with the request. See Appeal Br. 8. In
supporting this contention, Appellant points to Reiley’s paragraph 61
directed towards “an embodiment” where the content index has associated
with it a level of interest indicating a predicted popularity for a particular
content item (see Reiley ¶ 61), and concludes that the teachings in this
particular embodiment of Reiley purportedly operate in “a completely
different manner” than the request regime of Lee (see Appeal Br. 8–9).
The test for obviousness is not whether a secondary reference’s
features can be bodily incorporated into the structure of the primary
reference. In re Keller, 642 F.2d 413, 425 (CCPA 1981). Rather, the test is
what the combined teachings of the references would have suggested to
those of ordinary skill in the art. Id.

form a construct within the scope of the claim.” Genentech, Inc. v. Chiron
Corp., 112 F.3d 495, 501 (Fed. Cir. 1997) (citation omitted).
Appeal 2020-000969
Application 14/540,446

8
To be sure, one of Reiley’s objectives is to provide personalized
content using a content index. See Reiley ¶ 3; Abstract. To this end,
Reiley’s content index can be updated to predict user interest in particular
content. See Reiley ¶¶ 4, 61. However, regardless of how Reiley’s content
index is updated, to provide personalized content to a user, Reiley teaches in
numerous embodiments—including the embodiment relied upon by the
Examiner (see Final Act. 5–6 (citing Reiley ¶ 62))—that in response to a
content request, media items with metadata that match at least a subset of
the parameters associated with the request are identified. See, e.g., Reiley
¶¶ 11, 13–14, 45, 47, 49, 62–64, 70, 77; Abstract. Further, we agree with the
Examiner that Appellant’s focus on the embodiment disclosed in Reiley’s
paragraph 61 is not germane to the embodiment relied upon in the rejection.
See Ans. 18.
Therefore, we are not persuaded that using Reiley’s content index
with Lee’s image requesting using metadata would significantly change
Lee’s searching functionality, let alone render it unsatisfactory for its
intended purpose.
Nor do we find error in the Examiner’s proposed modification of (1)
Lee’s user terminal device that captures images to include augmented reality
functionality such as that disclosed by Adhikari; or (2) Lee’s metadata
tagging to include metadata of the wearer of one of the devices. See Final
Act. 4–5, 7. Appellant’s contention that the Examiner’s reliance upon Hunt
to modify a feature of Adhikari absent in Lee purportedly constitutes
impermissible hindsight reconstruction is unavailing. Appeal Br. 9–10.
Appellant’s reliance on the non-precedential Board decision, Ex
Appeal 2020-000969
Application 14/540,446

9
parte Stefanic, Appeal No. 2009-007463 (PTAB Aug. 31, 2010) (Appeal Br.
10) is likewise unavailing. Not only do the facts in that case differ from
those at issue here, as a non-precedential decision, Stefanic is not binding on
this panel in any event. See PTAB Standard Operating Procedure 2 (Rev.
10) § I(B), https://www.uspto.gov/sites/default/files/documents/
SOP2%20R10%20FINAL.pdf. Nor has that case been designated as
informative. See Alphabetical Listing of Informative Decisions, USPTO,
https://www.uspto.gov/patents/ptab/decisions-and-opinions/informative-
opinions-0.
To the extent that Appellant contends that modifying a “secondary”
reference versus a “base” reference under a proposed combination of
references somehow implicates impermissible hindsight or is otherwise
prohibited under current obviousness law (see Appeal Br. 9–10), Appellant
cites no binding authority for such a proposition. It is well settled that
where, as here, the factual inquiries underlying an obviousness
determination are otherwise clear, the Examiner’s characterizing cited prior
art references as “primary” and “secondary” is merely a matter of
presentation with no legal significance. In re Mouttet, 686 F.3d 1322, 1333
(Fed. Cir. 2012). Therefore, regardless of the cited references’ order of
presentation in the rejection, we are not persuaded of error in the Examiner’s
rejection that is based on the cited references’ collective teachings.
Nor are we persuaded that the Examiner failed to consider the claims
as a whole or that the Examiner’s rejection is based on impermissible
hindsight reconstruction despite Appellant’s arguments to the contrary. See
Appeal Br. 9–11. Appellant’s arguments pertaining to Reiley’s individual
shortcomings regarding the recited aggregating step (Appeal Br. 11) do not
Appeal 2020-000969
Application 14/540,446

10
show nonobviousness where, as here, the rejection is based on the cited
references’ collective teachings. See In re Merck & Co., 800 F.2d 1091,
1097 (Fed. Cir. 1986).
Finally, we find no error in the Examiner’s articulated reason to
combine the cited references. According to Appellant, claim limitations are
not puzzle pieces to be matched to atomized prior art reference suggestions.
Appeal Br. 11. But as the U.S. Supreme Court in KSR Int’l Co. v. Teleflex
Inc., 550 U.S. 398, 420 (2007) indicates, “familiar items may have obvious
uses beyond their primary purposes, and in many cases a person of ordinary
skill will be able to fit the teachings of multiple patents together like pieces
of a puzzle.”
Appellant’s contentions that the Examiner’s motivation to combine
the cited references is conclusory and lacks rational underpinning because
Lee, Adhikari, and Reiley are ostensibly devoid of any teaching of a
“managed wireless distribution network” taught by Hunt (see Appeal Br.
11–12) are unavailing. Notably, Lee’s image providing system interacts
with user and creator terminals over a network (see Lee ¶ 54), Adhikari’s
tagged videos are uploaded over a WiFi network (see Adhikari ¶ 74), and
Reiley’s client device communicates with a content providing engine over a
network (see Reiley ¶ 43). Leaving aside the fact that the cited prior art is
replete with teachings of network-based functionality as noted above, we
nonetheless find the Examiner’s proposed combination of the cited
references is supported by articulated reasoning with some rational
underpinning to justify the Examiner’s obviousness conclusion.
Appeal 2020-000969
Application 14/540,446

11
Therefore, we are not persuaded that the Examiner erred in rejecting
claim 13, and claims 15–19, 22, and 24–26 not argued separately with
particularity.

THE OTHER OBVIOUSNESS REJECTIONS
We also sustain the Examiner’s obviousness rejection of independent
claim 6. Final Act. 9–10. Notwithstanding that Tarumoto is further cited in
the rejection against claim 6, Appellant states that the reasoning set forth
regarding claim 13 applies to claim 6 as well, as claim 6 recites similar
though not identical recitations as claim 13. See Appeal Br. 12–13; Reply
Br. 7.
Unlike independent claims 13 and 22, independent claim 6 does not
recite an aggregating step. Appellant’s contentions, then, regarding the
recited aggregation made in connection with claim 13 are, therefore, not
commensurate with the scope of claim 6 that lacks this aggregation.
Therefore, we are not persuaded of error in the Examiner’s rejection of claim
6 for this additional reason.
Finally, we also sustain the Examiner’s obviousness rejections of
dependent claims 7, 8, 10, 14, 23, and 27–32. Final Act. 9–15. Despite
nominally arguing these claims separately (Appeal Br. 12–13), we are not
persuaded by these arguments for the reasons previously discussed.

CONCLUSION
In summary:
Appeal 2020-000969
Application 14/540,446

12
Claim(s)
Rejected
35 U.S.C. § Reference(s)/
Basis
Affirmed Reversed
13, 15–19,
22, 24–26
103 Lee, Adhikari,
Reiley, Hunt
13, 15–19, 22,
24–26

6–8, 10,
14, 23
103 Lee, Adhikari,
Reiley, Hunt,
Tarumoto
6–8, 10, 14,
23

27 103 Lee, Adhikari,
Reiley, Hunt,
Tarumoto,
Sinha
27
29, 31 103 Lee, Adhikari,
Reiley, Hunt,
Sinha
29, 31
28 103 Lee, Adhikari,
Reiley, Hunt,
Tarumoto,
Sinha, Rolston
28
30, 32 103 Lee, Adhikari,
Reiley, Hunt,
Sinha, Rolston
30, 32
Overall
Outcome
6–8, 10, 13–
19, 22–32

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv). See 37
C.F.R. § 41.50(f).

AFFIRMED