Gracenote, Inc.

Patent Trials and Appeals Board

Jun 15, 2021

IPR2020-00219 (P.T.A.B. Jun. 15, 2021)

Trials@uspto.gov Paper 36
571-272-7822 Date: June 15, 2021

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

FREE STREAM MEDIA CORP.
d/b/a SAMBA TV,
Petitioner

v.

GRACENOTE, INC.,
Patent Owner.
____________

IPR2020-00219
Patent 8,171,030 B2
____________

Before MICHELLE N. ANKENBRAND, GARTH D. BAER,
and AARON W. MOORE, Administrative Patent Judges.

MOORE, Administrative Patent Judge.

JUDGMENT
Final Written Decision
Determining No Claims Unpatentable
Granting as to Cancelation and Denying as to Substitution Patent Owner’s
Revised Motion to Amend
35 U.S.C. § 318(a)
IPR2020-00219
Patent 8,171,030 B2

1
TABLE OF CONTENTS
I. INTRODUCTION ................................................................................ 3
A. Background ................................................................................ 3
B. Related Matters .......................................................................... 4
C. The ’030 Patent .......................................................................... 4
D. The Challenged Claims .............................................................. 5
II. LEVEL OF ORDINARY SKILL IN THE ART .................................. 6
III. CLAIM CONSTRUCTION ................................................................. 7
A. “robust hash” .............................................................................. 8
B. “traversal index” ......................................................................... 8
C. “leaf node” .................................................................................. 9
IV. ORIGINAL CLAIMS ........................................................................... 9
A. Evidence Relied Upon ................................................................ 9
B. Legal Standard ......................................................................... 10
C. Obviousness of Claims 6 and 10 .............................................. 11
1. Claim 1 ........................................................................... 12
a. “method of organization of a multi-
dimensional video database using a
robust hash of a multi-dimensional
vector signature as a traversal index” .................. 12
b. “generation of a robust hash value as
a traversal index from multiple
parameters extracted from a region
of interest in a frame of a video sequence” ......... 13
c. “storing data associated with the video
sequence at a leaf node addressed by the
robust hash value, wherein the leaf node
is a member of a plurality of leaf nodes
in a multi-dimensional video database” .............. 15
2. Claim 6 ........................................................................... 17
IPR2020-00219
Patent 8,171,030 B2

2
3. Claim 10 ......................................................................... 18
D. Our Analysis ............................................................................. 18
V. MOTION TO AMEND ...................................................................... 23
A. Applicable Law ........................................................................ 23
B. Procedural Background ............................................................ 24
C. Proposed Substitute Claim ....................................................... 25
D. Statutory and Procedural Requirements ................................... 26
1. Number of Substitute Claims ......................................... 26
2. Claim Scope ................................................................... 26
3. Responsiveness to a Ground of Unpatentability ............ 27
4. New Matter .................................................................... 27
a. 35 U.S.C. § 316(d)(3) .......................................... 28
b. Simultaneously “Traversing” a Tree and
“Addressing” a Leaf Node .................................. 29
c. “Storing the Traversal Index, Multiple
Signatures Associated with the Video
Sequence, and Data Associated with
theVideo Sequence at a Leaf Node
Addressed by the Robust Hash Value” ............... 34
5. Conclusion ..................................................................... 37
VI. CONSTITUTIONALITY ................................................................... 38
VII. CONCLUSION .................................................................................. 39
VIII. ORDER ............................................................................................... 40
IPR2020-00219
Patent 8,171,030 B2

3
I. INTRODUCTION
A. Background
Free Stream Media Corp. d/b/a Samba TV (“Petitioner”) filed a
Petition requesting inter partes review of claims 1, 6, 8, 10, 11, 13, and 15 of
U.S. Patent No. 8,171,030 B2 (Ex. 1001, “the ’030 patent”). Paper 1
(“Pet.”). Gracenote, Inc. (“Patent Owner”) filed a Preliminary Response.
Paper 10. On June 17, 2020, we instituted an inter partes review of claims
1, 6, 8, 10, 11, 13, and 15. Paper 11 (“Decision”), 22.
The following substantive papers have since been filed by the parties
or entered by the Board:
Paper Title Abbreviation
16 Patent Owner’s Motion to Amend MTA
17 Patent Owner’s Response PO Resp.
21 [Petitioner’s] Opposition to Patent Owner’s
Motion to Amend
MTA Opp.
22 Preliminary Guidance PG
23 [Petitioner’s] Reply in Support of its Petition
for Inter Partes Review
Pet. Reply
24 Patent Owner’s Revised Motion to Amend RMTA
27 [Patent Owner’s] Sur-Reply PO Sur-Reply
28 [Petitioner’s] Opposition to Patent Owner’s
Revised Motion to Amend
RMTA Opp.
30 Patent Owner’s Reply to Petitioner’s
Opposition to Revised Motion to Amend
RMTA Reply
33 [Petitioner’s] Sur-Reply in Further Opposition
to Patent Owner’s Revised Motion to Amend
RMTA Sur-
Reply

IPR2020-00219
Patent 8,171,030 B2

4
An oral hearing was held on April 14, 2020, by video only, and a
transcript of the hearing is included in the record. Paper 35 (“Tr.”). The
demonstratives used in the oral hearing are in the record as Paper 32 (Patent
Owner) and Paper 34 (Petitioner).
The Board has jurisdiction under 35 U.S.C. § 6. This Final Written
Decision is issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73.
For the reasons that follow, we determine that Petitioner has not
shown claims 6 and 10 to be unpatentable. Further, we grant Patent Owner’s
Revised Motion to Amend as to canceling original claims 1, 8, 11, 13, and
15, but deny that motion as to entry of proposed substitute claim 28.
B. Related Matters
The parties identify Gracenote, Inc. v. Free Stream Media Corp. d/b/a
Samba TV, No. 1:18-cv-01608-RGA (D. Del.), filed October 17, 2018, as a
related matter. See Pet. 4; Patent Owner Mandatory Notices (Paper 5).
C. The ’030 Patent
The ’030 patent is titled “Method and Apparatus for Multi-
Dimensional Content Search and Video Identification” and describes the
invention as “generally relat[ing] to information retrieval systems including
systems related to complex objects, multi-dimensional data, rich media, and
video.” Ex. 1001, 1:20–23.
A first search method described in the patent uses “a multi-
dimensional object or video database with traversal nodes constructed from
differentiating parts of traversal indexes and leaf nodes storing associated
data and associated indexes.” This structure is illustrated in Figure 1A,
reproduced below.
IPR2020-00219
Patent 8,171,030 B2

5

Figure 1A “illustrates a hierarchical representation of a
multidimensional . . . database with traversal nodes . . . and . . . leaf nodes
storing associated data and associated indexes.” Ex. 1001, 3:14–16.
This method features “a traversal index associated with elements, such
as video sequences 104, stored in a multi-dimensional video database 106,
and leaf nodes 110.” Ex. 1001, 5:8–10. “The video database is traversed via
nodes 108 to reach the leaf nodes 110 storing associated data and associated
indexes” and “[t]he traversal indexes store differentiating information
between different video frames.” Id. at 5:10–14.
An “alternate method of indexing avoids tree traversal altogether and
accesses the leaf node directly . . . by generating a direct address or hash for
each leaf node.” Ex. 1001, 5:17–19. In this method, “the traversal indexes
are a direct address or a hash to each leaf node” and are “equivalent to the
hash or direct address of the leaf node.” Id. at 5:20–23.
D. The Challenged Claims
In the Revised Motion to Amend, Patent Owner non-contingently
requests to cancel claims 1, 8, 11, 13, and 15 (see RMTA, 1), which we
IPR2020-00219
Patent 8,171,030 B2

6
grant, leaving claims 6 and 10 as the only challenged claims remaining in
this proceeding. These claims, as well as claim 1, from which they both
directly depend, are reproduced below.
1. A method of organization of a multi-dimensional video
database using a robust hash of a multi-dimensional vector
signature as a traversal index, the method comprising:
generation of a robust hash value as a traversal index from
multiple parameters extracted from a region of interest in a
frame of a video sequence; and
storing data associated with the video sequence at a leaf node
addressed by the robust hash value, wherein the leaf node is a
member of a plurality of leaf nodes in a multi-dimensional
video database.
6. The method of claim 1 wherein the traversal index is
generated from a shape signature of a region around a selected
object in the frame of the video sequence.
10. The method of claim 1 wherein the traversal index is a
multi-dimensional vector comprising a single bit or a few bits
per each dimension of the multi-dimensional vector.
Ex. 1001, 15:29–38, 15:57–59, 16:9–11.
II. LEVEL OF ORDINARY SKILL IN THE ART
The level of skill in the art is a factual determination that provides a
primary guarantee of objectivity in an obviousness analysis. See Al-Site
Corp. v. VSI Int’l Inc., 174 F.3d 1308, 1323 (Fed. Cir. 1999) (citing Graham
v. John Deere Co., 383 U.S. 1, 17–18 (1966)). The level of skill in the art
also informs the claim construction analysis. See Teva Pharms. USA, Inc. v.
Sandoz, Inc., 574 U.S. 318, 332 (2015) (explaining that claim construction
seeks the meaning “a skilled artisan would ascribe” to the claim term “in the
context of the specific patent claim”).
IPR2020-00219
Patent 8,171,030 B2

7
Petitioner asserts that a person of ordinary skill in the art (“POSITA”)
at the time of the invention “would have had a degree in computer
engineering, computer science, information systems, or a similar discipline,
along with 3-4 years of experience with the design and/or implementation of
video or other content identification software and associated databases” and
“would [have] be[en] aware of and generally knowledgeable about different
methods employed to generate video signatures, hashing algorithms, and
various database structures that can be used to store videos and related data.”
Pet. 25 (citing Ex. 1003 ¶¶ 37–40). Patent Owner does not contest
Petitioner’s description.
Because there is no dispute, and because we find Petitioner’s
characterization of the level of skill in the art generally consistent with the
subject matter of the ’030 patent and cited references, we adopt it for
purposes of this analysis.
III. CLAIM CONSTRUCTION
We construe claims using the claim construction standard that would
be used in a civil action under 35 U.S.C. § 282(b), as articulated by the
Federal Circuit in Phillips v. AWH Corp., 415 F.3d 1303 (Fed. Cir. 2005)
(en banc), and subsequent cases. See 37 C.F.R. § 42.100(b).
The parties identify for construction the terms “robust hash,”
“traversal index,” “leaf node,” “addressed by,” and “around.” See Pet. 10–
13; PO Resp. 30–39. Because our resolution of this case does not require us
to resolve the disputes over “addressed by” and “around,” we need not, and
therefore do not, interpret those terms. See Nidec Motor Corp. v. Zhongshan
Broad Ocean Motor Co. Ltd., 868 F.3d 1013, 1017 (Fed. Cir. 2017)
IPR2020-00219
Patent 8,171,030 B2

8
(explaining that construction is needed only for disputed terms needed to
resolve the controversy). We consider the other terms below.
A. “robust hash”
Petitioner argues that a “robust hash” is a “corresponding smaller
numerical value calculated from another value such that it is more immune
to noise and disturbances due to distortion.” Pet. 11. According to
Petitioner, “hash” is “a well-known term in the computer field” and “those
of ordinary skill in the art at the relevant time would [have] consider[ed] a
‘hash’ to be ‘robust’ if a similar value [was] output by the hashing function
even if nonsignificant changes, like compression, rescaling, cropping, and
the like, are made to the input value.” Id.
At institution, we construed this term to mean “a corresponding
smaller numerical value calculated from another value, or set of values, so as
to provide some immunity to noise and disturbances due to distortion.”
Decision 11. Patent Owner agreed with our construction, with the proviso
that “smaller” means “fewer bits” (see PO Resp. 30), and Petitioner did not
address the matter further. “Smaller” does mean “fewer bits” in this context,
and we maintain our earlier interpretation of this term.
B. “traversal index”
The parties agree that traversal index means “an index for stepping or
traversing through the nodes of a hierarchical database.” See Pet. 12; PO
Resp. 30–31. Because we find that definition consistent with the disclosures
of the ’030 patent and the prior art, we adopt it.
IPR2020-00219
Patent 8,171,030 B2

9
C. “leaf node”
Petitioner argues that “leaf node” means “a node in a hierarchical data
structure without any children.” Pet. 13. Patent Owner does not address this
issue. As we did at institution, we find Petitioner’s construction of “leaf
node” consistent with how the term is used in the ’030 patent (see, e.g.,
Ex. 1001, 5:8–10, Fig. 1A), and we thus maintain the construction.
IV. ORIGINAL CLAIMS
We address remaining original claims 6 and 10 before turning to
Patent Owner’s Revised Motion to Amend.
A. Evidence Relied Upon
Because we granted Patent Owner’s request to cancel claims 1, 8, 11,
13, and 15, only the following references are still being asserted in this
proceeding:
Reference Exhibit
WO 2005/036877A1 (“Deng”) 1006
US 2002/0126872A1 (“Brunk”) 1007
Deng is an international patent application, titled “Digital Video
Signature Apparatus and Methods for Use with Video Program
Identification Systems,” describing “[s]ystems that identify video images
and/or sequences of video images.” Ex. 1006 ¶ 2. The system “extract[s]
audio and/or video information associated with the program . . . and
process[es] that extracted information to generate audio and/or video
signatures” that “are substantially unique to each portion of [the]
audio/video content or program.” Id. ¶ 2.
IPR2020-00219
Patent 8,171,030 B2

10
In the embodiment Petitioner cites, Deng uses “a set of tables
organized in a linked tree-like data structure,” where “each of the tables may
be indexed using centroids or coordinate pairs (e.g., horizontal and vertical
coordinates).” Ex. 1006 ¶ 106. “[A] first coordinate pair or centroid
associated with a signature is used to index to a link to a subsequent table,”
“[t]he next coordinate pair of the signature is then used to index within the
subsequent table to another table,” and the “process continues until all
coordinate pairs associated with all of the signatures within a signature
sequence have been exhausted at a final table.” Id. “The last coordinate pair
is then used to index to textual information . . . describing the video program
associated with the sequence of signatures information (i.e., sequence of
centroids or coordinate pairs used to index through the linked tables).” Id.
Deng also explains that its signatures are created to avoid errors from
differences associated with cropping, padding, scaling, shifting, or the like
(see Ex. 1006 ¶¶ 61, 67, 75, 78), and thus, are “robust” as that term is used
in the art.
Petitioner cites Brunk for the concept that a signature can be subjected
to a hashing algorithm to generate a hash. See Ex. 1007 ¶¶ 20–21.
Petitioner also relies on a Declaration of Borko Furht, filed as Exhibit
1003. Patent Owner relies on Declarations of Pierre Moulin, filed as
Exhibits 2005 and 2012.
B. Legal Standard
A claim is unpatentable under 35 U.S.C. § 103(a) if the differences
between the claimed subject matter and the prior art are such that the subject
matter, as a whole, would have been obvious at the time the invention was
made to a person having ordinary skill in the art to which said subject matter
IPR2020-00219
Patent 8,171,030 B2

11
pertains. KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 406 (2007). The
question of obviousness is resolved on the basis of underlying factual
determinations including (1) the scope and content of the prior art; (2) any
differences between the claimed subject matter and the prior art; (3) the level
of ordinary skill in the art; and (4) any secondary considerations, if in
evidence.1 Graham v. John Deere Co., 383 U.S. 1, 17–18 (1966).
C. Obviousness of Claims 6 and 10
Petitioner argues that claims 6 and 10 would have been obvious in
view of Deng and Brunk. Because claims 6 and 10 depend from claim 1
(which Patent Owner has canceled) and include all of the limitations of
claim 1, we also discuss the limitations of claim 1.
For claim 1, Petitioner argues that “Deng discloses almost all of the
limitations” and that “Brunk makes clear that Deng’s already robust
signature can be hashed.” Pet. 45. For claim 6, Petitioner points to Brunk’s
use of edges and distances, and, for claim 10, Petitioner contends that
Brunk’s signature is multi-dimensional. See id. at 55–57.
More specifically, Petitioner reads the claims on Deng and Brunk as
follows:

1 Patent Owner does not present any objective evidence of nonobviousness
(i.e., secondary considerations) as to any of the challenged claims.
IPR2020-00219
Patent 8,171,030 B2

12
1. Claim 1
a. “method of organization of a multi-dimensional
video database using a robust hash of a multi-
dimensional vector signature as a traversal index”
Petitioner argues that Deng “relates to ‘[s]ystems that identify video
images and/or sequences of video images (e.g., television commercials or
programs) being broadcast and/or viewed on an output device (e.g., a
television or video monitor)’” and that “these videos can be stored in a
multi-dimensional database.” Pet. 45 (citing Ex. 1006 ¶ 2). In particular,
Petitioner points to Deng’s data structure that employs “a set of tables
organized in a linked tree-like data structure.” Id. (citing Ex. 1006 ¶ 106).
Petitioner contends that “[t]his ‘tree-like’ database is hierarchical, and
includes a series of nested ‘tables’ with the ‘final table’—the database’s leaf
nodes—including information relating to video programs.” Id. at 45–46.
Petitioner argues that “Deng also employs a ‘multi-dimensional vector
signature’” and that it “works by ‘extracting audio and/or video information
associated with the program . . . and processing that extracted information to
generate audio and/or video signatures.” Pet. 46 (citing Ex. 1006 ¶ 2). This
results in signatures that are “digital sequences or codes that, at a given
instant of time, are substantially unique to each portion of audio/video
content or program.” Ex. 1006 ¶ 2.
Petitioner argues that the multidimensional signatures Deng generates
are intended for use as a database traversal index, as it states that “an
unidentified video program can be reliably identified by finding a matching
signature within a database or library containing the signatures of known
available programs.” Pet. 46 (citing Ex. 1006 ¶¶ 2, 18, 106).
IPR2020-00219
Patent 8,171,030 B2

13
Petitioner further argues that Deng’s “signatures are designed with
‘robustness’ and that this “reduc[es] or eliminat[es] the effects of image
scaling, shifting, etc.” on the searching. Pet. 47 (citing and quoting Ex. 1006
¶¶ 60, 61, 63, 67, 75, 78.)
Petitioner acknowledges that “Deng does not explicitly state that its
signatures are subject to a hashing algorithm to generate a ‘robust hash,’”
but argues that “as of the ’030 patent’s effective filing date the use of robust
hashes as video signatures was well known in the art” and that “Brunk
provides an example.” Pet. 47. Petitioner argues that in Brunk, “[a]
signature for a video can be generated using multiple ‘relevant features (e.g.,
perceptual relevant features represented via edges; magnitude peaks,
frequency characteristics, etc.) identified’ on ‘sets’ such as ‘frames.’” Id.
(citing Ex. 1007 ¶¶ 22–24, 42). “Then, a ‘hash (e.g., a manipulation which
represents the content item . . . as an item having fewer bits)’ can be applied
to the signature to produce a ‘unique identifier.’” Id. at 47–48 (citing
Ex. 1007 ¶¶ 5, 20–21, 29, 31, 46, 49). Petitioner asserts that “Brunk
explains that this hash can be designed so as to be ‘robust.’” Id. at 48 (citing
Ex. 1007 ¶ 5; Ex. 1003 (Furht) ¶¶ 284–292).
b. “generation of a robust hash value as a traversal
index from multiple parameters extracted from
a region of interest in a frame of a video sequence”
As explained above, Petitioner argues that Deng discloses the
extraction of “multiple parameters” from “a region of interest in a frame of a
video sequence.” Petitioner argues that Deng may use selected frames of a
video, may use only portions of the frames, and that each of the images or
IPR2020-00219
Patent 8,171,030 B2

14
frames may also be recursively sub-divided into a plurality of sub-regions or
areas. See Pet. 48–49 (citing Ex. 1006 ¶¶ 39, 46, 49, 61, 63, 85, 88).
In Deng, after the frames or portions of frames have been identified,
the “center of mass or centroid information for each of the brightness and
color components” is calculated to generate a “substantially unique[] . . .
signature.” Ex. 1006 ¶ 39. “The data acquisition unit 206 . . . calculat[es]
the centroids or centers of mass for each of the image color . . . and
brightness components.” Id. ¶ 63. For example “a signature ‘S’ for an
image could be defined as S = (Redx, Redy, Greenx, Greeny, Bluex, Bluey,
Yellowx, Yellowy, Brightnessx, Brightnessy), where Redx is the horizontal
position of the centroid from the color red, Redy is the vertical position of
the color red, etc.” Id. ¶ 71; see id. ¶¶ 72–74.
Petitioner argues that “Deng also teaches that its multidimensional
signatures can form a database ‘traversal index’” in that, “[a]fter signatures
are generated, a ‘central processing unit 210 . . . compare[s] the sequence of
signatures . . . to a plurality of known or reference signatures that are
associated with known video programs and which are stored within a data
structure (e.g., a table).’” Pet. 50–51 (quoting Ex. 1006 ¶ 42). Petitioner
argues that “the ‘set of tables’ forming the ‘tree-like data structure’ can be
‘indexed using centroids or coordinate pairs (e.g., horizontal and vertical
coordinates)’ that underlie the signatures. Id. at 51 (quoting Ex. 1006
¶ 106). “[A] first coordinate pair or centroid associated with a signature is
used to index to a link to a subsequent table,” “[t]he next coordinate pair of
the signature is then used to index within the subsequent table to another
table,” and “[t]his process continues until all coordinate pairs associated with
all of the signatures within a signature sequence have been exhausted at a
IPR2020-00219
Patent 8,171,030 B2

15
final table” and a signature match is identified. Ex. 1006 ¶ 106. Petitioner
contends that “the extracted values making up a given signature are used to
step or traverse through a hierarchical database by comparing those values to
reference signature values stored in nested indexing tables” and that “[t]his
nested ‘set of tables,’ and the signature data it contains, is the claimed
‘traversal index.’” Pet. 51 (citing Ex. 1003 (Furht) ¶¶ 303–308).
Petitioner also argues that Deng’s signatures are “robust” because it
uses weighting and “normalized centroid information” to “reduc[e] or
eliminat[e] the effects of image scaling, shifting, etc.” Pet. 51–52 (citing
Ex. 1006 ¶¶ 60–61, 63, 67, 75, 78).
For hashing, Petitioner cites Brunk’s explanation that a “‘hash (e.g., a
manipulation which represents the content item . . . as an item having fewer
bits)’ can be applied to the signature to produce a ‘unique identifier.’”
Pet. 52 (citing Ex. 1007 ¶¶ 5, 20–21, 29, 31, 46, 49). According to
Petitioner, “just like Deng, Brunk explains that a hashed ‘robust signature’
can be used to identify video in a database by ‘match[ing]’ a query signature
‘to . . . corresponding signatures stored in the database.’” Id. (citing
Ex. 1007 ¶¶ 6, 32).
c. “storing data associated with the video sequence at
a leaf node addressed by the robust hash value,
wherein the leaf node is a member of a plurality of
leaf nodes in a multi-dimensional video database”
According to Petitioner, “Deng explains that video can be stored in a
multidimensional ‘searchable tree-like data structure’” and that “[a] POSITA
would have understood this to be referring to well known, hierarchical
databases.” Pet. 53 (citing Ex. 1006 ¶ 106; Ex. 1003 (Furht) ¶¶ 321–324).
Petitioner contends that “[s]uch a database includes a root node, multiple
IPR2020-00219
Patent 8,171,030 B2

16
intermediate nodes below this root node, and multiple child or leaf nodes
associated with each intermediate node.” Id. (citing Ex. 1003 (Furht)
¶¶ 321–324). Petitioner describes Deng as “explain[ing] that its ‘tree-like’
database is hierarchical in that it includes a nested ‘set of tables’ that are
‘indexed’ and ‘linked’ with each other. Id. (citing Ex. 1006 ¶ 106).
Petitioner also argues that Deng’s database “includes a ‘final table’ of
signatures” and that “[t]his ‘final table’ includes a series of signatures
against which a query signature can be compared to identify a ‘video
program associated with the sequence of signature information’ that forms
the query signature.” Id. Petitioner contends that “[t]he entries in this ‘final
table,’ which do not include any children and are used to generate search
results, are the claimed ‘leaf nodes’ of Deng’s multidimensional database.”
Id.
According to Petitioner, “Deng explains that its system arrives at this
‘final table’ using video signatures as an addressing ‘index,’” as “the various
‘coordinate pair[s]’ making up the signatures are used to traverse through the
hierarchical nested levels of the database, [to] arrive at the ‘final table,’ and
identify ‘the video program associated with the sequence signature
information.’” Pet. 54. Petitioner argues that “Deng’s signature
‘address[es]’ its leaf nodes (the entries of its final table) as required by claim
1.” Id. (citing Ex. 1003 (Furht) ¶ 325).
Petitioner further argues that “Deng’s leaf nodes . . . are also designed
to ‘stor[e] data associated with the video sequence,’” in that “‘[t]he last
coordinate pair’ used to match an entry in the ‘final table’ of its database
‘is . . . used to index to textual information (e.g., in the form of metadata)
describing the video program associated with the sequence of signature
IPR2020-00219
Patent 8,171,030 B2

17
information.’” Pet. 54 (quoting Ex. 1006 ¶ 106). Petitioner asserts that
“[t]his ‘textual information’ and ‘metadata’ is the claimed ‘data associated
with the video sequence.’” Id. (citing Ex. 1003 ¶¶ 326–327.)
2. Claim 6
Claim 6 recites that the “traversal index” is “generated from a shape
signature of a region around a selected object in the frame of the video
sequence.”
Petitioner argues that Deng describes how “video signatures are
generated by ‘extracting . . . information’ from the video,” as well as “the
use of the ‘centroids’ of the various color and brightness components
making up a video frame to generate a signature.” Pet. 55 (citing Ex. 1006,
¶¶ 2, 39, 46, 61.) Petitioner further argues that “[w]hile Brunk also
recognizes that color centroids can be used to generate a signature, it
discusses a number of other alternatives,” including how “edges can . . . be
used to calculate an object’s center of mass” and “[a] distance … may be
calculated form the center of mass to each edge, or to a subset of edges, and
such dimensions may be used as a signature for the object or for the frame.”
Id. at 54–55 (citing Ex. 1007 ¶¶ 24, 42). According to Petitioner, “[t]his is
the claimed ‘shape signature.’” Id. at 55 (citing Ex. 1003 (Furht) ¶¶ 330–
335.)
Petitioner argues that because “Brunk identifies . . . a shape-based
signature as an alternative to a color centroid signature, a POSITA would
have considered it highly obvious to use such a signature with Deng” and,
“in view of Brunk’s disclosure . . . would have had a more than reasonable
expectation that a shape signature could be used with Deng.” Pet. 56 (citing
Ex. 1003 (Furht) ¶¶ 336–338).
IPR2020-00219
Patent 8,171,030 B2

18
3. Claim 10
Claim 10 recites that the “traversal index” is a “multi-dimensional
vector comprising a single bit or a few bits per each dimension of the multi-
dimensional vector.”
Petitioner contends that Brunk’s “signature may be ‘an N-bit hash of
N real numbers, where N is an integer,’” that “[t]he N real numbers may
correspond to (or represent) . . . a data set,” and that “[a] POSITA would
have understood that this ‘data set’ includes the multiple features making up
Brunk’s hash.” Pet. 57 (citing Ex. 1007 ¶ 73; Ex. 1003 (Furht) ¶¶ 343–345).
Petitioner argues that “each bit making up Brunk’s N-bit signature
corresponds to one of the signature features or dimensions.” Id. Petitioner
additionally argues that “a signature can include multiple smaller
‘sequence[s],’ such as a ‘1-bit hash’ or larger sequences like ‘4-bit hashes, 8-
bit hashes, etc.,’” and that “[e]ach of [those] ‘sequences’ is another
dimension of Brunk’s signature.” Id. (citing Ex. 1007 ¶ 100; Ex. 1003
(Furht) ¶¶ 346–347).
D. Our Analysis
We expressed concerns about the combination of Deng and Brunk in
the Institution Decision. We noted that “Petitioner specifically relies on the
Deng’s ‘linked tree-like data structure,’ in which the system steps through
linked tables using individual pairs of the query signature,” but questioned
“how this could work if all of the pairs are hashed into a single value.”
Decision 20.
Patent Owner focuses on that issue, arguing that “[i]f Deng’s
signature sequence were to be hashed, the resulting single hash value would
not be usable to index into Deng’s nested tables because the nested tables
IPR2020-00219
Patent 8,171,030 B2

19
require individual coordinate pairs to index into a table to retrieve a pointer
to a subsequent table until a final table is reached.” PO Resp. 40. Patent
Owner further argues that “[h]ashing Deng’s signature sequence into a
single result value would destroy the . . . ordering and the hash result value
would not be operable to navigate through the nested table ‘database’ of
Deng.” Id. at 45.
Petitioner responds that the combination is viable because one could
use a “robust hash [that is] specifically meant and designed to preserve the
ordering and other data present in a signature—albeit in reduced or
compressed form—to maintain its ‘discriminating capabilities.’” Reply 10
(emphasis omitted). Petitioner contends that “[b]ecause a ‘robust hash’
preserves a signature’s ‘discriminating capabilities,’ a POSITA would also
have understood that a robust hash of Deng’s signature could be used to step
through Deng’s tree-like linked tables in exactly the same way as a
signature.” Id. According to Petitioner, “[a] ‘hash’ is simply a string of
letters and numbers” and “[t]hus, hashing Deng’s signatures amounts to
nothing more than a change in data form” where “[t]he signature function is
not impacted.” Id.
Petitioner then suggests that Deng’s “signature can be hashed by, for
instance, transforming the comma separated list into a string of letters and
numbers that similarly represents the centroid coordinates:
‘RxRyGxGyBxByYxYyBrxBry.’” Reply 11.
Petitioner further argues that “[t]he size of the hash relative to the
signature [could] be decreased by reducing the number of unique values in
the x and y directions (i.e., if the signature employs an x coordinate with
numerical values ranging from 0-100, the hash can employ a coordinate that
IPR2020-00219
Patent 8,171,030 B2

20
covers the same distance but does so using only numerical values ranging
from 0-50)” and that “Deng is expressly open to signatures—like this hash—
that employ less data.” Reply 11. Petitioner argues that “[b]ecause it is
robust and not cryptographic, this hash preserves the ordering and data
present in the original signature.” Id.
Petitioner alternatively argues that “rather than employing a more
complicated string of letters and numbers, Deng’s signature could be hashed
into a series of 0s and 1s.” Reply 11. In this scenario, “the locations of the
frame’s red, green, blue, yellow, and brightness centroids could be used to
generate a frame identifying hash” where “[e]ach portion of the hash
corresponds to one centroid,” resulting in a “ten-bit hash of ‘0010010011.’”
Id. at 12. Petitioner illustrates traversing “Deng’s linked tables using the
complete hash ‘0010010011’” on page 13 of the Reply, as follows:

Petitioner’s Representation of its “0s and 1s” Example
We find Petitioner’s arguments unpersuasive.
First, we do not agree that, because a hash is intended to “preserve a
signature’s discriminating capabilities,” “[t]he signature function is not
impacted.” Reply 10. The “discriminating capabilities” that are preserved
are those relative to the original material to which the signature relates (i.e.,
the hashed signature generally still identifies the original material from
IPR2020-00219
Patent 8,171,030 B2

21
which the signature was derived), not between the signature and the hash of
the signature. Petitioner cites no evidence suggesting that a hash of the
multiple individual signatures of Deng––cryptographic, robust, or
otherwise––would preserve the ability to step through the coordinate pairs as
Deng describes.
Second, we do not agree that Deng’s “signature can be hashed by, for
instance, transforming the comma separated list into a string of letters and
similarly represents the centroid coordinates.” Reply 11. This does not
appear to be a “hash” because it does not appear to reduce the size of the list
(which of course does not include commas when represented in the
computer), and Petitioner does not explain otherwise. Nor does Petitioner
explain how this would amount to “generation of a robust hash value as a
traversal index from multiple parameters,” as claim 1 requires, because
Petitioner does not explain how the “string of letters” would be “a” robust
hash value.
Third, we do not agree with the argument that one could prepare a
signature with values in the range of 1–100 and then “hash it” by reducing
the value ranges to 1–50. See Reply 11. Petitioner again does not explain
how this would reduce the bits required to represent the values, does not
explain why one would do that instead of just starting with values in the 1–
50 range (suggesting the improper influence of hindsight2), and also does not
explain how that would amount to generation of “a” robust hash value. It

2 See St. Jude Med., Inc. v. Access Closure, Inc., 729 F.3d 1369, 1381 (Fed.
Cir. 2013) (“Even under our ‘expansive and flexible’ obviousness
analysis . . . we must guard against ‘hindsight bias’ and ‘ex post
reasoning.’”) (citing KSR, 550 U.S. at 415, 421).
IPR2020-00219
Patent 8,171,030 B2

22
appears that this would instead use a set of individual values ranging from 1
to 50.
Finally, we also do not agree with Petitioner’s “0s and 1s” example,
which does not appear to include hashing at all, but is instead an entirely
different approach in which the centroids are identified only by the quadrant
in which they appear, reducing the resolution substantially. See Reply 11–
13. Petitioner does not explain how this amounts to a “hash,” and it appears
instead that “0010010011” is simply an unhashed signature obtained using a
method slightly different than what is described in Deng. In order to
correspond to what is claimed, one would need to hash “0010010011” to
some smaller value, in which case it is unclear how the traversal would or
could operate.
The question is whether “a skilled artisan would have been motivated
to combine the teachings of the prior art references to achieve the claimed
invention.” Allied Erecting & Dismantling Co., Inc. v. Genesis Attachments,
LLC, 825 F.3d 1373, 1381 (Fed. Cir. 2016) (quoting Pfizer, Inc. v. Apotex,
Inc., 480 F.3d 1348, 1361 (Fed. Cir. 2007)). Absent a showing of how
Brunk’s signature hash teachings could be combined with Deng’s method
without destroying Deng’s method, we find that Petitioner has not
established that one of skill in the art would have been motivated to combine
these references as described in the Petition. We thus conclude that
Petitioner has not shown claims 6 and 10, which include all of the
limitations of claim 1, to be unpatentable in view of those references.
We need not, and therefore do not, reach Patent Owner’s arguments
that the “decrease in resolution would be contrary to the teachings of Deng,
that “there would be little benefit” to the combination “because Deng’s
IPR2020-00219
Patent 8,171,030 B2

23
signature was already a relatively small amount of data,” that Deng’s leaf
nodes would not be “addressed by the robust hash value,” or that claims 6
and 10 are separately patentable over Deng and Brunk. See PO Resp. 47–
62, 66–75.
V. MOTION TO AMEND
Patent Owner filed a Non-Contingent Motion to Amend and then a
Revised Non-Contingent Motion to Amend, both requesting to cancel claims
1, 8, 11, 13, and 15, and seeking to add proposed substitute claim 28. For
the reasons that follow, we deny Patent Owner’s motion to add proposed
substitute claim 28.
A. Applicable Law
A patent owner does not have a right to amend claims in an inter
partes review, but instead must move for amendment. See 35 U.S.C.
§ 316(d). The Board will assess the patentability of proposed substitute
claims “without placing the burden of persuasion on the patent owner.”
Aqua Prods., Inc. v. Matal, 872 F.3d 1290, 1328 (Fed. Cir. 2017) (en banc);
see Bosch Automotive Service Solutions, LLC v. Matal, 878 F.3d 1027 (Fed.
Cir. 2017), as amended and corrected on rehearing; Lectrosonics, Inc. v.
Zaxcom, Inc., IPR2018-01129, Paper 15 at 3‒4 (PTAB Feb. 25, 2019)
(precedential). “[T]he petitioner bears the burden of proving that the
proposed amended claims are unpatentable by a preponderance of the
evidence,” Bosch, 878 F.3d at 1040, and we focus on “arguments and
theories raised by the petitioner in its petition or opposition to the motion to
amend,” Nike, Inc. v. Adidas AG, 955 F.3d 45, 51 (Fed. Cir. 2020).
IPR2020-00219
Patent 8,171,030 B2

24
Proposed substitute claims must meet the requirements of 35 U.S.C.
§ 316(d) and 37 C.F.R. § 42.121. Lectrosonics, Paper 15 at 4–8. The statute
and rules require a patent owner to demonstrate that: (1) the amendment
proposes a reasonable number of substitute claims; (2) the proposed claims
are supported in the original disclosure; (3) the amendment responds to a
ground of unpatentability involved in the trial; and (4) the amendment does
not seek to enlarge the scope of the claims of the patent or introduce new
matter. See 35 U.S.C. § 316(d); 37 C.F.R. § 42.121.
B. Procedural Background
In the Institution Decision, we determined that Petitioner had shown a
reasonable likelihood that it would prevail in proving that claims 1, 8, 11,
13, and 15 were obvious in view of Satoh and DeRoover. In particular, we
concluded that “Petitioner ha[d] shown, sufficiently for institution, that it
would have been obvious in view of Satoh and De Roover to create a
multidimensional video database using a traversal index (as in Satoh) where
that traversal index was a hash (to save space and increase speed, as taught
in Feinleib) and also was robust (to match despite immaterial differences, as
taught in De Roover).” Decision 16.
Patent Owner then filed a Motion to Amend in which it non-
contingently requested to cancel claims 1, 8, 11, 13, and 15, and proposed
substitute claim 28. See MTA 1. Proposed substitute claim 28 was based on
and added a number of limitations to claim 1, including pivots, that the
hashing involved use of a threshold, that the leaf node included the traversal
index and multiple signatures associated with the video sequence, and use of
the traversal index to access the database by comparing the query index to
the pivots and then traversing the hierarchy. See MTA, App. A.
IPR2020-00219
Patent 8,171,030 B2

25
Petitioner opposed the Motion to Amend, and we issued Preliminary
Guidance in which we tentatively concluded that proposed substitute claim
28 added new matter because we agreed with Petitioner that the disclosure
describes “(a) an embodiment in which the hash is used to traverse a
hierarchical tree structure and (b) an embodiment in which the hash is used
as a direct address, but not (c) an embodiment in which the hash is used as
both a means for traversing a tree and as a direct address.” PG 5.
Patent Owner then filed a Revised Motion to Amend, in which it
again requested to cancel claims 1, 8, 11, 13, and 15, and revised proposed
substitute claim 28 by removing from the claim “and wherein the traversal
index is a direct address to the leaf node.” See RMTA, App. A, iii.
C. Proposed Substitute Claim
Proposed substitute claim 28 is reproduced below, with underlining
showing the matter added to original claim 1, straight brackets showing the
matter deleted from original claim 1, and curly brackets showing the matter
deleted in the Revised MTA from the version of proposed substitute claim
28 presented in the Patent Owner’s original MTA:
28. A method of organization and access of a multi-
dimensional video database using a robust hash of a multidimensional
vector signature as a traversal index, the method comprising:
partitioning the multi-dimensional video database by a plurality
of pivots, wherein each pivot of the plurality of pivots is a central
point around which a respective cluster of the multi-dimensional
video database is organized, and wherein a pivot of the plurality of
pivots is coupled to a hierarchical arrangement of traversal nodes and
leaf nodes;
generation of a robust hash value as a traversal index from
multiple parameters extracted from a region of interest in a frame of a
video sequence, wherein generating the robust hash value comprises
IPR2020-00219
Patent 8,171,030 B2

26
comparing at least one feature with a threshold value that is selected
such that the robust hash value has some immunity to noise and
disturbances due to distortion in a query video as compared to an
original video; [and]
storing the traversal index, multiple signatures associated with
the video sequence, and data associated with the video sequence at a
leaf node addressed by the robust hash value, wherein the leaf node is
a member of a plurality of leaf nodes in a multi-dimensional video
database {and wherein the traversal index is a direct address to the
leaf node}; and
using the traversal index to access the multi-dimensional video
database by:
comparing an incoming query index against the plurality of
pivots; and then traversing the hierarchical arrangement via a plurality
of the traversal nodes to reach the leaf node, wherein the plurality of
traversal nodes are constructed from differentiating parts of the
traversal index.
RMTA, App. A.
D. Statutory and Procedural Requirements
1. Number of Substitute Claims
The number of substitute claims must be reasonable. See 35 U.S.C.
§ 316(d)(1)(B); 37 C.F.R. § 42.121(a)(3). Patent Owner presents one
substitute claim corresponding to original claim 1, which is reasonable given
that more than one claim was challenged. See 37 C.F.R. § 42.121(a)(3)
(describing “[t]he presumption . . . that only one substitute claim would be
needed to replace each challenged claim”).
2. Claim Scope
Substitute claims may not enlarge the scope of the issued claims. See
35 U.S.C. § 316(d)(3); 37 C.F.R. § 42.121(a)(2)(ii). Patent Owner contends
that “proposed substitute independent Claim 28 retains all features of issued
IPR2020-00219
Patent 8,171,030 B2

27
Claim 1 and does not enlarge the scope.” RMTA 2. Petitioner does not
argue otherwise, and we agree that proposed substitute claim 28 does not
enlarge the scope of the issued claims, as it includes a number of additional
features making it narrower than original claim 1.
3. Responsiveness to a Ground of Unpatentability
The amendment must “respond to a ground of unpatentability
involved in the trial.” 37 C.F.R. § 42.121(a)(2)(i). Patent Owner contends
that “[p]roposed substitute Claim 28 is responsive to one or more grounds of
alleged unpatentability” because “limitations of the substitute claim are not
obvious from the prior art cited in Petitioner’s Opposition To Patent
Owner’s Motion to Amend.” RMTA 2. Petitioner does not argue that this
requirement is not met, and we conclude that it is, because the added claim
language is intended to distinguish over the prior art.
4. New Matter
A substitute claim may not introduce new subject matter, and the
motion to amend must identify the support in the original disclosure for each
substitute claim. See 35 U.S.C. § 316(d)(3); 37 C.F.R. § 42.121(a)(2)(ii);
37 C.F.R. § 42.121(b).
The written description requirement is met if the application as
originally filed reasonably conveys to a person of ordinary skill in the art
that the inventor had possession of the claimed subject matter at the time of
filing. See Ariad Pharms., Inc. v. Eli Lilly & Co., 598 F.3d 1336, 1351 (Fed.
Cir. 2010) (en banc); Vas-Cath, Inc. v. Mahurkar, 935 F.2d 1555, 1563 (Fed.
Cir. 1991); In re Kaslow, 707 F.2d 1366, 1375 (Fed. Cir. 1983). “[I]t is the
specification itself that must demonstrate possession” and “a description that
IPR2020-00219
Patent 8,171,030 B2

28
merely renders the invention obvious does not satisfy the requirement.”
Ariad, 598 F.3d at 1351–52.
a. 35 U.S.C. § 316(d)(3)
We first address Patent Owner’s argument that the issue of written
description support “could not form a statutory basis for denying the
[RMTA]” because “[t]he relevant statute limits such denials to situations
where the proposed amendment itself introduces new matter.” RMTA Reply
3. Patent Owner points out that “[t]he term ‘addressed by’ is found in
original Claim 1 as well as substitute Claim 28.” Id. at 4.
We do not agree with Patent Owner that we cannot consider whether
there is support for language presented in proposed substitute claim 28
simply because those same words appeared in original claim 1. We do not
read 35 U.S.C. § 316(d)(3) as limiting our inquiry to words that did not
appear in the original claims. Section 316(d)(1) states that the patent owner
may file “a motion to amend the patent” by “[c]ancel[ing] any challenged
patent claim” or “propos[ing] a reasonable number of substitute claims.”
Therefore, an “amendment” in this context is the cancelation or addition of a
claim, not the differences in the words in the original and new claims.
Proposed substitute claim 28––in its entirety––is an “amendment” of the
’030 patent that may not introduce new matter. Cf. Uniloc 2017 LLC v.
Hulu, LLC, 966 F.3d 1295, 1304 (Fed. Cir. 2020) (“The plain language of
the IPR Statutes demonstrates Congress’s intent for the PTAB to review
proposed substitute claims for overall ‘patentability’ . . . of the claims.”); see
id. at 1306 (applying the reasoning from In re NTP, 654 F.3d 1268, 1275
(Fed. Cir. 2011), to inter partes review proceedings and explaining that
“[p]roposed substitute claims in an IPR proceeding have not undergone a
IPR2020-00219
Patent 8,171,030 B2

29
patentability review by the USPTO . . . and so the ‘substantial new questions
of patentability’ that ‘have not previously been considered by the [US]PTO’
include[] all patentability questions”).
We also note that we determine whether there is new matter by
comparing the proposed substitute claim to the original disclosure, not the
issued claims. See 37 C.F.R. § 42.121(b). Patent Owner may not rely on
issued claim 1 because it was not part of the original disclosure––the
applicant added the “addressed by” language to original application claim 1
during prosecution. See Ex. 1002, 27.
b. Simultaneously “Traversing” a Tree
and “Addressing” a Leaf Node
Petitioner argues that “[w]hile Patent Owner changed the words of
claim 28” in the RMTA by “removing the phrase ‘the traversal index is a
direct address to the leaf node,’” Patent Owner’s “interpretation of the claim
remains the same,” as “Patent Owner continues to read claim 28 as requiring
the claimed ‘robust hash value’ ‘traversal index’ to be the ‘address’ of one
particular database leaf node.” RMTA Opp. 1. This is a problem, according
to Petitioner, because “the ’030 patent’s specification provides no written
description support for a claim that simultaneously requires (1) the
‘traversal’ and searching of a multidimensional database to arrive at a leaf
node and (2) an ‘address’ to a leaf node.” Id.; see id. at 3–4.
Petitioner also argues that “[t]he RMTA cites to nothing in the ’030
patent that provides written description support for a ‘robust hash’ that ‘is …
the logical address of the leaf node.’” RMTA Opp. 5. Petitioner observes
that “Patent Owner no longer cites the ‘direct address’ passage as support
IPR2020-00219
Patent 8,171,030 B2

30
[for] this ‘addressed by’ limitation” and that “reference to that passage has
completely disappeared from Patent Owner’s motion.” Id.
Patent Owner responds that proposed substitute “[c]laim 28 recites
that the leaf node is ‘addressed by’ the hash, but uses the traversal index, not
the address, to access the database (‘using the traversal index to access the
multi-dimensional video database.’).” RMTA Reply 1–2. Patent Owner
argues that “‘addressed by’ is a passive property that need not be used in
operation of the claimed method and that this ‘addressed by’ property is
supported.” Id. at 2.
Patent Owner then points to Figure 2A and the accompanying
description in column 8 of the patent (corresponding to page 15 of the non-
provisional application), which states that “the database of signatures of
original videos is also indexed by the traversal indexes computed.” RTMA
Reply 2 (citing Ex. 1001, 8:53–55; Ex. 1002, 15:5–11). Patent Owner
contends that this “clearly provides written description support for the active
traversal combined with a passive organizational feature of a database
indexed by (or addressed by) the computed traversal indexes.” Id. (citing
Ex. 2023 (Moulin) ¶ 53). Patent Owner cites its expert declaration for the
opinion that “a POSITA would have understood the phrase ‘indexed by’ in
this context to yield the same result as the ‘addressed by’ language of Claim
28, where leaf nodes ‘indexed by’ traversal indexes provide the property of
having the hash as an index (or address).” Id. at 2–3.
Patent Owner also points to the Background section of the
’030 patent, which “identifies three problems with conventional large and
complex database systems: (1) update time to update new entries into the
database; (2) query performance; and (3) thoroughness of querying” and
IPR2020-00219
Patent 8,171,030 B2

31
argues that the subject matter of proposed substitute claim 28 “provid[es] a
solution to all three of these problems.” RMTA Reply 3 (citing Ex. 2023
¶¶ 56–62).
Petitioner responds that Patent Owner has “changed course” and
“proposed a third construction” of “addressed by” as “‘simply a passive
property of the leaf node’ that ‘need not be used in operation of the claimed
method.’” RMTA Sur-Reply 2. Petitioner also argues that Patent Owner
“cites an entirely new portion of the specification—four lines from column
8—as providing written description support for its newfound understanding
of ‘addressed by.’” Id. Petitioner contends that “any interpretation of claim
28 that requires . . . the ‘traversal index’ to actually be an address—
regardless of whether that address is used as part of the claimed method or a
‘passive’ feature—causes the claim to lack written description.” Id. at 1–2.
Petitioner further argues that the description in column 8 “does not
describe a database indexed by a ‘robust hash value’ as claim 28 requires”
and “does not mention an ‘address’ to one particular leaf node, or even leaf
node ‘addressing’ in general,” but “instead . . . relates only to the use [of] the
‘traversal index’ to query or search the database to generate a list of ‘likely’
leaf node candidates.” RMTA Sur-Reply 4 (emphasis omitted). According
to Petitioner, “it remains true that the only place the ’030 patent references
the use of an ‘address’ is in column 5.” Id.
Having carefully considered the arguments, we agree with Petitioner
that the RMTA does not identify adequate written description support for a
claim that includes “a robust hash value as a traversal index” and “storing
the traversal index . . . at a leaf node addressed by the robust hash value.”
IPR2020-00219
Patent 8,171,030 B2

32
As a preliminary matter, we find that Patent Owner did not identify in
the RMTA the material at column 8 of the ’030 patent and in the
Background section, as our rules require, and that the RMTA Reply is too
late to raise new evidence. See 37 C.F.R. § 42.121(b) (requiring that a
motion to amend “set forth . . . [t]he support in the original disclosure of the
patent for each claim that is added or amended”); 37 C.F.R. § 42.23(b) (“All
arguments for the relief requested in a motion must be made in the
motion.”); Consolidated Office Patent Trial Practice Guide (Nov. 2019)
(“CTPG”) § II.I (identifying as “[e]xamples of indications that a new issue
has been raised in a reply . . . new evidence necessary to make out a prima
facie case for the patentability or unpatentability of an original or proposed
substitute claim”). Even if we were to consider that material, however, we
would find it insufficient.
The passage from column 8 of the patent and page 15 of the non-
provisional reads as follows: “This traversal index is used to access the
database efficiently. Also, the database of signatures of original videos is
also indexed by the traversal indexes computed.” It does not say that the
leaf nodes are “addressed by” traversal indices or have the traversal indices
as their logical addresses.3 Instead, it says that the database is indexed by
the traversal indexes. We do not agree with Patent Owner that the database
being “indexed by the traversal indexes” provides sufficient support for the
leaf nodes being “addressed by traversal index.”

3 For purposes of considering the proposed substitute claim, we will interpret
“addressed by” as Patent Owner proposes: “whose logical address within
the database is.” PO Resp. 31 (citing Ex. 2012 (Moulin) ¶¶ 89–93).
IPR2020-00219
Patent 8,171,030 B2

33
We have reviewed cited paragraph 53 of the Moulin MTA Declaration
(Ex. 2023) and find it too conclusory to sufficiently correlate these different
concepts. Dr. Moulin states that “a POSITA would understand ‘indexed by’
to . . . yield the same (or similar) functional end result as the ‘addressed by’
language . . . , such that a database indexed by computed traversal indexes
would be understood to include leaf nodes indexed by (or addressed by) the
computed traversal indexes.” Ex. 2023 ¶ 53. We find the testimony that one
of skill in the art would find the “functional end result” to be “the same (or
similar)” insufficient to show support for the specific limitation at issue. See
D Three Enterprises, LLC v. SunModo Corp., 890 F.3d 1042, 1050 (Fed.
Cir. 2018) (“It is not sufficient for purposes of the written description
requirement of § 112 that the disclosure, when combined with the
knowledge in the art, would lead one to speculate as to [the] modifications
that the inventor might have envisioned, but failed to disclose.”) (quoting
Lockwood v. Am. Airlines, Inc., 107 F.3d 1565, 1572 (Fed. Cir. 1997)).
Moreover, the testimony fails to explain how or why “indexing” a database
by traversal indices would require that the leaf nodes be “addressed by” the
traversal indices. We thus cannot conclude that a skilled artisan would have
understood the cited passage to mean that the inventors possessed the
concept of leaf nodes being “addressed by” the traversal index at the time of
filing.
We also find the testimony that “[a] video database that is indexed by
computed traversal index values would be understood to be one in which the
leaf nodes of the database have the property of being indexed by (or
addressed) by the computed traversal indexes” (Ex. 2023 ¶ 55) unpersuasive
IPR2020-00219
Patent 8,171,030 B2

34
because it too is conclusory, and because it also lacks support for the opinion
that “indexing” would require storage of the index in the leaf nodes.
Patent Owner’s argument about the Background section is also
insufficient to show possession of leaf nodes that are “addressed by traversal
index.” The fact that leaf nodes addressed by a traversal index could
provide benefits that can be said to line up with what is described in the
Specification is not sufficient to make up for the lack of a description of that
feature. In order for this argument to carry logical weight, Patent Owner
would need to show that the disclosure ties the benefits to the feature, or that
there is no other way to achieve the stated benefits. Patent Owner has not
done either.
For the foregoing reasons, we find that Patent Owner has not
identified sufficient support for a claim that incudes “a robust hash value as
a traversal index” and “storing the traversal index . . . at a leaf node
addressed by the robust hash value.”
c. “Storing the Traversal Index, Multiple Signatures
Associated with the Video Sequence, and Data
Associated with the Video Sequence at a Leaf Node
Addressed by the Robust Hash Value”
Proposed substitute claim 28 recites “storing the traversal index,
multiple signatures associated with the video sequence, and data associated
with the video sequence at a leaf node addressed by the robust hash value.”
The RMTA cites various portions of the provisional and non-
provisional applications as supporting the “storing” limitation, but does not
specify exactly what in those disclosures might support “storing the traversal
index . . . at a leaf node addressed by the robust hash value.” See RMTA
IPR2020-00219
Patent 8,171,030 B2

35
10–12. We identify in bold text below the specific passages of the non-
provisional application that Patent Owner might be relying on:
Location in
Ex. 1002 Identified Support (emphasis added)
8:12–14 “[A] hierarchical representation 100 of a multi-
dimensional object or video database with traversal
nodes constructed from differentiating parts of
traversal indexes and leaf nodes storing associated
data and associated indexes.”
3:9–10 “In general, the architecture includes associated data
and differentiating indexes at the leaf node.”
3:15 “Multiple associated data or signatures are stored
at a leaf node.”
4:21–23 “Fig. 1A illustrates a hierarchical representation of a
multidimensional object or video database with
traversal nodes constructed from differentiating parts
of traversal indexes and the leaf nodes storing
associated data and associated indexes.”
8:16–17 “The video database is traversed via nodes 108 to
reach the leaf nodes 110 storing associated data and
associated indexes.”
15:18–19 “At step 203, a similarity search computation is
performed, which involves reading the leaf nodes
for associated data.”
16:11–12 “Additional computations can also be performed by
using additional indexes or signatures such as texture,
motion, and associated data such as location and
size.”
20:3–5 “A given set of query indexes 301A derived from a
query video sequence is received in step 301. In step
302, a database index is traversed for this set to
access leaf nodes for associated data and
associated indexes which are stored in step 303.”
Patent Owner also identifies Figures 1A, 2A, and 3A of the non-
provisional application (which have the same numbers in the issued patent),
IPR2020-00219
Patent 8,171,030 B2

36
all of which place the letters “AD,” which stand for “Associated Indexes and
Information” in leaf nodes 110 or storage steps 203 and 303. See RMTA 10.
Our review of the material cited in the provisional application does not
identify any disclosures that are more pertinent.
Petitioner argues that “the ’030 patent does not disclose storing the
specific ‘traversal index’ required by claim 28 in a multi-dimensional
database’s leaf nodes.” RTMA Opp. 11. Petitioner argues that the patent’s
“limited description” of “additional indexes” at the nodes suggests “they are
substantive data used for making computations upon arrival at the leaf
node,” and that nothing “suggests that they are used for finding the node,
much less as a ‘logical address’ of the node.” Id. at 12–13.
Patent Owner argues in the RMTA Reply that “[t]he ’030 Abstract
states ‘[t]raversal indexes are stored in leaf nodes’” and that “[t]he
specification is replete with examples of storing indexes at leaf nodes, which
are described as ‘differentiating indexes’ and ‘associated indexes’ among
other things.” RMTA Reply 6 (citing Ex. 1002, 8:12–24, 20:4–5, 3:9–10).
Citing Dr. Moulin, Patent Owner argues that “[a] POSITA would recognize
the connection of a given traversal index with a corresponding leaf node and
readily understand that ‘associated’ and ‘differentiating’ describe the
traversal index corresponding to a leaf node.” Id. at 7 (citing Ex. 2023
¶¶ 77–79).
We find the material cited in the RMTA insufficient to support
“storing the traversal index . . . at a leaf node addressed by the robust hash
value.” The cited passages refer to storing “associated data and associated
indexes,” “associated data and differentiating indexes,” or “[m]ultiple
associated data or signatures” at the leaf nodes. None of these describe
IPR2020-00219
Patent 8,171,030 B2

37
storing traversal indices in the leaf nodes, and Patent Owner does not
identify support in the disclosure that equates the “associated” or
“differentiating” indices with the traversal index.
Dr. Moulin testifies that a “POSITA would understand that the
traversal index a leaf node is addressed by would be most closely
‘associated’ with that leaf node because the leaf node is addressed by the
traversal index” and that “a POSITA would recognize that a differentiating
index would be one that differentiates a leaf node from other leaf nodes.”
Ex. 2023 ¶ 78. We are not persuaded. The first contention is circular and
conclusory, lacking a reasoned explanation of why the person of skill in the
art would have such an understanding. The second is insufficient to show
that “associated indices” must include the traversal indices because it does
not rule out other the possibility that other types of indices might also
differentiate the leaf nodes.
Finally, though it is true that the Abstract says “[t]raversal indexes are
stored in leaf nodes,” we find this evidence and the related argument also
untimely, coming as they did in the RMTA Reply. See 37 C.F.R. §
42.121(b); 37 C.F.R. § 42.23(b); CTPG § II.I. We also find this sentence
too vague to indicate to a person of ordinary skill in the art that the inventors
were in possession of the concept of storing traversal indices in the leaf
nodes as “addresses” in an embodiment that searches using the traversal
method, as recited in proposed substitute claim 28.
5. Conclusion
Because we find that the proposed substitute claim would add new
matter, we deny Patent Owner’s request in the RMTA to substitute claim 28
for claim 1 for failure to meet the requirements of 35 U.S.C. § 316(d) and
IPR2020-00219
Patent 8,171,030 B2

38
37 C.F.R. § 42.121. We do not reach the issues of “pivots,” indefiniteness,
or whether proposed substitute claim 28 would have been obvious.
VI. CONSTITUTIONALITY
Patent Owner argues that “[t]his IPR should be dismissed as
unconstitutional under Arthrex, Inc. v. Smith and Nephew, Inc., 941 F.3d
1320 (Fed. Cir. 2019) and Tumey v. Ohio, 273 U.S. 510, 523 (1927).”
PO Resp. 75.
We decline to dismiss this case on the grounds that we were
unconstitutionally appointed because our reviewing Court has addressed this
issue, severed the statute, and held that the severance “cures the
constitutional violation.” Arthrex, 941 F.3d at 1337 (Fed. Cir. 2019); see
also Arthrex, Inc. v. Smith & Nephew, Inc., 953 F.3d 760, 764 (Fed. Cir.
2020) (Moore, J., concurring in denial of rehearing) (“Because the APJs
were constitutionally appointed as of the implementation of the severance,
inter partes review decisions going forward were no longer rendered by
unconstitutional panels.”).
We also decline to dismiss this case under Tumey, at least because
Patent Owner has presented no evidence or argument that any member of
this panel “has a direct, personal, substantial, pecuniary interest in reaching a
conclusion against [Patent Owner] in his case,” nor are we aware of any
such evidence. Tumey, 273 U.S. at 523.
IPR2020-00219
Patent 8,171,030 B2

39
VII. CONCLUSION
Claims 6 and 10 of the ’030 patent have not been shown to be
unpatentable. The RMTA is granted as to Patent Owner’s request to cancel
claims 1, 8, 11, 13, and 15, and denied as to proposed substitute claim 28.
The results are summarized below.
Claims 35 U.S.C. § Reference(s)/Basis
Claims
Shown
Unpatentable
Claims Not
Shown
Unpatentable
6, 10 103 Deng, Brunk 6, 10
Overall Outcome 6, 10

Motion to Amend Outcome Claim(s)
Original Claims Cancelled by Amendment 1, 8, 11, 13, 15
Substitute Claims Proposed in the Amendment 28
Substitute Claims: Motion to Amend Granted
Substitute Claims: Motion to Amend Denied 28
Substitute Claims: Not Reached

IPR2020-00219
Patent 8,171,030 B2

40
VIII. ORDER
For the reasons given, it is:
ORDERED that claims 6 and 10 of U.S. Patent 8,171,030 B2 have not
been shown to be unpatentable;
FURTHER ORDERED that Patent Owner’s Revised Motion to
Amend is granted as to Patent Owner’s request to cancel claims 1, 8, 11, 13,
and 15 of U.S. Patent 8,171,030 B2 and denied as to proposed substitute
claim 28; and
FURTHER ORDERED that, because this is a Final Written Decision,
the parties to the proceeding seeking judicial review of the decision must
comply with the notice and service requirements of 37 C.F.R. § 90.2.4

4 Should Patent Owner wish to pursue amendment of the challenged claims
in a reissue or reexamination proceeding subsequent to the issuance of this
decision, we draw Patent Owner’s attention to the April 2019 Notice
Regarding Options for Amendments by Patent Owner Through Reissue or
Reexamination During a Pending AIA Trial Proceeding. See 84 Fed. Reg.
16654 (Apr. 22, 2019). If Patent Owner chooses to file a reissue application
or a request for reexamination of the challenged patent, we remind Patent
Owner of its continuing obligation to notify the Board of any such related
matters in updated mandatory notices. See 37 C.F.R. § 42.8(a)(3), (b)(2).
IPR2020-00219
Patent 8,171,030 B2

41
FOR PETITIONER:

T. Vann Pearce, Jr.
K. Patrick Herman
ORRICK, HERRINGTON & SUTCLIFFE LLP
TVPPTABDocket@orrick.com
P52PTABDocket@orrick.com

FOR PATENT OWNER:

James T. Carmichael
Stephen W. Aycock II
James L. Lovsin
CARMICHAEL IP, PLLC
jim@carmichaelip.com
steve@carmichael.com
lovsin@mbhb.com