Parus Holdings, Inc.

Patent Trials and Appeals Board

Sep 9, 2021

IPR2020-00686 (P.T.A.B. Sep. 9, 2021)

Trials@uspto.gov Paper 37
571-272-7822 Date: September 9, 2021

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

APPLE INC.,
Petitioner,

v.

PARUS HOLDINGS, INC.,
Patent Owner.
____________

IPR2020-00686
Patent 7,076,431 B2
____________

Before DAVID C. MCKONE, STACEY G. WHITE, and
SHELDON M. MCGEE, Administrative Patent Judges.

MCKONE, Administrative Patent Judge.

JUDGMENT
Final Written Decision
Determining No Challenged Claims Unpatentable
35 U.S.C. § 318(a)
Denying Patent Owner’s Motion to Exclude
37 C.F.R. § 42.64

IPR2020-00686
Patent 7,076,431 B2
2
I. INTRODUCTION
A. Background and Summary
Apple Inc. (“Petitioner”) filed a Petition requesting an inter partes
review of claims 1–7, 9, 10, 13, 14, 18–21, and 25–30 of U.S. Patent
No. 7,076,431 B2 (Ex. 1001, “the ’431 patent”). Paper 1 (“Pet.”). Parus
Holdings, Inc. (“Patent Owner”) filed a Preliminary Response to the
Petition. Paper 6 (“Prelim. Resp.”). Pursuant to 35 U.S.C. § 314, we
instituted this proceeding. Paper 9 (“Dec.”).
Patent Owner filed a Patent Owner’s Response (Paper 15, “PO
Resp.”), Petitioner filed a Reply to the Patent Owner’s Response (Paper 19,
“Reply”), and Patent Owner filed a Sur-reply to the Reply (Paper 21, “Sur-
reply”). Patent Owner filed a Motion to Exclude certain evidence submitted
by Petitioner (Paper 29, “Mot. Excl.”), to which Petitioner filed an
Opposition (Paper 30, “Opp. Mot. Excl.”). Patent Owner filed a Reply to
Petitioner’s Opposition to its Motion to Exclude (styled a “Sur-reply”).
Paper 32 (“Reply Mot. Excl.”). An oral argument was held in this
proceeding and IPR2020-00687 on June 22, 2021. Paper 36 (“Tr.”).
We have jurisdiction under 35 U.S.C. § 6. This Decision is a final
written decision under 35 U.S.C. § 318(a) as to the patentability of claims 1–
7, 9, 10, 13, 14, 18–21, and 25–30. Based on the record before us, Petitioner
has not proved, by a preponderance of the evidence, that claims 1–7, 9, 10,
13, 14, 18–21, and 25–30 are unpatentable.
We also deny Patent Owner’s Motion to Exclude.

B. Related Matters
The parties identify the following district court proceedings as related
to the ’431 patent: Parus Holdings Inc. v. Apple, Inc., No. 6:19-cv-00432
IPR2020-00686
Patent 7,076,431 B2
3
(W.D. Tex.) (“the Texas case”); Parus Holdings Inc. v. Amazon.com, Inc.,
No. 6:19-cv-00454 (W.D. Tex.); Parus Holdings Inc. v. Samsung
Electronics Co., Ltd., No. 6:19-cv-00438 (W.D. Tex.); Parus Holdings Inc.
v. Google LLC, No. 6:19-cv-00433 (W.D. Tex.); and Parus Holdings Inc. v.
LG Electronics, Inc., No. 6:19-cv-00437 (W.D. Tex.). Pet. 72; Paper 5, 1.
The parties also identify U.S. Patent No. 6,721,705 and U.S. Patent
No. 9,451,084 as related to the ’431 patent, and further identify that U.S.
Patent No. 9,451,084 has been asserted in the district court proceedings
listed above, and is the subject of IPR2020-00687. Pet. 72; Paper 5, 1.

C. The ’431 Patent
The ’431 patent describes a system that allows users to browse web
sites and retrieve information using conversational voice commands.
Ex. 1001, 1:20–23. Figure 1, reproduced below, illustrates an example:

IPR2020-00686
Patent 7,076,431 B2
4
Figure 1 is a block diagram of a voice browsing system. Id. at 4:16–17.
Figure 3, reproduced below, shows additional details of media server 106, a
component shown in Figure 1:

Figure 3 is a block diagram of Figure 1’s media server 106. Id. at 4:20–21.
IPR2020-00686
Patent 7,076,431 B2
5
Media server 106 includes speech recognition engine 300, speech
synthesis engine 302, Interactive Voice Response (IVR) application 304, call
processing system 306, and telephony and voice hardware 308 to
communicate with Public Switched Telephone Network (PTSN) 116. Id. at
5:62–6:1. When a user speaks into voice enable device 112 (e.g., a wireline
or wireless telephone), speech recognition engine 300 converts voice
commands into data messages. Id. at 6:4–8. Media server 106 uses results
(e.g., keywords) generated by speech recognition engine 300 to retrieve web
site record 200 stored in database 100 that can provide the information
requested by the user. Id. at 6:44–50. Media server 106 selects the web site
record of highest rank and transmits it to web browsing server 102 along
with an identifier indicating what information is being requested. Id. at
6:52–56. Speech synthesis engine converts the data retrieved by web
browsing server 102 into audio messages that are transmitted to voice enable
device 112. Id. at 6:57–60.
According to the ’431 patent, with its system,
[u]sers are not required to learn a special language or command
set in order to communicate with the voice browsing system of
the present invention. Common and ordinary commands and
phrases are all that is required for a user to operate the voice
browsing system. The voice browsing system recognizes
naturally spoken voice commands and is speaker-independent;
it does not have to be trained to recognize the voice patterns of
each individual user. Such speech recognition systems use
phonemes to recognize spoken words and not predefined voice
patterns.
Id. at 4:34–43.
IPR2020-00686
Patent 7,076,431 B2
6
Claim 1, reproduced below, is illustrative of the invention:1
1. A system for retrieving information from pre-selected
web sites by uttering speech commands into a voice enabled
device and for providing to users retrieved information in an
audio form via said voice enabled device, said system
comprising:
[a] a computer, said computer operatively connected to
the internet;
[b] a voice enabled device operatively connected to said
computer, said voice enabled device configured to
receive speech commands from users;
[c] at least one speaker-independent speech recognition
device, said speaker-independent speech
recognition device operatively connected to said
computer and to said voice enabled device;
[d] at least one speech synthesis device, said speech
synthesis device operatively connected to said
computer and to said voice enabled device;
[e] at least one instruction set for identifying said
information to be retrieved, said instruction set
being associated with said computer, said
instruction set comprising:
a plurality of pre-selected web site addresses, each
said web site address identifying a web site
containing said information to be retrieved;
[f] at least one recognition grammar associated with said
computer, each said recognition grammar
corresponding to each said instruction set and
corresponding to a speech command;
[g] said speech command comprising an information
request selectable by the user;

1 For consistency with the parties’ arguments, we add bracketed lettering to
track the lettering supplied by Petitioner. See Pet. 74–79 (Claims Listing
Appendix).
IPR2020-00686
Patent 7,076,431 B2
7
[h] said speaker-independent speech recognition device
configured to receive from users via said voice
enabled device said speech command and to select
the corresponding recognition grammar upon
receiving said speech command;
[i] said computer configured to retrieve said instruction
set corresponding to said recognition grammar
selected by said speaker-independent speech
recognition device;
[j] said computer further configured to access at least one
of said plurality of web sites identified by said
instruction set to obtain said information to be
retrieved,
[k] said computer configured to first access said first web
site of said plurality of web sites and, if said
information to be retrieved is not found at said first
web site, said computer configured to sequentially
access said plurality of web sites until said
information to be retrieved is found or until said
plurality of web sites has been accessed;
[l] said speech synthesis device configured to produce an
audio message containing any retrieved
information from said pre-selected web sites, and
said speech synthesis device further configured to
transmit said audio message to said users via said
voice enabled device.

D. Evidence
Petitioner relies on the references listed below.
Reference Date Exhibit
No.
Ladd US 6,269,336 B1 July 31, 2001 1004
IPR2020-00686
Patent 7,076,431 B2
8
Reference Date Exhibit
No.
Kurosawa2 JP H9-311869 A Dec. 2, 1997 1005
Goedken US 6,393,423 B1 May 21, 2002 1006
Madnick US 5,913,214 June 15, 1999 1007
Houser US 5,774,859 June 30, 1998 1008
Rutledge US 6,650,998 B1 Nov. 18, 2003 1010

Petitioner also relies on the Declaration of Loren Terveen, Ph.D.
(Ex. 1003) and the Supplemental Declaration of Dr. Terveen (Ex. 1040).
Patent Owner relies on the Declaration of Benedict Occhiogrosso
(Ex. 2025).

E. The Instituted Grounds of Unpatentability
Claims Challenged 35 U.S.C. § References
1–6, 9, 10, 13, 14, 18,
20, 21, 25 103(a)
3 Ladd, Kurosawa, Goedken
7, 19, 26–30 103(a) Ladd, Kurosawa, Goedken, Madnick
5, 6 103(a) Ladd, Kurosawa, Goedken, Houser
9, 25 103(a) Ladd, Kurosawa, Goedken, Rutledge

2 We rely on the certified translation of JP H09-311869 (Ex. 1005).
3 The Leahy-Smith America Invents Act (“AIA”), Pub. L. No. 112-29, 125
Stat. 284, 287–88 (2011), amended 35 U.S.C. § 103. Because the ’431
patent was filed before March 16, 2013, the effective date of the relevant
amendment, the pre-AIA version of § 103 applies.
IPR2020-00686
Patent 7,076,431 B2
9
II. ANALYSIS
A. Claim Construction
For petitions filed after November 13, 2018, we construe claims
“using the same claim construction standard that would be used to construe
the claim in a civil action under 35 U.S.C. 282(b), including construing the
claim in accordance with the ordinary and customary meaning of such claim
as understood by one of ordinary skill in the art and the prosecution history
pertaining to the patent.” 37 C.F.R. § 42.100(b) (2019); see also Phillips v.
AWH Corp., 415 F.3d 1303 (Fed. Cir. 2005) (en banc).
In the Petition, Petitioner contended that we should give the claim
terms their plain and ordinary meaning, and did not identify any claim term
for construction. Pet. 11.
In the Institution Decision, we made clear that the plain and ordinary
meaning of “instruction set,” as recited in each of the independent claims,
does not require “[a] set of machine language instructions that a processor
executes,” rejecting Patent Owner’s arguments to the contrary. Dec. 22–23;
Prelim. Resp. 48.
After the pre-institution briefing was completed, but before we issued
the Institution Decision, the court in the Texas case issued a claim
construction ruling, construing “speaker-independent speech recognition
device” to mean “speech recognition device that recognizes spoken words
without adapting to individual speakers or using predefined voice patterns.”
Ex. 1041, 2.4 The parties agree that this term at least requires a “speech

4 The court in the Texas case issued other constructions pertaining to the
challenged claims, but the parties do not advance them in this proceeding
and we do not find it necessary to adopt them in order to resolve the parties’
dispute.
IPR2020-00686
Patent 7,076,431 B2
10
recognition device that recognizes spoken words without . . . using
predefined voice patterns,” but disagree as to whether it should require a
device that recognizes spoken words “without adapting to individual
speakers.” PO Resp. 21 (“The proper construction of ‘speaker-independent
speech recognition device’ is consistent with the construction issued by the
Western District of Texas, though it does not include all of that court’s
construction, and requires at least ‘speech recognition device that recognizes
spoken words without using predefined voice patterns.’”); Reply 2 (“For
purposes of this IPR, Apple submits the Court’s construction should be
applied.”).
The dispute as to whether the term should preclude adapting to
individual speakers does not impact any issue in this proceeding, and
Petitioner has agreed to Patent Owner’s construction in this proceeding, so
long as we do not resolve the dispute over adapting to individual speakers.
Tr. 12:24–13:4 (“JUDGE McKONE: So you’d be happy if we essentially
adopted Parus’s construction with a footnote or some kind of note that we’re
not resolving the issue of adapting to individual speakers? MS. BAILEY:
That would be fine for purposes of this IPR, Your Honor.”). We adopt the
parties’ agreed approach. For purposes of this proceeding, “speaker-
independent speech recognition device” means “speech recognition device
that recognizes spoken words without using predefined voice patterns.” This
is consistent with the ’431 patent’s statement (relied on by both parties) that
“[t]he voice browsing system recognizes naturally spoken voice commands
and is speaker-independent; it does not have to be trained to recognize the
voice patterns of each individual user. Such speech recognition systems use
phonemes to recognize spoken words and not predefined voice patterns.”
Ex. 1001, 4:38–43; see also PO Resp. 21–22 (citing Ex. 1001, 4:34–43);
IPR2020-00686
Patent 7,076,431 B2
11
Reply 2–3 (citing Ex. 1001, 4:38–43). We take no position on whether the
construction also should include “without adapting to individual speakers.”
Based on the record before us, we do not find it necessary to provide
express claim constructions for any other terms. See Nidec Motor Corp. v.
Zhongshan Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed. Cir. 2017)
(noting that “we need only construe terms ‘that are in controversy, and only
to the extent necessary to resolve the controversy’”) (quoting Vivid Techs.,
Inc. v. Am. Sci. & Eng’g, Inc., 200 F.3d 795, 803 (Fed. Cir. 1999)).

B. Obviousness of Claims 1–6, 9, 10, 13, 14, 18, 20, 21, and 25
over Ladd, Kurosawa, and Goedken
Petitioner contends that claims 1–6, 9, 10, 13, 14, 18, 20, 21, and 25
would have been obvious over Ladd, Kurosawa, and Goedken. Pet. 17–61.
For the reasons given below, Petitioner has not shown obviousness by a
preponderance of the evidence.
A claim is unpatentable under 35 U.S.C. § 103 (pre-AIA) if the
differences between the claimed subject matter and the prior art are “such
that the subject matter as a whole would have been obvious at the time the
invention was made to a person having ordinary skill in the art to which said
subject matter pertains.” We resolve the question of obviousness on the
basis of underlying factual determinations, including (1) the scope and
content of the prior art; (2) any differences between the claimed subject
matter and the prior art; (3) the level of skill in the art; and (4) objective
evidence of nonobviousness, i.e., secondary considerations.5 See Graham v.
John Deere Co., 383 U.S. 1, 17–18 (1966).

5 The record does not include allegations or evidence of objective indicia of
nonobviousness or obviousness.
IPR2020-00686
Patent 7,076,431 B2
12
1. Level of Skill in the Art
Petitioner, relying on the testimony of Dr. Terveen, contends that a
person of ordinary skill in the art “would have had a Bachelor’s degree in
Electrical Engineering, Computer Engineering, Computer Science, or
equivalent degree, with at least two years of experience in interactive voice
response systems, automated information retrieval systems, or related
technologies, such as web-based information retrieval systems.” Pet. 6
(citing Ex. 1003 ¶ 28). Patent Owner does not contest Petitioner’s proposal
or offer an alternative. Also, neither party argues that the outcome of this
case would differ based on our adoption of any particular definition of one
of ordinary skill in the art. Petitioner’s proposal is consistent with the
technology described in the Specification and the cited prior art. On the
complete record, we adopt Petitioner’s proposed level of skill.

2. Scope and Content of the Prior Art
a) Overview of Ladd
Ladd describes a voice browser for allowing a user to access
information from an information source. Ex. 1004, 1:20–25. Figure 3,
reproduced below, illustrates an example:
IPR2020-00686
Patent 7,076,431 B2
13

Figure 3 is a block diagram of a system that enables a user to access
information. Id. at 4:62–64.
A user accesses electronic network 206 by dialing a telephone number
from communication device 202 (e.g., a landline or wireline device, or a
wireless device). Id. at 5:20–23, 5:29–35. Communication node 212
answers the incoming call from carrier network 216 and plays an
announcement to the user. Id. at 6:13–17. In response to audio inputs from
the user, communication node 212 retrieves information from content
providers 208 and 209. Id. at 6:17–21. For example, voice recognition
(VRU) client 232 generates pre-recorded voice announcements and
messages to prompt the user to provide inputs using speech commands.
Id. at 7:48–51. VRU client 232 receives and processes speech
communications and routes them to VRU server 234, which processes the
communications and compares them to a vocabulary or grammar stored in
database server unit 244. Id. at 8:3–9, 8:55–61.
IPR2020-00686
Patent 7,076,431 B2
14
According to Ladd,
The ASR [automatic speech recognition] unit 254 of the VRU
server 234 provides speaker independent automatic speech
recognition of speech inputs or communications from the
user. . . . The ASR unit 254 processes the speech inputs from
the user to determine whether a word or a speech pattern
matches any of the grammars or vocabulary stored in the
database server unit 244 or downloaded from the voice browser.
When the ASR unit 254 identifies a selected speech pattern of
the speech inputs, the ASR unit 254 sends an output signal to
implement the specific function associated with the recognized
voice pattern. The ASR unit 254 is preferably a speaker
independent speech recognition software package, Model No.
RecServer, available from Nuance Communications. It is
contemplated that the ASR unit 254 can be any suitable speech
recognition unit to detect voice communications from a user.
Id. at 9:27–44.
After receiving information from content providers 208, 209,
communication node 212 provides a response to the user based on the
retrieved information. Id. at 6:21–24. Specifically, text-to-speech (TTS)
unit 252 of VRU server 234 receives textual data (e.g., web pages) from
application server unit 242, processes the textual data to voice data, and
provides the voice data to VRU client 232, which reads or plays the voice
data to the user. Id. at 9:1–23.

b) Overview of Kurosawa
Kurosawa describes an Internet search server that obtains requested
information from a plurality of URLs, and delivers a search report to a
client. Ex. 1005, Abst. Figure 2 of Kurosawa, reproduced below, illustrates
an example:
IPR2020-00686
Patent 7,076,431 B2
15

Figure 2 is a functional block diagram of an Internet search server. Id. ¶ 20.
Internet search server 10 includes URL database 11, which has a
comparison table (URL table 22, shown in Figure 6) that compares a
plurality of keywords representing search condition elements to URLs that
relate to the keywords. Id. ¶¶ 20–21. Figure 5 is reproduced below:
IPR2020-00686
Patent 7,076,431 B2
16

Figure 5 is a picture of keyword table 21 in URL database 11. Id. ¶ 21.
According to Kurosawa, “anything that is not listed in the keyword table 21
cannot be searched for.” Id. Figure 6 is reproduced below:
IPR2020-00686
Patent 7,076,431 B2
17

Figure 6 is a picture of URL table 22 in URL database 11. Id. ¶ 21. Search
server 10 regularly updates URL table 22 in URL database 11 using
automatic search tools, such as Internet web crawlers. Id. ¶ 23.
When a client sends a search request to Internet search server 10,
search condition element extraction unit 13 extracts search condition
elements from the client’s search request, and URL search unit 14 extracts
keywords (included in the search condition elements) that match those of
keyword table 21 and selects URLs (from URL table 22) having the
extracted keywords listed therein. Id. ¶¶ 26–28. URL listing order
arranging unit 15 determines a listing order for the selected URLs based on
priority conditions for efficient searching. Id. ¶ 29. Thereafter, URL listing
unit 16 sequentially lists the addresses of the respective URLs in the
IPR2020-00686
Patent 7,076,431 B2
18
determined order, and accesses the respective webpages of the URLs.
Id. ¶ 30. URL information gathering unit 17 sequentially accumulates
information from the URL pages for presentation to the client. Id. ¶¶ 30–31.

c) Overview of Goedken
Goedken describes a method and apparatus for facilitating information
exchange, via a network, between an information requestor/searcher and one
or more information custodians, who are persons that “know[] where to
locate and/or ha[ve] custody of the information that interests the searcher.”
Ex. 1006, Abst, 1:42–44. The searcher creates an information request
message and sends it to the apparatus, and the apparatus determines an
appropriate custodian and sends a request message to that custodian. Id.
The identified custodian replies to the request message with an intermediate
answer message or with a reroute message. Id. Based on the messages, the
apparatus provides a final answer message to the searcher, and may also
record the answer message for subsequent retrieval. Id. For example, the
apparatus may record portions of final answer messages developed by
information custodians and store those records in a knowledge database.
Id. at 19:43–48. “Preferably, the knowledge database 136 is populated by
earlier questions and answers routed through the apparatus 10, as well as any
number of preprogrammed questions and answers (e.g., an existing help line
database).” Id. at 25:15–19.
Petitioner relies on the embodiment of Goedken relating to searching
the knowledge database for previously stored answers. Pet. 41–44 (citing
Ex. 1006, 25:9–26:23, Fig. 18). Figure 18 of Goedken, reproduced below,
illustrates this embodiment:
IPR2020-00686
Patent 7,076,431 B2
19

Figure 18 is a flowchart of a program implementing the apparatus of
Goedken’s Figure 1 (an apparatus for facilitating information exchange
IPR2020-00686
Patent 7,076,431 B2
20
between an information requester and an information custodian via a
network). Ex. 1006, 7:39–42, 8:13–15.
“Once the category of a received information request message 18 has
been determined, the database manager 140 is activated to search the
knowledge database 136 for a responsive answer.” Id. at 25:19–22. The
database manager retrieves the category associated with a first file from the
knowledge database (block 332). Id. at 15:24–26. The database manager
compares the retrieved category to the requested category (block 334) and, if
there is no match, the database manager determines whether there are more
files to consider (block 336). Id. at 25:26–32. If there are more files to
consider (block 338), the category of the next file is retrieved and compared
to the category of the file (blocks 334, 336). Id. at 25:32–35. “The database
manager 140 continues to loop through blocks 332–338 until all of the
categories of all of the files in the knowledge database 136 are compared to
the category associated with the information request message 18 or until a
match is found at block 334.” Id. at 25:35–40.
After a file corresponding to the category has been found, Goedken’s
algorithm similarly loops through a set of “synonyms” for the user’s
question to identify whether there is a match for those synonyms in the
identified file (blocks 340, 342, 344, 348, 348). Id. at 25:4–26:7. “If a
question synonym is found at block 344, the database manager 140 passes
the answer associated with that file to the message composer 122, and the
message composer 122 preferably attaches the ‘canned’ answer from the
knowledge database 136 to the information request message 18 (block
350).” Id. at 26:8–13.

IPR2020-00686
Patent 7,076,431 B2
21
3. Claims 1–6, 9, 10, 13, 14, 18, 20, 21, and 25, Differences
Between the Claimed Subject Matter and Ladd, Kurosawa, and
Goedken
a) The Parties’ Contentions for Claim 1
Petitioner contends that Ladd teaches claim limitations 1[a]–1[d],
1[f]–1[j], and 1[l]; that Kurosawa teaches limitation 1[e] and aspects of
limitation 1[i]; and that Goedken teaches limitation 1[k].
Regarding the preamble of claim 1, Petitioner contends that Ladd
describes a system for retrieving information by uttering speech commands
into a voice-enabled device and for providing retrieved information in audio
form to users via the voice-enabled device. Pet. 17–18 (citing Ex. 1004,
1:22–25, 2:19–64, 3:8–23, 3:40–53, 3:58–4:3, 4:62–64, 5:30–36, 9:1–10,
9:19–21, 11:50–63, Figs. 1, 3). As to claim elements 1[a] and 1[b],
Petitioner maps Ladd’s communication node 212 to the “computer
operatively connected to the internet” and Ladd’s communication devices
201, 202, 203, 204 to “a voice enabled device operatively connected to said
computer.” Id. at 18–22 (citing Ex. 1004, 1:48–54, 1:61–64, 2:59–64, 4:62–
5:11, 5:12–39, 6:50–55, 7:7–17, 7:24–32, 7:52–56, 10:34–36).
Petitioner contends that Ladd’s ASR 254 within VRU server 234 is
“at least one speaker-independent speech recognition device,” as recited in
claim element 1[c]. Id. at 22–23 (citing Ex. 1004, 6:65–7:7, 7:28–33, 8:19–
28, 8:55–67, 9:1–3, 9:28–44). As to claim element 1[d], Petitioner contends
that Ladd’s TTS unit 252 within VRU server 234 is “at least one speech
synthesis device.” Id. at 23–25 (citing Ex. 1004, 3:40–57, 4:51–5:20, 5:24–
29, 5:34–35, 7:28–33, 8:55–56, 9:1–23).
Regarding claim element 1[f], Petitioner, inter alia, points to Ladd’s
description of VRU server 234 “process[ing] the speech communications
IPR2020-00686
Patent 7,076,431 B2
22
and compar[ing] the speech communications against a vocabulary or
grammar.” Id. at 32–34 (quoting Ex. 1004, 8:55–61; citing id. at 4:36–49,
9:28–44, 10:12–17, 14:13–28, 19:12–36). As to claim element 1[g],
Petitioner argues that Ladd describes a user speaking a request to access
information such as news, weather, and traffic. Id. at 34–35 (citing
Ex. 1004, 2:48–58, 4:62–5:11, 7:49–56, 10:58–66). As to claim element
1[h], Petitioner argues that Ladd describes VRU server 234 (including ASR
unit 254) receiving speech commands from a communication device and
determining whether a word or speech pattern matches any of the grammars
or vocabulary stored in database server unit 244. Id. at 35–36 (citing
Ex. 1004, 4:62–5:35, 6:65–7:7, 7:27-32, 8:3–28, 8:55–58, 9:1–3, 9:28–39).
As to claim element 1[l], Petitioner argues that the text TTS unit 252
converts to speech can be information retrieved from web sites. Id. at 47–48
(citing Ex. 1004, 4:51–5:36, 6:13–25, 7:27–33, 8:55–56, 9:1–26).
As to claim element 1[e], Petitioner contends that Kurosawa teaches
this limitation. Pet. 25–32. In particular, Petitioner contends that URL table
22, shown in Kurosawa’s Figure 6 (reproduced above), illustrates a plurality
of web site addresses, each matching keywords in a user’s search condition
and identifying a web site containing information to be retrieved related to
the keywords. Id. at 25–29 (citing Ex. 1005 ¶¶ 9, 11, 12, 21, 24, 27, 28, 37).
Petitioner contends that the URLs are “pre-selected” because they are
known, cross-referenced to keywords, and stored in URL database 11 before
the search. Id. at 28 (citing Ex. 1005 ¶¶ 20–21). Petitioner contends that the
“instruction set” is Kurosawa’s plurality of URLs picked out based on
keyword matching, and argues that this instruction set is associated with
search server 10 shown in Kurosawa’s Figure 1. Id. at 29 (citing Ex. 1005
IPR2020-00686
Patent 7,076,431 B2
23
¶ 21). Petitioner contends that Ladd’s system would have been modified to
include the plurality of URLs in Kurosawa’s database. Id. at 29–30.
Regarding claim element 1[i], Petitioner contends that Ladd describes
communication node 212 (including ASR unit 254) as monitoring speech
commands to detect keywords (such as “weather”) corresponding to
information the user desires. Pet. 36–37 (citing Ex. 1004, 4:36–49, 5:37–39,
6:14–29, 7:52–56, 8:55–67, 9:35–39, 11:50–63). Petitioner pairs this
teaching with Kurosawa, which Petitioner contends teaches accessing a
plurality of pre-selected URLs from a database table to sequentially access
websites to retrieve information desired by users. Id. (citing Ex. 1005 ¶¶ 9,
10, 15, 20, 21). As to claim element 1[j], Petitioner argues that Ladd teaches
accessing websites based on speech commands and that Kurosawa teaches
sequentially accessing URLs to gather information. Id. at 37–39 (citing
Ex. 1004, 3:7–39, 4:37–49, 6:18–25, 6:65–7:7, 7:44–56, 11:31–36, 14:1–9;
Ex. 1005 ¶¶ 9, 15, 40).
As to claim element 1[k], Petitioner contends that Kurosawa teaches
sequentially accessing the URL addresses listed in URL table 22 in a priority
order determined by URL listing order arranging unit 15. Pet. 39–40 (citing
Ex. 1005 ¶¶ 15, 29, 35, 40). According to Petitioner,
Goedken discloses a procedure that accesses a first file of a
plurality of files for an answer to a question. If the information
to be retrieved is not found at the first file (it fails to match the
category or synonym), the procedure sequentially accesses the
next file of the plurality of files until the information to be
retrieved is found (matching both the category and synonym) or
until all files have been accessed via repeated application.
Id. at 43–44 (citing Ex. 1006, 25:59–26:7; Ex. 1003 ¶¶ 119–120). Petitioner
contends that, in combination, “[t]he Ladd system as further modified by
Goedken sequentially accesses the plurality of preselected websites
IPR2020-00686
Patent 7,076,431 B2
24
efficiently and quickly via the Goedken procedure, which returns an answer
once found or continues accessing information sources (websites, when
applied to the system of Ladd modified by Kurosawa), until all websites are
accessed.” Id. at 46–47.
Patent Owner argues that Goedken does not teach sequentially
accessing pre-selected web sites (PO Resp. 38–39); that Kurosawa does not
teach sequentially accessing pre-selected web sites until requested
information is found or all pre-selected web sites have been accessed (id. at
40–41); that Petitioner’s obviousness combinations are based on
impermissible hindsight (id. at 41–43); that Petitioner has not shown a
motivation to combine Ladd and Kurosawa (id. at 43–45); that Petitioner has
not shown a motivation to combine Goedken with Ladd and Kurosawa
(id. at 46–48); and that the prior art teaches away from the proposed
combination (id. at 48–56). Patent Owner contends that Ladd does not teach
the “speaker-independent speech recognition device” of claim limitation
1[c]. Id. at 34–38; Sur-reply 2–18. For the reasons given below, Petitioner
has not made the requisite showing as to claim limitation 1[c]; thus, it is
unnecessary to resolve the remaining disputes raised by Patent Owner.

b) Petitioner has not shown that Ladd teaches the
“speaker-independent speech recognition device” of
claim limitation 1[c]
Claim limitation 1[c] recites “at least one speaker-independent speech
recognition device, said speaker-independent speech recognition device
operatively connected to said computer and to said voice enabled device.”
(emphasis added). The parties dispute whether Ladd teaches this limitation.
IPR2020-00686
Patent 7,076,431 B2
25
In the Petition, Petitioner contended that Ladd’s ASR 254 is a
“speaker-independent speech recognition device.” Pet. 22. Petitioner (id. at
22–23) referred to Ladd’s statement that “[t]he ASR unit 254 of the VRU
server 234 provides speaker independent automatic speech recognition of
speech inputs or communications from the user.” Ex. 1004, 9:27–29.
Petitioner also expressly quoted Ladd’s description that “[t]he ASR unit
‘processes the speech inputs from the user to determine whether a word or a
speech matter matches any of the [stored] grammars.’” Pet. 23 (quoting
Ex. 1004, 9:31–33 (addition of [stored] by Petitioner)). Petitioner also cited
to disclosure in Ladd that “[t]he voice communication boards may include a
voice recognition engine having a vocabulary for detecting a speech pattern
(i.e., a key word or phrase),” and that “[w]hen the ASR unit 254 identifies a
selected speech pattern of the speech inputs, the ASR unit 254 sends an
output signal to implement the specific function associated with the
recognized voice pattern.” Id. (citing Ex. 1004, 8:19–28, 9:28–44). Thus, in
the Petition, Petitioner expressly relied on Ladd’s descriptions of detecting
or identifying speech patterns or voice patterns when Petitioner identified
the disclosure in Ladd that teaches a “speaker-independent speech
recognition device.” Petitioner confirmed this reliance at the oral argument.
Tr. 23:21–23 (“This paragraph at column 9, beginning at line 28, was relied
on in the petition for supporting the speaker independent speech recognition
device.”). Dr. Terveen provided expert testimony supporting the arguments
in the Petition, characterizing Ladd in the same manner, and citing to
essentially the same disclosure in Ladd. Ex. 1003 ¶¶ 90–91 (citing
Ex. 1004, 8:19–28, 9:28–44). Petitioner appears to still rely on Ladd’s
statements of detecting or identifying speech or voice patterns. Tr. 22:4
(“We’re still relying on the same disclosure that is in Ladd.”), 23:24–24:3
IPR2020-00686
Patent 7,076,431 B2
26
(“JUDGE McKONE: Are you withdrawing your reliance on this paragraph
or portions of this paragraph? MS. BAILEY: No, Your Honor. I’m not
withdrawing it at all. In fact, I think it does still continue to support our
position.”).
As noted above, after the Petition was filed and the rest of the pre-
institution briefing was completed, the court in the Texas case issued a claim
construction order construing “speaker-independent speech recognition
device” to require, inter alia, a “speech recognition device that recognizes
spoken words without . . . using predefined voice patterns.” Ex. 1041, 2. In
the Reply, Petitioner advocated for the District Court’s construction. Reply
14–15. As explained above, we accept the parties’ agreement, and the
District Court’s ruling, that “speaker-independent speech recognition
device” requires a speech recognition device that recognizes spoken words
without using predefined voice patterns.
Against the backdrop of this construction, Patent Owner argues that,
despite Ladd’s statement that ASR 254 “provides speaker independent
automatic speech recognition of speech inputs,” Ex. 1004, 9:27–29, Ladd’s
ASR “is expressly reliant on recognizing a voice pattern, something that is
expressly disclaimed from the ’431 patent.” PO Resp. 26–27, 35–36. Patent
Owner relies on the statements (quoted above) in Ladd cited in the Petition
to show that ASR 254 performs speech recognition by recognizing pre-
defined voice patterns. Id. at 36–38 (citing Ex. 1004, 8:19–28, 9:35–40).
According to Patent Owner, “[a]lthough, Ladd uses the term ‘speaker
independent’ twice in its disclosure, it expressly teaches away from the type
of speech recognition required by the ’431 Patent.” Id. at 37.
In the Reply, Petitioner acknowledges that “Ladd includes several
teachings of determining whether a speech (or voice) pattern matches stored
IPR2020-00686
Patent 7,076,431 B2
27
grammars or vocabulary and defines the speech/voice pattern as a key word
or phrase.” Reply 3 (citing Ex. 1004, 4:23–25, 8:23–25, 9:32–39).
Nevertheless, Petitioner argues, the voice patterns in Ladd are different from
those excluded by the ’431 patent, both because Ladd’s voice patterns are
used in a different step of the speech recognition process and because they
correspond to keywords, rather than spectral energy. Id. at 3–4.
As to the steps in the speech recognition process, Petitioner argues,
citing to its cross-examination of Mr. Occhiogrosso, “[s]peech recognition
that determines the content of the spoken word is broadly divided into two
steps: (1) converting the spoken utterance into text, i.e., words; and
(2) determining the content of the recognized words, e.g., determining if
words are keywords that issue a command.” Id. at 5 (citing Ex. 1039,
51:14–52:22); see also id. at 6 (“Thus, per Mr. Occhiogrosso, speech
recognition as described in the ’431 Patent first recognizes a word by
converting speech into text, and then compares the word to a recognition
grammar to identify keywords.” (citing Ex. 1039, 38:5–39:2, 52:19–22)). In
the first step, Petitioner argues, a speech recognition device can convert a
user’s spoken words into text using various techniques, including those that
do and do not analyze voice patterns. Id. at 4–6 (citing Ex. 1039, 11:9–13,
15:4–20, 24:17–25:6, 33:6–20, 40:10–22, 43:16–44:7, 50:21–51:8, 51:14–
52:22, 54:6–55:7). After speech is converted to text, the text can be
compared to a recognition grammar in a second step to determine the
content of the text. Id. at 4 (citing Ex. 1039, 66:5–67:17).
Petitioner argues that, “[p]er Parus, the ’431 Patent excludes using
voice patterns to convert speech into text during the first step of speech
recognition, i.e., when converting the spoken utterance into text.” Reply 7
(citing Ex. 1039, 29:14–23). In contrast, Petitioner argues, “Ladd’s
IPR2020-00686
Patent 7,076,431 B2
28
speech/voice pattern is determined at the second step of identifying the
instructed command in the Ladd IVR system (i.e., identifying a key word or
phrase) and subsequent to converting the speech into text.” Id. at 4; see also
id. at 7 (“In contrast, Ladd teaches the automatic speech recognition (ASR)
unit 254 first recognizes the words from the user’s speech input and then
performs the second step of determining whether the speech inputs match
any key word or phrase via comparison to a stored grammar or vocabulary.”
(citing Ex. 1004, 8:23–25, 9:31–36; Ex. 1040 ¶¶ 4–8)), 11 (“Per
Mr. Occhiogrosso, using a vocabulary (or grammar) to detect speech occurs
after the spoken words are converted into text.” (citing Ex. 1039, 49:9–19,
50:17–51:6)).
We note that Petitioner’s position is not entirely clear. On one hand,
Petitioner appears to argue that the speech recognition process is divided
into two steps, first converting speech into text, e.g., using phonemes, and
second determining the content of the word, e.g., using a recognition
grammar. Reply 4. On the other hand, Petitioner also appears to argue that
the speech recognition process only corresponds to the first step of
converting an utterance into text, while the second step of determining the
content of the recognized speech (e.g., matching the text to vocabularies or
grammars) occurs after speech recognition has completed. Id. at 7–8.6
Neither argument is persuasive. If Petitioner is arguing that only the
first step is speech recognition, the Petition expressly identifies Ladd’s
disclosure of identifying or matching voice patterns as part of that first step
of speech recognition claimed in limitation 1[c]. Pet. 22–23; Ex. 1003 ¶ 90.

6 For its part, Patent Owner disagrees that speech recognition requires two
steps, and accuses Petitioner of attempting to conflate speech recognition
and natural language processing. Sur-reply 10–15.
IPR2020-00686
Patent 7,076,431 B2
29
Petitioner’s characterization, in the Reply, of this disclosure in Ladd as
corresponding to an unclaimed second step not part of speech recognition
would be an impermissible new argument presented for the first time in the
Reply. As explained in the Board’s Trial Practice Guide, Petitioner’s change
in direction exceeds the proper scope of a reply:
Generally, a reply or sur-reply may only respond to arguments
raised in the preceding brief. 37 C.F.R. § 42.23, except as
noted above. “Respond,” in the context of 37 C.F.R.
§ 42.23(b), does not mean proceed in a new direction with a
new approach as compared to the positions taken in a prior
filing. While replies and sur-replies can help crystalize issues
for decision, a reply or sur-reply that raises a new issue or
belatedly presents evidence may not be considered. The Board
is not required to attempt to sort proper from improper portions
of the reply or sur-reply.
Patent Trial and Appeal Board Consolidated Trial Practice Guide 74
(November 2019) (“TPG”).7 If Petitioner is arguing that Ladd’s description
of voice patterns and speech patterns is not describing speaker-independent
speech recognition, then Petitioner’s Reply does not simply respond to the
Patent Owner Response or attempt to crystalize the issues in dispute.
Rather, the Petition expressly relied on certain disclosure in Ladd to show
the functionality claimed in limitation 1[c], while the Reply attempts to re-
cast that same disclosure as describing functionality separate from that of
limitation 1[c]. This would be a new theory.
If it is Petitioner’s theory that “speaker-independent speech
recognition” is a two-step process and the District Court’s restriction on
using predefined voice patterns only applies to the first step, Petitioner has
not supported this attempt to narrowly construe (indeed, redraft) the District

7 Available at https://www.uspto.gov/TrialPracticeGuideConsolidated.
IPR2020-00686
Patent 7,076,431 B2
30
Court’s construction. As noted above, at Petitioner’s insistence, we construe
“speaker-independent speech recognition device” to mean “speech
recognition device that recognizes spoken words without using predefined
voice patterns.” On its face, this construction precludes a speech recognition
device that recognizes spoken words using predefined voice patterns at any
step, not just Petitioner’s newly advocated first step.
The parties appear to agree (PO Resp. 21–22 (citing Ex. 1001, 4:34–
43); Reply 2–3 (citing Ex. 1001, 4:38–43)) that the basis for the District
Court’s construction is the ’431 patent’s description that “[t]he voice
browsing system recognizes naturally spoken voice commands and is
speaker-independent; it does not have to be trained to recognize the voice
patterns of each individual user. Such speech recognition systems use
phonemes to recognize spoken words and not predefined voice patterns.”
Ex. 1001, 4:38–43. This disclosure does not break speech recognition into
multiple steps or suggest that the restriction on using predefined voice
patterns only applies to a part of the process of speech recognition. The
process of recognizing speech, according to the ’431 patent, uses phonemes,
not predefined voice patterns.
The cross-examination testimony of Mr. Occhiogrosso on which
Petitioner relies explains that the ’431 patent’s descriptions of matching
keywords to grammar or vocabulary happen after its speech recognition
based on phonemes. See, e.g., Ex. 1039, 5:4–20, 38:5–39:2, 51:14–52:22.
He does not testify that the ’431 patent defines “speaker-independent speech
recognition device” as the two-step process Petitioner now proposes.
Thus, we are not persuaded that the District Court’s construction of
“speaker-independent speech recognition” should be construed further as a
two-step process, only the first step of which includes a restriction on using
IPR2020-00686
Patent 7,076,431 B2
31
predefined voice patterns. Ladd describes identifying selected speech
patterns and recognized voice patterns as part of its speaker-independent
automatic speech recognition. This is contrary to claim limitation 1[c], as
construed.
As noted above, Petitioner also contends that the “voice patterns”
described in Ladd are different in kind from those discussed (and
distinguished) in the ’431 patent. Reply 8–12. Relying on Dr. Terveen’s
testimony and Ladd’s disclosure, Petitioner argues that the “voice patterns”
described in Ladd are key words or phrases. Id. at 8–9 (citing Ex. 1040
¶¶ 11–14), 9–10 (citing Ex. 1004, 4:15–18, 6:50–57, 8:58–61, 9:28–44,
10:3–20, 17:17–27, 23:40–44). Petitioner further argues that
Mr. Occhiogrosso testified in cross-examination that a voice pattern is “a
word or utterance, and its spectral energy . . . as a function of time.” Id. at 3
(citing Ex. 1039, 25:12–17, 25:22–26:13, 30:10–16).
According to Petitioner, Patent Owner has not established that Ladd’s
voice patterns are the same as what Mr. Occhiogrosso testified is the type of
voice pattern excluded by the ’431 patent. Id. at 10–12. Rather, Petitioner
argues, Ladd’s keyword voice patterns would be used in the second step of
the two steps discussed above, while voice patterns of the ’431 patent,
represented as spectral energy, would be used in the first step. Id. at 11
(“Because the ‘voice patterns’ defined by Parus are analyzed to convert
words into text, and because Ladd’s speech patterns are detected with a
vocabular[y]/grammar, Ladd’s speech patterns cannot be the same as the
excluded ‘voice patterns.’”); see also id. at 7 (“Per Parus, the ’431 Patent
excludes using voice patterns to convert speech into text during the first step
of speech recognition, i.e., when converting the spoken utterance into text.
In contrast, Ladd teaches the automatic speech recognition (ASR) unit 254
IPR2020-00686
Patent 7,076,431 B2
32
first recognizes the words from the user’s speech input and then performs
the second step of determining whether the speech inputs match any key
word or phrase via comparison to a stored grammar or vocabulary.”
(citations omitted)). Thus, Petitioner argues, “because Ladd’s determined
‘voice patterns’ are materially different than the ‘predefined voice patterns’
Parus contends are excluded by its claim construction, Ladd teaches
Claim 1(c).” Id. at 11–12.
Patent Owner argues that “voice pattern” should be given its ordinary
meaning and that no additional construction is necessary. Sur-reply 4–6.
We are not persuaded by Petitioner’s arguments. As explained above,
Petitioner has not shown that the ’431 patent distinguishes between
predefined voice patterns used in a first step of voice recognition and voice
patterns used in a second step. Nor does Petitioner point to any persuasive
support in the ’431 patent to exclude certain predefined voice patterns (e.g.,
keywords) from the District Court’s construction.
Patent Owner has introduced persuasive evidence that Ladd’s speaker-
independent automatic speech recognition uses predefined voice patterns
under the District Court’s construction of “speaker-independent speech
recognition device.” PO Resp. 34–38; Sur-reply 6–8. Petitioner has not
convinced us to narrow the District Court’s construction or that Ladd’s
express description of selected speech patterns and recognized voice patterns
(Ex. 1004, 9:35–38) do not correspond to the predefined voice patterns
excluded by the District Court’s construction. Accordingly, we find that
Ladd’s speaker-independent automatic speech recognition uses predefined
voice patterns, contrary to claim limitation 1[c], as we construe it. In any
case, Petitioner has not met its burden to show that Ladd performs speaker-
independent speech recognition without using predefined voice patterns.
IPR2020-00686
Patent 7,076,431 B2
33
Moreover, even if we were to determine that Ladd’s descriptions of
voice patterns pertained to a second step apart from its description of
speaker-independent automatic speech recognition, Petitioner still cannot
prevail because Petitioner has not shown that Ladd’s speaker-independent
automatic speech recognition (assuming it corresponds to the newly alleged
first step) is performed without using predefined voice patterns. If we accept
Petitioner’s argument that the statements in Ladd discussing speech or voice
patterns correspond to a second step of speech recognition (or a second step
that happens after speech recognition is completed), Petitioner’s cited
support in Ladd for limitation 1[c] would be:
The ASR unit 254 of the VRU server 234 provides speaker
independent automatic speech recognition of speech inputs or
communications from the user. . . . The ASR unit 254 is
preferably a speaker independent speech recognition software
package, Model No. RecServer, available from Nuance
Communications. It is contemplated that the ASR unit 254 can
be any suitable speech recognition unit to detect voice
communications from a user.
Ex. 1004, 9:28–32, 9:39–44. Here, Ladd does not describe how it
recognizes speech. Petitioner acknowledged this at the oral argument.
Tr. 47:20–24 (“I think what’s going on at this discussion regarding the
detection unit in Ladd is speech recognition is so well-known that the patent
drafter simply said we’re going to look at audio inputs and compare to the
recognition grammar and didn’t describe all the intermediate steps.”).
Mr. Occhiogrosso testified on cross-examination that there are a variety of
methods in the art for recognizing words spoken by a user, including some
that use voice patterns and some (e.g., artificial intelligence or using
phonemes) that do not. Ex. 1039, 54:6–55:7. Petitioner confirmed at the
oral hearing that this is true:
IPR2020-00686
Patent 7,076,431 B2
34
MS. BAILEY: . . . During Mr. Occhiogrosso’s deposition, we
started talking about speech recognition algorithms, and he
explained that voice patterns and analyzing the voice patterns is
just one class of speech recognition algorithms. He talked
about voice patterns. He talked about phonemes. But he said
there’s all kinds of speech recognition algorithms that can be
used to recognize the speech and convert it into text. For
example, you can --
JUDGE McKONE: Do you agree with that statement? That’s
not something you’re challenging, is it?
MS. BAILEY: We’re not challenging that there are different
types of speech recognition algorithms.
JUDGE McKONE: Okay. Some that use voice patterns and
some that do not.
MS. BAILEY: That is correct. And there are others beyond
that. He talked about during his deposition that you could use
AI, you could use neural networks, statistical analyses. There’s
probably even more but those are the ones that he mentioned
during his deposition.
Tr. 15:20–16:12. Dr. Terveen testifies that “[t]here are a number of methods
by which a system may perform this first step of converting the spoken
words into text, but Ladd is not specific on how it requires step one to
occur.” Ex. 1040 ¶ 3 (emphasis added). Thus, based on the testimony of
both experts, simply saying that a device uses “speaker independent
automatic speech recognition,” as Ladd does (Ex. 1004, 9:28–30), does not
establish that the device performs that recognition “without using predefined
voice patterns,” as the claim construction for limitation 1[c] requires.
Petitioner does not introduce persuasive evidence to fill the gap
between Ladd’s silence on how it performs “speaker independent automatic
speech recognition” and what Petitioner would need to establish
performance of speech recognition without using predefined voice patterns.
The argument and evidence submitted with the Reply does not add to Ladd’s
IPR2020-00686
Patent 7,076,431 B2
35
disclosure materially. The thrust of Petitioner’s Reply arguments is that
Patent Owner failed to show an equivalency between the ’431 patent’s
“voice patterns” and Ladd’s “speech” and “voice” patterns. Reply 8–12.
However, Petitioner bears the burden to show that Ladd performs
recognition without using predefined speech patterns; Patent Owner bears no
burden to show that Ladd performs recognition using predefined speech
patterns. See 35 U.S.C. § 316(e) (“In an inter partes review instituted under
this chapter, the petitioner shall have the burden of proving a proposition of
unpatentability by a preponderance of the evidence.”); Dynamic Drinkware,
LLC v. Nat’l Graphics, Inc., 800 F.3d 1375, 1378 (Fed. Cir. 2015).
Petitioner’s Reply and Dr. Terveen’s testimony do not show how
Ladd’s ASR 254 performs its speaker independent automatic speech
recognition, and specifically do not show that it performs recognition
without using predefined speech patterns. Reply 3–12; Ex. 1003 ¶ 90;
Ex. 1040 ¶¶ 2–25. At the oral argument, Petitioner was unable to point to
affirmative evidence supporting its position:
JUDGE McKONE: Now, does Ladd have any specific
disclosure that this speaker independent automatic speaker
recognition is happening without the use of voice patterns?
MS. BAILEY: Well, let me – that’s actually where I was about
to go. So --
JUDGE McKONE: I have read in your brief that you are
saying – you’re probably going to tell me that the parts you
highlighted in blue do not pertain to the speech recognition.
And I’d be happy to hear about that in a moment. But is there
anything affirmative you want to point me to that shows that
this ASR unit is performing speaker independent automatic
speech recognition without the use of voice patterns?
MS. BAILEY: There is nothing affirmative that says without
the use of voice patterns, using Parus’s definition of the spectral
IPR2020-00686
Patent 7,076,431 B2
36
energy as a function of time. Ladd doesn’t mention a voice
pattern that is a spectral energy as a function of time. It has no
disclosure to that respect. So Ladd doesn’t have anything that
says the speaker independent speech recognition device does
not identify the -- or analyze the spectral energy of the utterance
as a function of time.
Tr. 19:24–20:18.
As noted above, both the ’431 patent and Ladd mention “Nuance”
software. Ex. 1001, 6:16–21; Ex. 1004, 9:38–41. Ladd’s mention of
Nuance software does not provide persuasive evidence that Ladd’s ASR 254
performs speaker-independent speech recognition without using predefined
voice patterns. Dr. Terveen testifies that
Ladd describes using a commercially-available product from a
company called Nuance to transform speech into text. This is
the same commercially-available product from Nuance that the
’431 and ’084 Patents describe, further indicating to me that
Ladd . . . teaches a speaker independent speech recognition
device substantially similar to the speaker independent speech
recognition device described and claimed in the ’431 Patent.
Ex. 1040 ¶ 9 (citing Ex. 1001, 6:4–24; Ex. 1004, 8:23–28). Dr. Terveen’s
testimony does not state the basis for his conclusion that Ladd and the ’431
patent describe the same Nuance software, rather than different software
made by the same company,8 and we see no evidence to support that
testimony. See 37 C.F.R. § 42.65(a) (“Expert testimony that does not
disclose the underlying facts or data on which the opinion is based is entitled
to little or no weight.”). In any case, Petitioner admitted at the oral argument

8 The ’431 patent simply states that “[a] preferred speech recognition engine
is developed by Nuance Communications,” without further identifying the
software. Ex. 1001, 6:16–19; see also id. at 18:35–37 (same). Ladd states
that “[t]he voice recognition engine is preferably a RecServer software
package, available from Nuance Communications.” Ex. 1004, 8:25–28.
IPR2020-00686
Patent 7,076,431 B2
37
that the record does not include any evidence to support Dr. Terveen’s
opinion on the Naunce software. Tr. 46:10–14 (“JUDGE McKONE: . . . Do
we have any evidence in the record as to what this Nuance software is and
whether or not it is the same between Ladd and the challenged patents?
MS. BAILEY: No, we do not, Your Honor.”).
In light of both experts’ testimony that “speaker-independent speech
recognition” can be performed in a variety of ways (including using
predefined voice patterns), and Petitioner’s failure to offer evidence of how
Ladd’s ASR 254 operates, we find that, at most, Ladd is silent as to whether
its “speaker independent automatic speech recognition” is performed
“without using predefined voice patterns.” This is insufficient for Petitioner
to meet its burden of proof.9
At the oral hearing, Petitioner attempted to introduce another theory,
specifically, that the disclosure in Ladd that Petitioner relied upon for

9 AC Technologies S.A. v. Amazon.com, Inc., 912 F.3d 1358 (Fed. Cir.
2019), and Sud-Chemie, Inc. v. Multisorb Techs., Inc., 554 F.3d 1001 (Fed.
Cir. 2009) do not lead to a different outcome. In AC Technologies, the
Federal Circuit determined that substantial evidence supported a Board
finding that prior art taught copying that was independent of an access to a
computer unit, despite the lack of an express disclosure of this “negative
limitation,” based on inferences the Board drew from the prior art’s
disclosure and expert testimony explaining the prior art. 912 F.3d at 1366–
67. In Sud-Chemie, the Federal Circuit affirmed a district court finding that
prior art disclosed an “uncoated” film based on description in the prior art
that did not describe the film as coated and did not suggest the necessity of
coatings. 554 F.3d at 1004–05. In this proceeding, Petitioner does not need
to show that Ladd affirmatively states a negative limitation. However,
neither Petitioner nor its expert witness provide any persuasive evidence
showing why we should infer from Ladd’s silence that its ASR 254 performs
speaker-independent speech recognition in a manner consistent with claim
limitation 1[c].
IPR2020-00686
Patent 7,076,431 B2
38
claim 5 supports its allegations for claim limitation 1[c]. Tr. 21:14–22:15
(citing Ex. 1004, 14:17–42). Petitioner admits that it did not raise this
argument either in its Petition or Reply. Id. at 44:24–45:4 (“JUDGE
McKONE: . . . the evidence you point to for claim 5 that you . . . would now
like to point to for claim 1 in Ladd, was that raised at all in the reply for
claim 1? MS. BAILEY: It was raised with respect to claims 5 and 6.
Referring to claim 1, it was not raised.”). This argument is waived. See Dell
Inc. v. Acceleron, LLC, 884 F.3d 1364, 1369 (Fed. Cir. 2018) (The “Board
was obligated to dismiss [the petitioner’s] untimely argument . . . raised for
the first time during oral argument.”).
In sum, the evidence presented in the Petition shows that Ladd teaches
speaker-independent speech recognition using predefined voice patterns,
contrary to the requirement of claim limitation 1[c]. We are not persuaded
by Petitioner’s new reply arguments, which attempt, unsuccessfully, to
redraft the District Court’s construction of limitation 1[c]. Nevertheless,
those arguments would not be persuasive because Petitioner has not
introduced evidence sufficient to show that Ladd teaches speaker-
independent speech recognition without using predefined voice patterns if
claim limitation 1[c] is construed to be a two-step process. Petitioner has
not shown, by a preponderance of the evidence, that Ladd, Kurosawa, and
Goedken teach “at least one speaker-independent speech recognition device,
said speaker-independent speech recognition device operatively connected to
said computer and to said voice enabled device,” as recited in claim
limitation 1[c]. Thus, Petitioner has not shown, by a preponderance of the
evidence, that Ladd, Kurosawa, and Goedken teach all limitations of
claim 1.

IPR2020-00686
Patent 7,076,431 B2
39
c) Claims 2–6, 9, 10, 13, 14, 18, 20, 21, and 25
For independent claim 18, Petitioner relies primarily on its showing
for claim 1. Pet. 58–61. Claim limitation 18[b] recites, inter alia, “said
computer further being operatively connected to at least one speaker-
independent speech recognition engine.” Petitioner incorporates its
allegations for claim limitation 1[c] for claim limitation 18[b]. Id. at 58.
The Reply does not address claim 18 separately. As explained above,
Petitioner has not shown that Ladd, Kurosawa, and Goedken teach claim
limitation 1[c]. For the same reasons, they do not teach limitation 18[b].
Thus, Petitioner has not shown, by a preponderance of the evidence, that
Ladd, Kurosawa, and Goedken teach all limitations of claim 18.
Claims 2–6, 9, 10, 13, and 14 depend from claim 1. Claims 20, 21,
and 25 depend from claim 18. Petitioner’s allegations as to these dependent
claims do not cure the defects with respect to its presentation for
independent claims 1 and 18. Pet. 48–58, 61. Thus, Petitioner has not
shown, by a preponderance of the evidence, that Ladd, Kurosawa, and
Goedken teach all limitations of claims 2–6, 9, 10, 13, 14, 20, 21, and 25.

4. Conclusion
As explained above, Petitioner has not shown that Ladd, Kurosawa,
and Goedken teach all limitations of claims 1–6, 9, 10, 13, 14, 18, 20, 21,
and 25. Neither party argues or introduces evidence of objective indicia of
nonobviousness or obviousness. In sum, upon consideration of all the
evidence, we conclude that Petitioner has not proved by a preponderance of
the evidence that claims 1–6, 9, 10, 13, 14, 18, 20, 21, and 25 would have
been obvious over Ladd, Kurosawa, and Goedken.
IPR2020-00686
Patent 7,076,431 B2
40
C. Obviousness of Claims 7, 19, and 26–30 over Ladd, Kurosawa,
Goedken, and Madnick; Obviousness of Claims 5 and 6 over Ladd,
Kurosawa, Goedken, and Houser; and Obviousness of Claims 9 and
25 over Ladd, Kurosawa, Goedken, and Rutledge
Petitioner contends that claims 7, 19, and 26–30 would have been
obvious over Ladd, Kurosawa, Goedken, and Madnick. Pet. 62–67.
Petitioner adds Madnick to show the limitations of claims 7, 19, and 26
related to a “content descriptor” associated with each said web site address,
and not to show additional disclosure for claim limitation 1[c]. Id. at 62–66.
As to independent claim 26 in particular, Petitioner incorporates by
reference its allegations for claim limitation 1[c] as to claim limitation 26[b],
which recites “at least one speaker-independent speech recognition engine.”
Id. at 64. Claims 27–30 depend from claim 26. Petitioner incorporates by
reference its arguments for claims 2–4 (which depend from claim 1) as to
these claims. Id. at 66–67. Petitioner’s allegations as to these claims do not
cure the defects with respect to its presentation for claim 1. For the reasons
given for claim 1, Petitioner has not shown that Ladd, Kurosawa, Goedken,
and Madnick teach all limitations of claims 7, 19, and 26–30.
Petitioner proposes additional grounds for claims 5 and 6 (which
depend from claim 1) adding Houser, and for claims 9 and 25 (which depend
from claims 1 and 18, respectively) adding Rutledge. Pet. 27–71.
Petitioner’s allegations as to these claims do not cure the defects with
respect to its presentations for claims 1 and 18.
For the reasons given for claim 1, Petitioner has not shown that Ladd,
Kurosawa, Goedken, and Houser teach all limitations of claims 5 and 6 or
that Ladd, Kurosawa, Goedken, and Rutledge teach all limitations of claims
9 and 25.
IPR2020-00686
Patent 7,076,431 B2
41
Neither party argues or introduces evidence of objective indicia of
nonobviousness or obviousness for claims 5–7, 9, 19, and 25–30.
In sum, upon consideration of all the evidence, we conclude that
Petitioner has not proved by a preponderance of the evidence that claims 5–
7, 9, 19, and 25–30 would have been obvious.

III. PATENT OWNER’S MOTION TO EXCLUDE
Patent Owner moves to exclude paragraphs 2–25 of Dr. Terveen’s
Supplemental Declaration (Ex. 1040) for three reasons. First, Patent Owner
argues that this testimony does not respond to arguments raised in the Patent
Owner Response. Paper 29, 2–5. Second, Patent Owner argues that this
testimony is an unauthorized and late submission of supplemental
information under 37 C.F.R. § 42.123(b). Id. at 6–9. Third, Patent Owner
argues that this testimony was improperly incorporated by reference into the
Reply. Id. at 9.
Petitioner argues that Patent Owner’s bases for moving to exclude are
not proper and that, to challenge the scope of Dr. Terveen’s declaration,
Patent Owner should have filed a motion to strike. Paper 30, 1, 4–6.
A motion to exclude should be directed to the admissibility of
evidence. See 37 C.F.R. § 42.64; TPG 79 (“A motion to exclude must
explain why the evidence is not admissible (e.g., relevance or hearsay) but
may not be used to challenge the sufficiency of the evidence to prove a
particular fact.”). According to the Trial Practice Guide, a motion to exclude
is not a vehicle to “address arguments or evidence that a party believes
exceeds the proper scope of reply or sur-reply.” TPG 79. Rather, “[i]f a
party believes that a brief filed by the opposing party raises new issues, is
accompanied by belatedly presented evidence, or otherwise exceeds the
IPR2020-00686
Patent 7,076,431 B2
42
proper scope of reply or sur-reply, it may request authorization to file a
motion to strike.” Id. at 80. Patent Owner did not file a motion to strike.
Patent Owner’s Motion to Exclude does not address the admissibility
of Dr. Terveen’s testimony. Rather, it argues that the testimony exceeds the
proper scope of reply evidence, because it does not respond to the Patent
Owner Response, because Petitioner did not follow our rules in seeking to
submit supplemental information, or because Petitioner attempts to
incorporate material by reference improperly. Paper 29. Thus, we agree
with Petitioner that Patent Owner’s Motion to Exclude does not state a
proper basis for excluding evidence and, therefore, we deny the motion for
that reason.

IV. CONCLUSION10
Petitioner has not shown by a preponderance of the evidence that
claims 1–7, 9, 10, 13, 14, 18–21, and 25–30 would have been obvious.

10 Should Patent Owner wish to pursue amendment of the challenged claims
in a reissue or reexamination proceeding subsequent to the issuance of this
decision, we draw Patent Owner’s attention to the April 2019 Notice
Regarding Options for Amendments by Patent Owner Through Reissue or
Reexamination During a Pending AIA Trial Proceeding. See 84 Fed. Reg.
16,654 (Apr. 22, 2019). If Patent Owner chooses to file a reissue application
or a request for reexamination of the challenged patent, we remind Patent
Owner of its continuing obligation to notify the Board of any such related
matters in updated mandatory notices. See 37 C.F.R. § 42.8(a)(3), (b)(2).
IPR2020-00686
Patent 7,076,431 B2
43
In summary:

V. ORDER
In consideration of the foregoing, it is hereby:
ORDERED, based on a preponderance of the evidence, that claims 1–
7, 9, 10, 13, 14, 18–21, and 25–30 have not been shown to be unpatentable;
FURTHER ORDERED that Patent Owner’s Motion to Exclude
Exhibit 1040 is denied; and
FURTHER ORDERED, because this is a final written decision, the
parties to this proceeding seeking judicial review of our Decision must
comply with the notice and service requirements of 37 C.F.R. § 90.2.

Claims

35 U.S.C. § Reference(s)/
Basis
Claims
Shown
Unpatentable
Claims
Not shown
Unpatentable
1–6, 9,
10, 13,
14, 18,
20, 21, 25
103 Ladd,
Kurosawa,
Goedken
1–6, 9, 10, 13,
14, 18, 20, 21,
25
7, 19, 26–
30
103 Ladd,
Kurosawa,
Goedken,
Madnick
7, 19, 26–30
5, 6 103 Ladd,
Kurosawa,
Goedken,
Houser
5, 6
9, 25 103 Ladd,
Kurosawa,
Goedken,
Rutledge
9, 25
Overall
Outcome
1–7, 9, 10, 13,
14, 18–21,
25–30
IPR2020-00686
Patent 7,076,431 B2
44
FOR PETITIONER:
Jennifer C. Bailey
Adam P. Seitz
ERISE IP, P.A.
jennifer.bailey@eriseip.com
adam.seitz@eriseip.com

FOR PATENT OWNER:
Michael J. McNamara
Michael T. Renaud
William A. Meunier
MINTZ, LEVIN, COHN, FERRIS, GLOVSKY AND
POPEO, P.C.
mmcnamara@mintz.com
mtrenaud@mintz.com
wameunier@mintz.com