Toyota Motor Corporationv.American Vehicular Sciences LLC

Patent Trial and Appeal Board

Jan 12, 2015

10302105 (P.T.A.B. Jan. 12, 2015)

Trials@uspto.gov Paper 59
571-272-7822 Entered: January 12, 2015

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

TOYOTA MOTOR CORPORATION,
Petitioner,

v.

AMERICAN VEHICULAR SCIENCES LLC,
Patent Owner.
____________

Case IPR2013-00419
Patent 6,772,057 B2
____________

Before JAMESON LEE, TREVOR M. JEFFERSON, and
LYNNE E. PETTIGREW, Administrative Patent Judges.

PETTIGREW, Administrative Patent Judge.

FINAL WRITTEN DECISION
35 U.S.C. § 318(a) and 37 C.F.R. § 42.73

I. BACKGROUND
Petitioner, Toyota Motor Corporation (“Toyota”), filed a Petition for
inter partes review of claims 1–4, 7–10, 30–34, 37–41, 43, 46, 48, 49, 56,
59–62, and 64 of U.S. Patent No. 6,772,057 B2 (Ex. 1001, “the ’057
patent”). Paper 3 (“Pet.”). Patent Owner, American Vehicular Sciences
LLC (“AVS”), filed a Preliminary Response. Paper 17 (“Prelim. Resp.”).
IPR2013-00419
Patent 6,772,057 B2
2
On January 13, 2014, pursuant to 35 U.S.C. § 314, we instituted an inter
partes review for all challenged claims on certain grounds of unpatentability
asserted in the Petition. Paper 19 (“Dec. on Inst.”).
Subsequent to institution, AVS filed a Patent Owner Response (Paper
33), and later filed a Revised Patent Owner Response (Paper 45, “PO
Resp.”).
1
Toyota filed a Reply to Patent Owner’s Response (Paper 40, “Pet.
Reply”).
A consolidated oral hearing for this proceeding and Toyota Motor
Corp. v. American Vehicular Sciences, LLC, IPR2013-00424, involving the
same parties and similar issues, was held on August 18, 2014. A transcript
of the consolidated hearing is included in the record. Paper 58 (“Tr.”).
We have jurisdiction under 35 U.S.C. § 6(c). This Final Written
Decision is issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73.
As explained below, Toyota has shown by a preponderance of the
evidence that claims 30, 32–34, 37–40, 43, 46, 48, and 49 of the ’057 patent
are unpatentable, but Toyota has not shown by a preponderance of the
evidence that claims 1–4, 7–10, 31, 41, 56, 59–62, and 64 of the ’057 patent
are unpatentable.
A. Related Proceedings
Toyota and AVS indicate that the ’057 patent has been asserted by
AVS in the following district court cases: American Vehicular Sciences
LLC v. Toyota Motor Corp., No. 6:12-cv-00410 (E.D. Tex.) (filed June 25,
2012); American Vehicular Sciences LLC v. BMW Group, No. 6:12-cv-

1
We authorized AVS to file a Revised Patent Owner Response to make
certain non-substantive corrections to the Patent Owner Response. See
Papers 39, 44 (Orders on Conduct of the Proceedings).
IPR2013-00419
Patent 6,772,057 B2
3
00415 (E.D. Tex.) (filed June 25, 2012); American Vehicular Sciences LLC
v. Subaru of Am. Inc., No. 6:12-cv-00230 (E.D. Tex.) (filed Mar. 8, 2013);
and American Vehicular Sciences LLC v. Mercedes-Benz U.S. Int’l, Inc.,
No. 6:13-cv-00309 (E.D. Tex.) (filed Apr. 3, 2013). Pet. 1; Paper 27, 2–3.
B. The’057 Patent
The ’057 patent, titled “Vehicular Monitoring Systems Using Image
Processing,” generally relates to a vehicle monitoring arrangement for
monitoring an environment exterior of a vehicle. Ex. 1001, Abstract. One
embodiment of such an arrangement described in the ’057 patent includes a
transmitter that transmits electromagnetic waves into the environment
exterior of a vehicle and one or more receivers that receive reflections of the
transmitted waves from exterior objects, such as approaching vehicles.
Id. at 14:8–12, 14:32–37, 38:7–13, Fig. 7. In a preferred implementation,
the transmitter is an infrared transmitter, and the receivers are CCD (charge
coupled device) transducers that receive the reflected infrared waves.
Id. at 38:10–12, 39:25–28. One or more receivers may be arranged on a rear
view mirror of the vehicle. Id. at 14:58–60, 38:22–25. The system also may
include radar or pulsed laser radar (lidar) for measuring distance between the
vehicle and exterior objects. Id. at 14:38–40, 39:1–6.
The waves received by the receivers contain information about
exterior objects in the environment, and the receivers generate signals
characteristic of the received waves. Id. at 14:12–14, 39:44–49. A trained
pattern recognition means, such as a neural computer or neural network,
processes the signals to provide a classification, identification, or location of
an exterior object. Id. at 14:17–25, 39:49–54. Training of a neural network
to provide classification, identification, or location of objects is
IPR2013-00419
Patent 6,772,057 B2
4
accomplished by conducting a large number of experiments in which the
system is taught to differentiate among received signals corresponding to
different objects. Id. at 36:22–39 (describing a neural network training
session in connection with an embodiment that monitors an interior of a
vehicle, particularly the passenger seat). The classification, identification, or
location of an exterior object may be used to affect operation of other
systems in the vehicle, e.g., to show an image or icon on a display viewable
by a driver or to deploy an airbag. Id. at 14:21–31, 39:54–62.
C. Illustrative Claims
Of the challenged claims, claims 1, 30, 40, and 56 are independent.
Claims 1, 30, and 40 are illustrative:
1. A monitoring arrangement for monitoring an
environment exterior of a vehicle, comprising:
at least one receiver arranged to receive waves from the
environment exterior of the vehicle which contain information
on any objects in the environment and generate a signal
characteristic of the received waves; and
a processor coupled to said at least one receiver and
comprising trained pattern recognition means for processing the
signal to provide a classification, identification or location of
the exterior object, said trained pattern recognition means being
structured and arranged to apply a trained pattern recognition
algorithm generated from data of possible exterior objects and
patterns of received waves from the possible exterior objects to
provide the classification, identification or location of the
exterior object;
whereby a system in the vehicle is coupled to said
processor such that the operation of the system is affected in
response to the classification, identification or location of the
exterior object.
IPR2013-00419
Patent 6,772,057 B2
5
30. A vehicle including a monitoring arrangement for
monitoring an environment exterior of the vehicle, the
monitoring arrangement comprising:
at least one receiver arranged on a rear view mirror of
the vehicle to receive waves from the environment exterior of
the vehicle which contain information on any objects in the
environment and generate a signal characteristic of the received
waves; and
a processor coupled to said at least one receiver and
arranged to classify or identify the exterior object based on the
signal and thereby provide the classification or identification of
the exterior object;
whereby a system in the vehicle is coupled to said
processor such that the operation of the system is affected in
response to the classification or identification of the exterior
object.
40. A monitoring arrangement for monitoring an
environment exterior of a vehicle, comprising:
a plurality of receivers arranged apart from one another
and to receive waves from different parts of the environment
exterior of the vehicle which contain information on any objects
in the environment and generate a signal characteristic of the
received waves; and
a processor coupled to said receivers and arranged to
classify, identify or locate the exterior object based on the
signals generated by said receivers and thereby provide the
classification[,] identification or location of the exterior object,
whereby a system in the vehicle is coupled to said
processor such that the operation of the system is affected in
response to the classification, identification or location of the
exterior object.
Ex. 1001, 54:13–32, 55:58–56:6, 56:37–52 (emphases added).
IPR2013-00419
Patent 6,772,057 B2
6
D. Pending Grounds of Unpatentability
We instituted inter partes review based on the following grounds of
unpatentability asserted in the Petition:
Reference(s) Basis Claim(s)
Lemelson
2
35 U.S.C. § 102(e)
1–4, 7–10, 40, 41, 46, 48,
49, 56, 59–61, and 64
Lemelson and
Borcherts
3

35 U.S.C. § 103(a) 30–34, 37–39, and 62
Lemelson and
Asayama
4

35 U.S.C. § 103(a) 4, 43, and 59
Lemelson, Borcherts,
and Asayama
35 U.S.C. § 103(a) 34
Yamamura
5
and
Borcherts
35 U.S.C. § 103(a) 30, 32, and 37–39
Dec. on Inst. 38.
II. ANALYSIS
A. Claim Construction
In the Decision on Institution, we construed several claim terms of the
’057 patent, as set forth in the following table:

2
U.S. Patent No. 6,553,130, issued Apr. 22, 2003 (Ex. 1002).
3
U.S. Patent No. 5,245,422, issued Sept. 14, 1993 (Ex. 1004).
4
U.S. Patent No. 5,214,408, issued May 25, 1993 (Ex. 1005).
5
Japanese Unexamined Patent Application Publication No. H06-124340,
published May 6, 1994 (Ex. 1012). Citations to Yamamura refer to its
English translation (Ex. 1013).
IPR2013-00419
Patent 6,772,057 B2
7
Claim Term Board’s Construction
“trained pattern
recognition algorithm”
“an algorithm that processes a signal that is
generated by an object, or is modified by
interacting with an object, in order to
determine to which one of a set of classes
the object belongs, the algorithm having
been taught, through a variety of examples,
various patterns of received signals
generated or modified by objects”
“trained pattern
recognition means”
“a neural computer or neural network
trained for pattern recognition, and
equivalents thereof”
“identify”
“determine that the object belongs to a
particular set or class”
“exterior object”
“a material or physical thing outside the
vehicle, not a part of the roadway on which
the vehicle travels”
“rear view mirror” “a mirror that faces to the rear”
“transmitter”
“device that transmits any type of
electromagnetic waves, including visible
light”
Dec. on Inst. 8–15. AVS does not contest these constructions for purposes
of this proceeding, PO Resp. 9–10, and Toyota does not dispute these
constructions in its Reply. We maintain these constructions for this Final
Written Decision.
B. Principles of Law
To prevail in challenging AVS’s claims, Toyota must demonstrate by
a preponderance of the evidence that the claims are unpatentable. 35 U.S.C.
§ 316(e); 37 C.F.R. § 42.1(d). A claim is anticipated if a single prior art
reference either expressly or inherently discloses every limitation of the
claim. Orion IP, LLC v. Hyundai Motor Am., 605 F.3d 967, 975 (Fed. Cir.
2010). A claim is unpatentable under 35 U.S.C. § 103(a) if the differences
IPR2013-00419
Patent 6,772,057 B2
8
between the claimed subject matter and the prior art are such that the subject
matter, as a whole, would have been obvious at the time of the invention to a
person having ordinary skill in the art. KSR Int’l Co. v. Teleflex, Inc., 550
U.S. 398, 406 (2007).
C. Anticipation by Lemelson
Claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61, and 64
Toyota asserts that claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61,
and 64 are unpatentable under 35 U.S.C. § 102(e) as anticipated by
Lemelson. Pet. 10–24. In support of this assertion, Toyota provides detailed
analysis and claim charts explaining how Lemelson discloses each claim
limitation. Id. Toyota also relies on the testimony of Dr. Nikolaos
Papanikolopoulos, as set forth in his Declaration (Ex. 1016) and Reply
Declaration (Ex. 1023).
AVS responds that Lemelson does not disclose each limitation of the
challenged claims. PO Resp. 11–25. For support, AVS relies on the
testimony of Dr. Cris Koutsougeras, as set forth in his Declaration
(Ex. 2001).
Having considered the parties’ contentions and supporting evidence,
we determine that Toyota has demonstrated by a preponderance of the
evidence that claims 40, 46, 48, and 49 are anticipated by Lemelson, but
Toyota has not demonstrated by a preponderance of the evidence that claims
1–4, 7–10, 41, 56, 59–61, and 64 are anticipated by Lemelson.
1. Lemelson
Lemelson discloses a computerized system in a motor vehicle that
identifies possible obstacles on a roadway and either warns the driver or
controls the operation of vehicle systems, such as the brakes or steering
IPR2013-00419
Patent 6,772,057 B2
9
mechanism, to avoid or lessen the effect of a collision. Ex. 1002, Abstract,
5:15–29, 8:38–39. The system includes at least one video camera, mounted
at the front end of the vehicle, such as the front end of the roof, bumper, or
hood. Id. at 5:31–34. The system may include multiple cameras for front,
side, and rear viewing, and for stereo imaging capabilities. Id. at 6:27–42.
The video camera also may be implemented with other technologies,
including infrared imaging methods. Id. at 6:34–37. In addition, the system
may use radar or lidar for range detection. Id. at 5:67–6:4. “[V]ideo
scanning and radar or lidar scanning may be jointly employed to identify and
indicate distances between the controlled vehicle and objects ahead of, to the
side(s) of, and to the rear of the controlled vehicle.” Id. at 6:5–8.
The analog signal output from the video camera(s) is digitized in an
analog-to-digital convertor and passed to an image analyzing computer
(IAC), which is
provided, implemented and programmed using neural networks
and artificial intelligence as well as fuzzy logic algorithms to
(a) identify objects on the road ahead such as other vehicles,
pedestrians, barriers and dividers, turns in the road, signs and
symbols, etc., and generate identification codes, and (b) detect
distances from such objects by their size (and shape) and
provide codes indicating same for use by a decision computer[]
23, which generates coded control signals which are applied
through the computer 11 or are directly passed to various
warning and vehicle operating devices such as a braking
computer or drive[] 35, which operates a brake servo 33, a
steering computer or drive(s) 39 and 40 which operate steering
servos 36; . . . a headlight controller 41 for flashing the head
lights, a warning light control 42 for flashing external and/or
internal warning lights; a horn control 43, etc.

Id. at 5:39–59. The IAC also may display symbols representing hazard
objects. Id. at 6:52–55, 9:60–62.
IPR2013-00419
Patent 6,772,057 B2
10
Lemelson discloses further details regarding a neural network
embodiment of the IAC for identifying objects:
Neural networks used in the vehicle . . . warning system are
trained to recognize roadway hazards which the vehicle is
approaching including automobiles, trucks, and pedestrians.
Training involves providing known inputs to the network
resulting in desired output responses. The weights are
automatically adjusted based on error signal measurements until
the desired outputs are generated. Various learning algorithms
may be applied. Adaptive operation is also possible with on-
line adjustment of network weights to meet imaging
requirements. The neural network embodiment of the image
analysis computer 19 provides a highly parallel image
processing structure with rapid, real-time image recognition
necessary for the Motor Vehicle Warning and Control System.

Id. at 8:1–14.
2. “Generated from” Limitation
A central dispute between the parties is whether Lemelson discloses a
“trained pattern recognition algorithm generated from data of possible
exterior objects and patterns of received waves from the possible exterior
objects.” This limitation appears in independent claims 1 and 56 and in
claim 41, which depends from claim 40. Although we construed “trained
pattern recognition algorithm” in our Decision on Institution, we did not
provide an express construction for the “generated from” language following
that term in the claims. For this Final Written Decision, we construe the
“generated from” limitation according to its broadest reasonable
interpretation in light of the specification of the ’057 patent. See 37 C.F.R.
§ 42.100(b).
AVS contends that the “generated from” limitation in claims 1, 41,
and 56 requires a particular type of training to generate the “trained pattern
IPR2013-00419
Patent 6,772,057 B2
11
recognition algorithm” recited in the claims. PO Resp. 6. Relying on the
Declaration of Dr. Koutsougeras, AVS argues that the ’057 patent discloses
and claims a specific method for training the pattern recognition means
using “data of possible exterior objects and patterns of received waves from
the possible exterior objects.” Id. (citing Ex. 2001 ¶¶ 19, 20, 53). For
example, if a vehicle uses a radar receiver, AVS contends that the claims
require training using “real radar waves” received from actual examples of
possible exterior objects placed in front of the radar system, along with data
indicating the identity or classification of the objects. Id. at 6–7 (citing
Ex. 2001 ¶¶ 19–20). AVS contrasts the use of real waves to train a pattern
recognition system, as recited in the claims, with other methods of training,
such as using simulated data (e.g., a computer simulation of radar waves).
Id. at 7 (citing Ex. 2001 ¶¶ 46, 55–63).
Toyota argues that the “generated from” language of claims 1, 41, and
56 does not require training with real data. Pet. Reply 2, 4. Because the
claims refer to “data of possible exterior objects” and “patterns of received
waves,” rather than “data from” and “patterns from,” Toyota contends that
the claim language encompasses training using simulated data and patterns
that represent possible objects and received waves, respectively.
Id. at 2, 4–5.
As neither party asserts that the “generated from” language is defined
in the ’057 patent, we give the claim language its ordinary and customary
meaning as would be understood by a person having ordinary skill in the art
in the context of the entire patent disclosure. See In re Translogic Tech.,
Inc., 504 F.3d 1249, 1257 (Fed. Cir. 2007). In describing the training of
pattern recognition systems, such as neural networks, for use with the
IPR2013-00419
Patent 6,772,057 B2
12
invention, the ’057 patent explains that a large number of experiments are
conducted in which different objects are placed in numerous positions and
orientations, and signals from a CCD array are returned from the objects and
measured by sensors or transducers. Ex. 1001, 36:22–39. This is the sole
example of a pattern recognition training session provided in the ’057 patent.
Although the described training session relates to objects inside a vehicle,
the ’057 patent indicates that pattern recognition systems for identifying
exterior objects are trained in a similar manner. See, e.g., id. at 40:1–9.
According to the plain language of the disputed limitation, the
algorithm must be generated from both “data of possible exterior objects”
and “patterns of received waves from the possible exterior objects.” We are
not persuaded by Toyota’s argument that the claimed “patterns of received
waves,” in contrast to “patterns from received waves,” need only be patterns
representing what received waves would look like. Neither the written
description of the ’057 patent nor the claim language, in context, supports
such parsing, particularly when the claim language further requires the
waves to be received “from” possible exterior objects.
In view of the claim language and the description in the ’057 patent of
a training session using signal patterns actually received from real objects,
we see no reasonable basis for interpreting “generated from . . . patterns of
received waves from the possible exterior objects” to encompass training of
a pattern recognition algorithm using simulated wave patterns. Therefore,
the broadest reasonable construction of the claim language at issue requires a
pattern recognition algorithm that has been generated using patterns of
waves actually received from possible exterior objects.
IPR2013-00419
Patent 6,772,057 B2
13
3. Claims 1–4, 7–10, 41, 56, 59–61, and 64
The monitoring arrangement in independent claim 1 classifies,
identifies, or locates an exterior object using a “trained pattern recognition
algorithm generated from data of possible exterior objects and patterns of
received waves from the possible exterior objects.” Independent claim 56
and dependent claim 41, which depends from claim 40, recite the same
limitation. In its Petition, Toyota contends that Lemelson’s IAC classifies,
identifies, and locates objects through the use of a neural network for pattern
recognition that has been trained on a data set. Pet. 12 (citing Ex. 1016
¶¶ 52–55). According to Toyota, “Lemelson explains that the neural
network in the IAC may be ‘trained’ using ‘known inputs.’” Id. at 13 (citing
Ex. 1002, 7:47–8:10, 8:21–23).
AVS contends that Lemelson does not disclose, either expressly or
inherently, the specific type of training of the pattern recognition algorithm
recited in claims 1, 41, and 56. PO Resp. 11. AVS admits that Lemelson
discloses a system for identifying objects exterior to a vehicle using a type of
pattern recognition algorithm (a neural network). Id. at 12. AVS contends,
however, that Lemelson’s reference to “known inputs” fails to disclose a
trained pattern recognition algorithm that is generated in the manner
required by the claims. Id. at 14–21.
As discussed above, we have construed “trained pattern recognition
algorithm generated from data of possible exterior objects and patterns of
received waves from the possible exterior objects” to require training using
patterns of waves actually received from possible exterior objects. For the
following reasons, we find that Toyota has not demonstrated by a
IPR2013-00419
Patent 6,772,057 B2
14
preponderance of the evidence that Lemelson discloses a trained pattern
recognition algorithm generated in this manner.
As AVS correctly asserts, id. at 12, Toyota’s Petition and supporting
Declaration of Dr. Papanikolopoulos cite only one sentence from Lemelson
as relating to training a neural network for pattern recognition: “Training
involves providing known inputs to the network resulting in desired output
responses.” Ex. 1002, 8:4–6 (emphasis added); see Pet. 11, 13, 16–17;
Ex. 1016 ¶ 55. AVS’s arguments focus on whether a person having ordinary
skill in the art would have understood Lemelson’s “known inputs” to refer to
patterns of waves actually received from possible exterior objects. AVS
submits that there are other ways Lemelson’s system could have generated
its pattern recognition algorithm and, therefore, Lemelson’s known inputs
are not necessarily “patterns of received waves from the possible exterior
objects,” as recited in the claims. PO Resp. 14–21.
First, AVS argues that Lemelson could have involved generating the
pattern recognition algorithm using “simulated data.” PO Resp. 14–15
(citing Ex. 2001 ¶¶ 55–63). Relying on the testimony of Dr. Koutsougeras,
AVS explains that “[s]imulated data is data that does not include any
‘patterns of received waves from the possible exterior objects.’ Rather, it is
generated by computer programs that simulate what sensors would be
reading if they were detecting an object.” Id. at 15 (citing Ex. 2001 ¶¶ 55–
63). Dr. Koutsougeras testifies that Lemelson’s “known inputs” very well
could have been simulated data, as using simulated data for training neural
networks was widely known, and using simulated data for training a neural
network on a vehicle had been described in a published thesis. Ex. 2001
¶¶ 58–59 (citing Ex. 2004, 38). AVS also argues that Lemelson’s “known
IPR2013-00419
Patent 6,772,057 B2
15
inputs” could have involved training with waves actually received from
objects other than exterior objects to be classified or identified, e.g., training
with waves from license plates rather than vehicles. PO Resp. 19–21 (citing
Ex. 2001 ¶¶ 64–71).
Toyota responds that a person of ordinary skill in the art would have
understood that training a neural network to identify exterior objects in
Lemelson would have been done with “real data,” and not with simulated or
“partial data” (i.e., actual waves received from objects other than those being
detected). Pet. Reply 8 (citing Ex. 1023 ¶¶ 10–26). Initially, we note that
Toyota introduces this argument for the first time in its Reply, along with a
supporting Reply Declaration from Dr. Papanikolopoulos (Ex. 1023). A
reply may only respond to arguments raised in the patent owner response.
37 C.F.R. § 42.23(b). Furthermore, “a reply that raises a new issue or
belatedly presents evidence will not be considered.” Office Patent Trial
Practice Guide, 77 Fed. Reg. 48,756, 48,767 (Aug. 12, 2014).
Even if we consider Toyota’s newly proffered argument and evidence
as responsive to AVS’s Patent Owner Response, we are not persuaded that
Toyota has shown by a preponderance of the evidence that Lemelson’s
“known inputs” are patterns of waves actually received from possible
exterior objects, as required by the claims. Instead, we find credible the
testimony of AVS’s expert, Dr. Koutsougeras, that one of ordinary skill in
the art would have understood that Lemelson’s “known inputs” could
include real or simulated data for training a neural network. See Ex. 2001
¶¶ 55–63; see also Ex. 1022, 132:24–138:5, 157:12–159:14, 163:18–164:7
IPR2013-00419
Patent 6,772,057 B2
16
(deposition testimony of Dr. Koutsougeras).
6
This understanding is
supported by Toyota’s counsel, who was asked at the hearing whether
“known inputs” in Lemelson refers to “just real sensor data or is . . .
understood as both” real data and simulated data. Tr. 27:25–28:1. He
answered that “one of ordinary skill in the art would have understood
[known inputs] as real sensor data, but it is not to the exclusion of simulated
data.” Id. at 28:4–6 (emphasis added). In addition, Toyota’s expert,
Dr. Papanikolopoulos, acknowledges in his deposition testimony that use of
simulated data was a possibility for training pattern recognition algorithms.
See Ex. 2002, 102:5–14 (stating that “[i]n this particular domain, you go to
simulated data, or if you don’t have access to real data, to real images” for
training pattern recognition systems to detect automobiles); see also id.
at 48:2–9 (stating that using simulated data rather than images was a
possibility).
In his Reply Declaration, Dr. Papanikolopoulos opines that one of
ordinary skill in the art at the time of the invention would have understood
“known inputs” in Lemelson to refer to real data because an ordinarily
skilled artisan “would have known that training with ‘real data’ would have
yielded the best results for” the purpose of identifying exterior objects.
Ex. 1023 ¶ 9. In essence, Dr. Papanikolopoulos’s supporting analysis
indicates that one of ordinary skill in the art would have preferred real data
over simulated or partial data for various applications. See id. ¶¶ 10–26.
This preference, however, is not sufficient to show that Lemelson’s
description of training with “known inputs” expressly discloses generating a

6
We refer to the page numbers of the deposition transcript rather than the
exhibit page numbers.
IPR2013-00419
Patent 6,772,057 B2
17
pattern recognition algorithm using waves actually received from possible
exterior objects, as required by the claims.
7
Nor does Toyota’s evidence
show by a preponderance of the evidence that Lemelson’s “known inputs”
inherently, or necessarily, refer to waves actually received from possible
objects, because the “known inputs” could refer to simulated data.
We disagree with Toyota’s argument that Dr. Koutsougeras’s
testimony should be given little weight because he has limited experience
with pattern recognition in vehicles. See Pet. Reply 11–12.
Dr. Koutsougeras testified that his “dissertation [was] on neural networks,
particularly methods for training neural networks,” and he has taught classes
and directed student theses on neural networks or for which neural networks
were a substantial component. Ex. 1022, 19:19–20:24. We are not
persuaded by Toyota’s argument that experience in training neural networks
specifically for vehicle exterior monitoring systems is necessary to support
testimony regarding an ordinarily skilled artisan’s understanding of
Lemelson’s disclosure of training using “known inputs.”
In addition, we are not persuaded by Toyota’s untimely citation in its
Reply to a portion of Lemelson discussing “adaptive operation” and “online
adjustment” of a neural network. See Pet. Reply 11 (citing Ex. 1002, 8:9–
10, Figs. 1, 2). This passage refers to alterations to the neural network after
training has occurred. Thus, Toyota’s evidence of “real-world” data being

7
We are not persuaded by Toyota’s unsupported argument, improperly
raised for the first time in its Reply, that “known inputs” in Lemelson are
analogous to a genus, and “real data” is a claimed species, so that the
“generated from” claim language is met by Lemelson’s disclosure of
“known inputs.” See Pet. Reply 7.
IPR2013-00419
Patent 6,772,057 B2
18
used to adjust operations of a neural network does not show that Lemelson
discloses training with real data.
Toyota also argues for the first time in its Reply that the “generated
from” limitation is not a limitation for purposes of patentability because it is
a process step within an apparatus claim and should not be given patentable
weight. See Pet. Reply 3–4. We find this argument to be untimely.
Toyota’s Petition does not treat the “generated from” language as if it is not
a limitation, and Toyota makes no argument in its Petition that any claim
contains product-by-process language. See Pet. 12–13.
In any event, we are not persuaded by Toyota’s argument (Pet.
Reply 4) that because the resulting pattern recognition algorithm is the same
whether the algorithm is trained using real data or simulated data, the
“generated from” language is not a limitation for purposes of patentability.
See In re Thorpe, 777 F.2d 695, 697 (Fed. Cir. 1985) (“If the product in a
product-by-process claim is the same as or obvious from a product of the
prior art, the claim is unpatentable even though the prior product was made
by a different process.”). Toyota’s proffered expert testimony and attorney
argument suggest that a pattern recognition algorithm trained with real data
differs from one that is trained using simulated data. See, e.g., Ex. 1023
¶¶ 18, 21 (testimony from Dr. Papanikolopoulos that simulated data would
have been ineffective at the time of the invention for training a neural
network to distinguish between types of objects, resulting in “garbage in-
garbage out”); see also id. ¶¶ 13–15 (testimony from Dr. Papanikolopoulos
that training with partial data would have been ineffective). Accordingly,
we do not find that the “generated from” language of claims 1, 41, and 56 is
not a limitation for purposes of patentability.
IPR2013-00419
Patent 6,772,057 B2
19
Based on the foregoing, Toyota’s Petition and supporting evidence
fail to establish that the reference in Lemelson to training with “known
inputs” discloses training with patterns of waves actually received from
possible exterior objects. In addition, Toyota’s Reply fails to provide timely
and persuasive evidence or argument that a person having ordinary skill in
the art would have understood that training a neural computer using “known
inputs” in Lemelson necessarily describes training with patterns of waves
actually received from possible exterior objects.
Accordingly, Toyota has not shown by a preponderance of the
evidence that Lemelson discloses a “trained pattern recognition algorithm
generated from data of possible exterior objects and patterns of received
waves from the possible exterior objects,” as recited in independent claims 1
and 56 and dependent claim 41. Therefore, Toyota has not demonstrated by
a preponderance of the evidence that independent claims 1 and 56 and
dependent claim 41 are anticipated by Lemelson. Toyota also has not
demonstrated by a preponderance of the evidence that dependent claims 2–4
and 7–10, which depend from claim 1, and claims 59–61 and 64, which
depend from claim 56, are anticipated by Lemelson.
4. Claims 40, 46, 48, and 49
Toyota contends that Lemelson discloses all of the limitations of
independent claim 40 and dependent claims 46, 48, and 49. Pet. 11–15, 20–
21. AVS does not challenge Toyota’s evidence regarding the unpatentability
of these claims. PO Resp. 4 n.1. We find that Toyota has shown by a
preponderance of the evidence that Lemelson anticipates claims 40, 46, 48,
and 49.
IPR2013-00419
Patent 6,772,057 B2
20
Claim 40 requires a plurality of receivers arranged apart from one
another to receive waves from different parts of the environment exterior of
a vehicle and generate signals characteristic of the received waves.
Lemelson describes multiple cameras that may be used for front, side, and
rear viewing, as well as radar and lidar receivers. Ex. 1002, 5:67–6:8, 6:37–
38. Claim 40 also requires a processor that classifies, identifies, or locates
an exterior object based on the generated signals, which is met by
Lemelson’s IAC. Id. at 5:37–47. Finally, the operation of a system in
Lemelson, such as the brakes or steering mechanism, is affected in response
to the classification, identification, or location of the exterior object, as
required by claim 40. Id. at 5:47–59.
We also are persuaded, based on Toyota’s Petition, that Lemelson
discloses the additional limitations in dependent claims 46, 48, and 49.
See Pet. 14–15. Having reviewed Toyota’s arguments and evidence, we find
that Toyota has demonstrated by a preponderance of the evidence that
Lemelson anticipates claims 40, 46, 48, and 49.
D. Obviousness over Lemelson and Borcherts
Claims 30–34, 37–39, and 62
Toyota asserts that claims 30–34, 37–39, and 62 are unpatentable
under 35 U.S.C. § 103(a) for obviousness over Lemelson and Borcherts.
Pet. 22–24. To support this assertion, Toyota relies on the analysis provided
with respect to anticipation by Lemelson, additional analysis regarding
Borcherts, and the declaration testimony of Dr. Papanikolopoulos. Id. AVS
responds, relying on the declaration testimony of Dr. Koutsougeras.
PO Resp. 29–32.
IPR2013-00419
Patent 6,772,057 B2
21
Having considered the parties’ contentions and supporting evidence,
we determine that Toyota has demonstrated by a preponderance of the
evidence that claims 30, 32–34, and 37–39 are unpatentable for obviousness
over Lemelson and Borcherts, but Toyota has not demonstrated by a
preponderance of the evidence that claims 31 and 62 are unpatentable for
obviousness over Lemelson and Borcherts.
1. Borcherts
Borcherts describes a system for steering a vehicle automatically
using images detected by a video camera. Ex. 1004, 2:56–60. Figure 1 of
Borcherts, reproduced below, illustrates a vehicle with components of an
automatic steering system:

As shown in Figure 1, “[i]mage input device 12 [(e.g., a video camera)] may
be mounted in combination with the rear view mirror assembly” and
“monitors the road in front of the vehicle.” Id. at 2:56–60. Image
processor 14 detects lanes on the road. Id. at 4:67–68. A lane centering
algorithm then provides signals to steering controller 18, which controls
IPR2013-00419
Patent 6,772,057 B2
22
steering actuator 16 to steer the vehicle in a desired direction so that it
remains within the lane. Id. at 5:1–8.
2. Analysis
Independent claim 30 is similar to independent claim 40, but instead
of a plurality of receivers, it recites “at least one receiver arranged on a rear
view mirror of the vehicle to receive waves from the environment exterior of
the vehicle.” Lemelson describes mounting a video camera at the front end
of the roof of a vehicle. Ex. 1004, 5:31–33. In view of Borcherts’s
description of a monitoring system with a video camera that may be
mounted “in combination with the rear view mirror assembly,” Toyota
contends it would have been obvious to a person having ordinary skill in the
art to attach Lemelson’s camera to the rear view mirror. Pet. 22–23 (citing
Ex. 1004, 2:52–58, Fig. 1; Ex. 1016 ¶¶ 108–111). According to Toyota, one
of ordinary skill in the art would have wanted to place the front camera
inside the vehicle so that the sensing arrays did not get dirty from outside
debris and would have recognized that a receiver on the rear view mirror
would have a better view of obstacles in front of the vehicle than, for
example, a receiver on the vehicle’s grill. Pet. 23 (citing Ex. 1016 ¶¶ 110–
111).
AVS argues that Borcherts’s “receiver mounted ‘in combination’ with
the rear view ‘assembly’ is not expressly ‘on’ the rearview mirror itself, nor
is it inherently (i.e., necessarily) ‘on’ the rearview mirror.” PO Resp. 29.
The asserted ground of unpatentability, however, is obviousness, not
anticipation. Thus, the pertinent question is whether the claimed subject
matter as whole would have been obvious to one of ordinary skill in the art,
not whether a particular reference (Borcherts) expressly or inherently
IPR2013-00419
Patent 6,772,057 B2
23
discloses a limitation. See Beckson Marine, Inc. v. NFM, Inc., 292 F.3d 718,
727 (Fed. Cir. 2002). We are persuaded that Borcherts’s description of a
camera “in combination with” a rear view mirror, along with Figure 1 of
Borcherts, which appears to illustrate a camera on a vehicle’s rear view
mirror, would have suggested modifying Lemelson’s system to place a
receiver on the rear view mirror. We also determine that Toyota has
articulated sufficient reasoning with rational underpinning for combining the
teachings of Borcherts and Lemelson. See KSR, 550 U.S. at 418; Pet. 23.
Accordingly, Toyota has shown by a preponderance of the evidence that
claim 30 is unpatentable for obviousness over Lemelson and Borcherts.
We also are persuaded, based on Toyota’s Petition, that Lemelson
discloses the additional limitations in dependent claims 32–34 and 37–39.
See Pet. 23–24 (citing Pet. 13–14 for analysis of similar limitations in claims
2–4 and 8–10, respectively). Thus, Toyota has shown by a preponderance of
the evidence that claims 32–34 and 37–39 are unpatentable for obviousness
over Lemelson and Borcherts.
Claim 31 depends from claim 30 and further recites a “trained pattern
recognition algorithm generated from data of possible exterior objects and
patterns of received waves from the possible exterior objects.” Claim 62
includes the same limitation, by virtue of its dependency from claim 56. For
the reasons discussed above with respect to anticipation of claims 1 and 56,
Toyota has not shown that Lemelson discloses this limitation. Further,
Toyota’s Petition does not rely on Borcherts to teach or suggest this
limitation. See Pet. 23–24. Accordingly, Toyota has not demonstrated by a
preponderance of the evidence that claims 31 and 62 are unpatentable for
obviousness over Lemelson and Borcherts.
IPR2013-00419
Patent 6,772,057 B2
24
E. Obviousness Grounds Based in Part on Asayama
Claims 4, 34, 43, and 59
Toyota contends that claims 4, 43, and 59 are unpatentable under
35 U.S.C. § 103(a) for obviousness over Lemelson and Asayama, and claim
34 is unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson,
Borcherts, and Asayama. Pet. 24–26. Each of these claims depends directly
from an independent claim already discussed, and further requires a
transmitter for transmitting waves into the environment, whereby a receiver
(or receivers) is arranged to receive waves transmitted by the transmitter and
reflected by exterior objects. Toyota relies on Asayama for teaching the
transmitter limitation in these claims. Id.
Claims 4 and 59 depend from claims 1 and 56, respectively. For the
reasons discussed above, Toyota has not shown that Lemelson discloses a
“trained pattern recognition algorithm generated from data of possible
exterior objects and patterns of received waves from the possible exterior
objects,” as recited in claims 1 and 56. Further, Toyota’s Petition does not
rely on Asayama to teach or suggest this limitation. See Pet. 24–25.
Accordingly, Toyota has not demonstrated by a preponderance of the
evidence that claims 4 and 59 are unpatentable for obviousness over
Lemelson and Asayama.
AVS does not challenge Toyota’s evidence regarding the
unpatentability of claim 43, which depends from claim 40. PO Resp. 4 n.1.
With respect to claim 34, which depends from claim 30, AVS argues only
that Borcherts does not disclose a receiver arranged on a rear view mirror.
PO Resp. 29. As discussed with respect to claim 30, we are persuaded that a
IPR2013-00419
Patent 6,772,057 B2
25
person of ordinary skill in the art would have modified Lemelson’s system
in view of Borcherts to place a receiver on the rear view mirror.
We find that Asayama teaches the transmitter limitation of claims 34
and 43. Asayama describes an apparatus mounted on a vehicle for detecting
distances to vehicles or other objects ahead of and behind a vehicle.
Ex. 1005, 1:8–10, 4:12–14, 6:56–58. The apparatus includes an infrared
light generator for projecting a beam of infrared light that is reflected off the
surface of an object and received by a pair of image sensors. Id. at 7:6–22.
Toyota also has articulated sufficient reasoning with rational underpinning
for combining the teachings of Asayama with Lemelson (claim 43) and with
Lemelson and Borcherts (claim 34). Pet. 24–26 (citing Ex. 1016 ¶¶ 121,
124).
For these reasons, Toyota has demonstrated by a preponderance of the
evidence that claim 43 is unpatentable for obviousness over Lemelson and
Asayama and claim 34 is unpatentable for obviousness over Lemelson,
Borcherts, and Asayama.
F. Obviousness over Yamamura and Borcherts
Claims 30, 32, and 37–39
Toyota contends that claims 30, 32, and 37–39 are unpatentable under
35 U.S.C. § 103(a) for obviousness over Yamamura and Borcherts. Pet. 59–
60. This ground of unpatentability is similar to the obviousness ground
based on Lemelson and Borcherts, with Toyota relying on Yamamura in
place of Lemelson for teaching most of the claim limitations. Id. AVS
argues only that Borcherts does not disclose a receiver arranged on a rear
view mirror, as recited in independent claim 30. PO Resp. 29.
IPR2013-00419
Patent 6,772,057 B2
26
As explained below, we determine that Toyota has demonstrated by a
preponderance of the evidence that claims 30, 32, and 37–39 are
unpatentable for obviousness over Yamamura and Borcherts.
1. Yamamura
Yamamura describes a system for monitoring the environment outside
a vehicle in order to detect preceding vehicles. Ex. 1013 ¶ 1. The system
includes a laser or radar for measuring distance and direction to exterior
objects. Id. ¶ 7. Based on those measurements, an image sensor, such as a
video camera that uses CCDs, obtains images of the objects. Id. An image
processor, which may be a computer, then applies “pattern recognition” for
identifying objects as, for example, pedestrians, buildings, or traffic signs.
Id. ¶¶ 7, 9. Additional distance measurement means and image sensors can
be installed as necessary, “such as on the front, rear, left, or right of the
vehicle.” Id. ¶ 11. Once a preceding vehicle is identified, the vehicle’s
throttle, brakes, and steering wheel can be controlled to maintain a safe
following distance. Id. ¶ 15.
2. Analysis
We agree with Toyota that Yamamura teaches most of the limitations
of independent claim 30. Pet. 59 (citing Pet. 53–55 for analysis of similar
limitations in claim 1). Claim 30 recites at least one receiver arranged to
receive waves from the environment outside the vehicle and generate a
signal characteristic of the received waves. Yamamura discloses a distance
measurement means, such as a laser or radar, as well as a video camera that
obtains images of exterior objects. Ex. 1013 ¶ 7. Claim 30 also requires a
processor that classifies, identifies, or locates an exterior object based on the
generated signal, which is met by Yamamura’s image processor that
IPR2013-00419
Patent 6,772,057 B2
27
identifies objects. Id. ¶¶ 7, 9. In addition, the operation of a system in
Yamamura, such as the throttle, brakes, or steering wheel, is affected in
response to the classification, identification, or location of the exterior
object, as required by claim 30. Id. ¶15.
As with the obviousness ground based on Lemelson and Borcherts,
AVS argues that Borcherts does not disclose a receiver arranged on a rear
view mirror, as recited in claim 30. PO Resp. 29. Similar to our
determination with respect to Lemelson and Borcherts, we are persuaded
that Borcherts’s description of a camera “in combination with” a rear view
mirror, along with Figure 1 of Borcherts, which appears to illustrate a
camera on a vehicle’s rear view mirror, would have suggested modifying
Yamamura’s system to place a receiver on the rear view mirror. Toyota also
has articulated sufficient reasoning with rational underpinning for combining
the teachings of Borcherts and Yamamura—mounting Yamamura’s camera
on the rear view mirror “would have helped keep the image sensor clean
while still obtaining a good image of the road.” Pet. 60 (citing Ex. 1016
¶¶ 175–180). Accordingly, we determine that Toyota has shown by a
preponderance of the evidence that claim 30 is unpatentable for obviousness
over Yamamura and Borcherts.
We also are persuaded, based on Toyota’s Petition, that Yamamura
discloses the additional limitations in dependent claims 32 and 37–39. See
Pet. 59–60 (citing Pet. 55–56 for analysis of similar limitations in claims 2
and 8–10, respectively). Thus, Toyota has shown by a preponderance of the
evidence that claims 32 and 37–39 are unpatentable for obviousness over
Yamamura and Borcherts.
IPR2013-00419
Patent 6,772,057 B2
28
III. CONCLUSION
Based on the evidence and arguments, Toyota has demonstrated by a
preponderance of the evidence that:
(1) claims 40, 46, 48, and 49 are unpatentable under 35 U.S.C.
§ 102(e) as anticipated by Lemelson;
(2) claims 30, 32–34, and 37–39 are unpatentable under 35 U.S.C.
§ 103(a) for obviousness over Lemelson and Borcherts;
(3) claim 43 is unpatentable under 35 U.S.C. § 103(a) for obviousness
over Lemelson and Asayama;
(4) claim 34 is unpatentable under 35 U.S.C. § 103(a) for obviousness
over Lemelson, Borcherts, and Asayama; and
(5) claims 30, 32, and 37–39 are unpatentable under 35 U.S.C.
§ 103(a) for obviousness over Yamamura and Borcherts.
Based on the evidence and arguments, Toyota has not demonstrated
by a preponderance of the evidence that:
(1) claims 1–4, 7–10, 41, 56, 59–61, and 64 are unpatentable under
35 U.S.C. § 102(e) as anticipated by Lemelson;
(2) claims 31 and 62 are unpatentable under 35 U.S.C. § 103(a) for
obviousness over Lemelson and Borcherts; and
(3) claims 4 and 59 are unpatentable under 35 U.S.C. § 103(a) for
obviousness over Lemelson and Asayama.
IV. ORDER
Accordingly, it is:
ORDERED that claims 30, 32–34, 37–40, 43, 46, 48, and 49 of U.S.
Patent No. 6,772,057 are unpatentable; and
IPR2013-00419
Patent 6,772,057 B2
29
FURTHER ORDERED that claims 1–4, 7–10, 31, 41, 56, 59–62,
and 64 of U.S. Patent No. 6,772,057 have not been shown to be
unpatentable.
This is a final decision. Parties to the proceeding seeking judicial
review of the decision must comply with the notice and service requirements
of 37 C.F.R. § 90.2.

IPR2013-00419
Patent 6,772,057 B2
30

FOR PETITIONER:
Matthew Berkowitz
Antony Pfeffer
Thomas Makin
mberkowitz@kenyon.com
apfeffer@kenyon.com
tmakin@kenyon.com

FOR PATENT OWNER:
Thomas J. Wimbiscus
Scott P. McBride
Christopher M. Scharff
McAndrews Held & Malloy
twimbiscus@mcandrews-ip.com
smcbride@mcandrews-ip.com
cscharff@mcandrews-ip.com