Toyota Motor Corporationv.American Vehicular Sciences LLCDownload PDFPatent Trial and Appeal BoardJan 12, 201510302105 (P.T.A.B. Jan. 12, 2015) Copy Citation Trials@uspto.gov Paper 59 571-272-7822 Entered: January 12, 2015 UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ TOYOTA MOTOR CORPORATION, Petitioner, v. AMERICAN VEHICULAR SCIENCES LLC, Patent Owner. ____________ Case IPR2013-00419 Patent 6,772,057 B2 ____________ Before JAMESON LEE, TREVOR M. JEFFERSON, and LYNNE E. PETTIGREW, Administrative Patent Judges. PETTIGREW, Administrative Patent Judge. FINAL WRITTEN DECISION 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73 I. BACKGROUND Petitioner, Toyota Motor Corporation (“Toyota”), filed a Petition for inter partes review of claims 1–4, 7–10, 30–34, 37–41, 43, 46, 48, 49, 56, 59–62, and 64 of U.S. Patent No. 6,772,057 B2 (Ex. 1001, “the ’057 patent”). Paper 3 (“Pet.”). Patent Owner, American Vehicular Sciences LLC (“AVS”), filed a Preliminary Response. Paper 17 (“Prelim. Resp.”). IPR2013-00419 Patent 6,772,057 B2 2 On January 13, 2014, pursuant to 35 U.S.C. § 314, we instituted an inter partes review for all challenged claims on certain grounds of unpatentability asserted in the Petition. Paper 19 (“Dec. on Inst.”). Subsequent to institution, AVS filed a Patent Owner Response (Paper 33), and later filed a Revised Patent Owner Response (Paper 45, “PO Resp.”). 1 Toyota filed a Reply to Patent Owner’s Response (Paper 40, “Pet. Reply”). A consolidated oral hearing for this proceeding and Toyota Motor Corp. v. American Vehicular Sciences, LLC, IPR2013-00424, involving the same parties and similar issues, was held on August 18, 2014. A transcript of the consolidated hearing is included in the record. Paper 58 (“Tr.”). We have jurisdiction under 35 U.S.C. § 6(c). This Final Written Decision is issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73. As explained below, Toyota has shown by a preponderance of the evidence that claims 30, 32–34, 37–40, 43, 46, 48, and 49 of the ’057 patent are unpatentable, but Toyota has not shown by a preponderance of the evidence that claims 1–4, 7–10, 31, 41, 56, 59–62, and 64 of the ’057 patent are unpatentable. A. Related Proceedings Toyota and AVS indicate that the ’057 patent has been asserted by AVS in the following district court cases: American Vehicular Sciences LLC v. Toyota Motor Corp., No. 6:12-cv-00410 (E.D. Tex.) (filed June 25, 2012); American Vehicular Sciences LLC v. BMW Group, No. 6:12-cv- 1 We authorized AVS to file a Revised Patent Owner Response to make certain non-substantive corrections to the Patent Owner Response. See Papers 39, 44 (Orders on Conduct of the Proceedings). IPR2013-00419 Patent 6,772,057 B2 3 00415 (E.D. Tex.) (filed June 25, 2012); American Vehicular Sciences LLC v. Subaru of Am. Inc., No. 6:12-cv-00230 (E.D. Tex.) (filed Mar. 8, 2013); and American Vehicular Sciences LLC v. Mercedes-Benz U.S. Int’l, Inc., No. 6:13-cv-00309 (E.D. Tex.) (filed Apr. 3, 2013). Pet. 1; Paper 27, 2–3. B. The’057 Patent The ’057 patent, titled “Vehicular Monitoring Systems Using Image Processing,” generally relates to a vehicle monitoring arrangement for monitoring an environment exterior of a vehicle. Ex. 1001, Abstract. One embodiment of such an arrangement described in the ’057 patent includes a transmitter that transmits electromagnetic waves into the environment exterior of a vehicle and one or more receivers that receive reflections of the transmitted waves from exterior objects, such as approaching vehicles. Id. at 14:8–12, 14:32–37, 38:7–13, Fig. 7. In a preferred implementation, the transmitter is an infrared transmitter, and the receivers are CCD (charge coupled device) transducers that receive the reflected infrared waves. Id. at 38:10–12, 39:25–28. One or more receivers may be arranged on a rear view mirror of the vehicle. Id. at 14:58–60, 38:22–25. The system also may include radar or pulsed laser radar (lidar) for measuring distance between the vehicle and exterior objects. Id. at 14:38–40, 39:1–6. The waves received by the receivers contain information about exterior objects in the environment, and the receivers generate signals characteristic of the received waves. Id. at 14:12–14, 39:44–49. A trained pattern recognition means, such as a neural computer or neural network, processes the signals to provide a classification, identification, or location of an exterior object. Id. at 14:17–25, 39:49–54. Training of a neural network to provide classification, identification, or location of objects is IPR2013-00419 Patent 6,772,057 B2 4 accomplished by conducting a large number of experiments in which the system is taught to differentiate among received signals corresponding to different objects. Id. at 36:22–39 (describing a neural network training session in connection with an embodiment that monitors an interior of a vehicle, particularly the passenger seat). The classification, identification, or location of an exterior object may be used to affect operation of other systems in the vehicle, e.g., to show an image or icon on a display viewable by a driver or to deploy an airbag. Id. at 14:21–31, 39:54–62. C. Illustrative Claims Of the challenged claims, claims 1, 30, 40, and 56 are independent. Claims 1, 30, and 40 are illustrative: 1. A monitoring arrangement for monitoring an environment exterior of a vehicle, comprising: at least one receiver arranged to receive waves from the environment exterior of the vehicle which contain information on any objects in the environment and generate a signal characteristic of the received waves; and a processor coupled to said at least one receiver and comprising trained pattern recognition means for processing the signal to provide a classification, identification or location of the exterior object, said trained pattern recognition means being structured and arranged to apply a trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects to provide the classification, identification or location of the exterior object; whereby a system in the vehicle is coupled to said processor such that the operation of the system is affected in response to the classification, identification or location of the exterior object. IPR2013-00419 Patent 6,772,057 B2 5 30. A vehicle including a monitoring arrangement for monitoring an environment exterior of the vehicle, the monitoring arrangement comprising: at least one receiver arranged on a rear view mirror of the vehicle to receive waves from the environment exterior of the vehicle which contain information on any objects in the environment and generate a signal characteristic of the received waves; and a processor coupled to said at least one receiver and arranged to classify or identify the exterior object based on the signal and thereby provide the classification or identification of the exterior object; whereby a system in the vehicle is coupled to said processor such that the operation of the system is affected in response to the classification or identification of the exterior object. 40. A monitoring arrangement for monitoring an environment exterior of a vehicle, comprising: a plurality of receivers arranged apart from one another and to receive waves from different parts of the environment exterior of the vehicle which contain information on any objects in the environment and generate a signal characteristic of the received waves; and a processor coupled to said receivers and arranged to classify, identify or locate the exterior object based on the signals generated by said receivers and thereby provide the classification[,] identification or location of the exterior object, whereby a system in the vehicle is coupled to said processor such that the operation of the system is affected in response to the classification, identification or location of the exterior object. Ex. 1001, 54:13–32, 55:58–56:6, 56:37–52 (emphases added). IPR2013-00419 Patent 6,772,057 B2 6 D. Pending Grounds of Unpatentability We instituted inter partes review based on the following grounds of unpatentability asserted in the Petition: Reference(s) Basis Claim(s) Lemelson 2 35 U.S.C. § 102(e) 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61, and 64 Lemelson and Borcherts 3 35 U.S.C. § 103(a) 30–34, 37–39, and 62 Lemelson and Asayama 4 35 U.S.C. § 103(a) 4, 43, and 59 Lemelson, Borcherts, and Asayama 35 U.S.C. § 103(a) 34 Yamamura 5 and Borcherts 35 U.S.C. § 103(a) 30, 32, and 37–39 Dec. on Inst. 38. II. ANALYSIS A. Claim Construction In the Decision on Institution, we construed several claim terms of the ’057 patent, as set forth in the following table: 2 U.S. Patent No. 6,553,130, issued Apr. 22, 2003 (Ex. 1002). 3 U.S. Patent No. 5,245,422, issued Sept. 14, 1993 (Ex. 1004). 4 U.S. Patent No. 5,214,408, issued May 25, 1993 (Ex. 1005). 5 Japanese Unexamined Patent Application Publication No. H06-124340, published May 6, 1994 (Ex. 1012). Citations to Yamamura refer to its English translation (Ex. 1013). IPR2013-00419 Patent 6,772,057 B2 7 Claim Term Board’s Construction “trained pattern recognition algorithm” “an algorithm that processes a signal that is generated by an object, or is modified by interacting with an object, in order to determine to which one of a set of classes the object belongs, the algorithm having been taught, through a variety of examples, various patterns of received signals generated or modified by objects” “trained pattern recognition means” “a neural computer or neural network trained for pattern recognition, and equivalents thereof” “identify” “determine that the object belongs to a particular set or class” “exterior object” “a material or physical thing outside the vehicle, not a part of the roadway on which the vehicle travels” “rear view mirror” “a mirror that faces to the rear” “transmitter” “device that transmits any type of electromagnetic waves, including visible light” Dec. on Inst. 8–15. AVS does not contest these constructions for purposes of this proceeding, PO Resp. 9–10, and Toyota does not dispute these constructions in its Reply. We maintain these constructions for this Final Written Decision. B. Principles of Law To prevail in challenging AVS’s claims, Toyota must demonstrate by a preponderance of the evidence that the claims are unpatentable. 35 U.S.C. § 316(e); 37 C.F.R. § 42.1(d). A claim is anticipated if a single prior art reference either expressly or inherently discloses every limitation of the claim. Orion IP, LLC v. Hyundai Motor Am., 605 F.3d 967, 975 (Fed. Cir. 2010). A claim is unpatentable under 35 U.S.C. § 103(a) if the differences IPR2013-00419 Patent 6,772,057 B2 8 between the claimed subject matter and the prior art are such that the subject matter, as a whole, would have been obvious at the time of the invention to a person having ordinary skill in the art. KSR Int’l Co. v. Teleflex, Inc., 550 U.S. 398, 406 (2007). C. Anticipation by Lemelson Claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61, and 64 Toyota asserts that claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61, and 64 are unpatentable under 35 U.S.C. § 102(e) as anticipated by Lemelson. Pet. 10–24. In support of this assertion, Toyota provides detailed analysis and claim charts explaining how Lemelson discloses each claim limitation. Id. Toyota also relies on the testimony of Dr. Nikolaos Papanikolopoulos, as set forth in his Declaration (Ex. 1016) and Reply Declaration (Ex. 1023). AVS responds that Lemelson does not disclose each limitation of the challenged claims. PO Resp. 11–25. For support, AVS relies on the testimony of Dr. Cris Koutsougeras, as set forth in his Declaration (Ex. 2001). Having considered the parties’ contentions and supporting evidence, we determine that Toyota has demonstrated by a preponderance of the evidence that claims 40, 46, 48, and 49 are anticipated by Lemelson, but Toyota has not demonstrated by a preponderance of the evidence that claims 1–4, 7–10, 41, 56, 59–61, and 64 are anticipated by Lemelson. 1. Lemelson Lemelson discloses a computerized system in a motor vehicle that identifies possible obstacles on a roadway and either warns the driver or controls the operation of vehicle systems, such as the brakes or steering IPR2013-00419 Patent 6,772,057 B2 9 mechanism, to avoid or lessen the effect of a collision. Ex. 1002, Abstract, 5:15–29, 8:38–39. The system includes at least one video camera, mounted at the front end of the vehicle, such as the front end of the roof, bumper, or hood. Id. at 5:31–34. The system may include multiple cameras for front, side, and rear viewing, and for stereo imaging capabilities. Id. at 6:27–42. The video camera also may be implemented with other technologies, including infrared imaging methods. Id. at 6:34–37. In addition, the system may use radar or lidar for range detection. Id. at 5:67–6:4. “[V]ideo scanning and radar or lidar scanning may be jointly employed to identify and indicate distances between the controlled vehicle and objects ahead of, to the side(s) of, and to the rear of the controlled vehicle.” Id. at 6:5–8. The analog signal output from the video camera(s) is digitized in an analog-to-digital convertor and passed to an image analyzing computer (IAC), which is provided, implemented and programmed using neural networks and artificial intelligence as well as fuzzy logic algorithms to (a) identify objects on the road ahead such as other vehicles, pedestrians, barriers and dividers, turns in the road, signs and symbols, etc., and generate identification codes, and (b) detect distances from such objects by their size (and shape) and provide codes indicating same for use by a decision computer[] 23, which generates coded control signals which are applied through the computer 11 or are directly passed to various warning and vehicle operating devices such as a braking computer or drive[] 35, which operates a brake servo 33, a steering computer or drive(s) 39 and 40 which operate steering servos 36; . . . a headlight controller 41 for flashing the head lights, a warning light control 42 for flashing external and/or internal warning lights; a horn control 43, etc. Id. at 5:39–59. The IAC also may display symbols representing hazard objects. Id. at 6:52–55, 9:60–62. IPR2013-00419 Patent 6,772,057 B2 10 Lemelson discloses further details regarding a neural network embodiment of the IAC for identifying objects: Neural networks used in the vehicle . . . warning system are trained to recognize roadway hazards which the vehicle is approaching including automobiles, trucks, and pedestrians. Training involves providing known inputs to the network resulting in desired output responses. The weights are automatically adjusted based on error signal measurements until the desired outputs are generated. Various learning algorithms may be applied. Adaptive operation is also possible with on- line adjustment of network weights to meet imaging requirements. The neural network embodiment of the image analysis computer 19 provides a highly parallel image processing structure with rapid, real-time image recognition necessary for the Motor Vehicle Warning and Control System. Id. at 8:1–14. 2. “Generated from” Limitation A central dispute between the parties is whether Lemelson discloses a “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects.” This limitation appears in independent claims 1 and 56 and in claim 41, which depends from claim 40. Although we construed “trained pattern recognition algorithm” in our Decision on Institution, we did not provide an express construction for the “generated from” language following that term in the claims. For this Final Written Decision, we construe the “generated from” limitation according to its broadest reasonable interpretation in light of the specification of the ’057 patent. See 37 C.F.R. § 42.100(b). AVS contends that the “generated from” limitation in claims 1, 41, and 56 requires a particular type of training to generate the “trained pattern IPR2013-00419 Patent 6,772,057 B2 11 recognition algorithm” recited in the claims. PO Resp. 6. Relying on the Declaration of Dr. Koutsougeras, AVS argues that the ’057 patent discloses and claims a specific method for training the pattern recognition means using “data of possible exterior objects and patterns of received waves from the possible exterior objects.” Id. (citing Ex. 2001 ¶¶ 19, 20, 53). For example, if a vehicle uses a radar receiver, AVS contends that the claims require training using “real radar waves” received from actual examples of possible exterior objects placed in front of the radar system, along with data indicating the identity or classification of the objects. Id. at 6–7 (citing Ex. 2001 ¶¶ 19–20). AVS contrasts the use of real waves to train a pattern recognition system, as recited in the claims, with other methods of training, such as using simulated data (e.g., a computer simulation of radar waves). Id. at 7 (citing Ex. 2001 ¶¶ 46, 55–63). Toyota argues that the “generated from” language of claims 1, 41, and 56 does not require training with real data. Pet. Reply 2, 4. Because the claims refer to “data of possible exterior objects” and “patterns of received waves,” rather than “data from” and “patterns from,” Toyota contends that the claim language encompasses training using simulated data and patterns that represent possible objects and received waves, respectively. Id. at 2, 4–5. As neither party asserts that the “generated from” language is defined in the ’057 patent, we give the claim language its ordinary and customary meaning as would be understood by a person having ordinary skill in the art in the context of the entire patent disclosure. See In re Translogic Tech., Inc., 504 F.3d 1249, 1257 (Fed. Cir. 2007). In describing the training of pattern recognition systems, such as neural networks, for use with the IPR2013-00419 Patent 6,772,057 B2 12 invention, the ’057 patent explains that a large number of experiments are conducted in which different objects are placed in numerous positions and orientations, and signals from a CCD array are returned from the objects and measured by sensors or transducers. Ex. 1001, 36:22–39. This is the sole example of a pattern recognition training session provided in the ’057 patent. Although the described training session relates to objects inside a vehicle, the ’057 patent indicates that pattern recognition systems for identifying exterior objects are trained in a similar manner. See, e.g., id. at 40:1–9. According to the plain language of the disputed limitation, the algorithm must be generated from both “data of possible exterior objects” and “patterns of received waves from the possible exterior objects.” We are not persuaded by Toyota’s argument that the claimed “patterns of received waves,” in contrast to “patterns from received waves,” need only be patterns representing what received waves would look like. Neither the written description of the ’057 patent nor the claim language, in context, supports such parsing, particularly when the claim language further requires the waves to be received “from” possible exterior objects. In view of the claim language and the description in the ’057 patent of a training session using signal patterns actually received from real objects, we see no reasonable basis for interpreting “generated from . . . patterns of received waves from the possible exterior objects” to encompass training of a pattern recognition algorithm using simulated wave patterns. Therefore, the broadest reasonable construction of the claim language at issue requires a pattern recognition algorithm that has been generated using patterns of waves actually received from possible exterior objects. IPR2013-00419 Patent 6,772,057 B2 13 3. Claims 1–4, 7–10, 41, 56, 59–61, and 64 The monitoring arrangement in independent claim 1 classifies, identifies, or locates an exterior object using a “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects.” Independent claim 56 and dependent claim 41, which depends from claim 40, recite the same limitation. In its Petition, Toyota contends that Lemelson’s IAC classifies, identifies, and locates objects through the use of a neural network for pattern recognition that has been trained on a data set. Pet. 12 (citing Ex. 1016 ¶¶ 52–55). According to Toyota, “Lemelson explains that the neural network in the IAC may be ‘trained’ using ‘known inputs.’” Id. at 13 (citing Ex. 1002, 7:47–8:10, 8:21–23). AVS contends that Lemelson does not disclose, either expressly or inherently, the specific type of training of the pattern recognition algorithm recited in claims 1, 41, and 56. PO Resp. 11. AVS admits that Lemelson discloses a system for identifying objects exterior to a vehicle using a type of pattern recognition algorithm (a neural network). Id. at 12. AVS contends, however, that Lemelson’s reference to “known inputs” fails to disclose a trained pattern recognition algorithm that is generated in the manner required by the claims. Id. at 14–21. As discussed above, we have construed “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects” to require training using patterns of waves actually received from possible exterior objects. For the following reasons, we find that Toyota has not demonstrated by a IPR2013-00419 Patent 6,772,057 B2 14 preponderance of the evidence that Lemelson discloses a trained pattern recognition algorithm generated in this manner. As AVS correctly asserts, id. at 12, Toyota’s Petition and supporting Declaration of Dr. Papanikolopoulos cite only one sentence from Lemelson as relating to training a neural network for pattern recognition: “Training involves providing known inputs to the network resulting in desired output responses.” Ex. 1002, 8:4–6 (emphasis added); see Pet. 11, 13, 16–17; Ex. 1016 ¶ 55. AVS’s arguments focus on whether a person having ordinary skill in the art would have understood Lemelson’s “known inputs” to refer to patterns of waves actually received from possible exterior objects. AVS submits that there are other ways Lemelson’s system could have generated its pattern recognition algorithm and, therefore, Lemelson’s known inputs are not necessarily “patterns of received waves from the possible exterior objects,” as recited in the claims. PO Resp. 14–21. First, AVS argues that Lemelson could have involved generating the pattern recognition algorithm using “simulated data.” PO Resp. 14–15 (citing Ex. 2001 ¶¶ 55–63). Relying on the testimony of Dr. Koutsougeras, AVS explains that “[s]imulated data is data that does not include any ‘patterns of received waves from the possible exterior objects.’ Rather, it is generated by computer programs that simulate what sensors would be reading if they were detecting an object.” Id. at 15 (citing Ex. 2001 ¶¶ 55– 63). Dr. Koutsougeras testifies that Lemelson’s “known inputs” very well could have been simulated data, as using simulated data for training neural networks was widely known, and using simulated data for training a neural network on a vehicle had been described in a published thesis. Ex. 2001 ¶¶ 58–59 (citing Ex. 2004, 38). AVS also argues that Lemelson’s “known IPR2013-00419 Patent 6,772,057 B2 15 inputs” could have involved training with waves actually received from objects other than exterior objects to be classified or identified, e.g., training with waves from license plates rather than vehicles. PO Resp. 19–21 (citing Ex. 2001 ¶¶ 64–71). Toyota responds that a person of ordinary skill in the art would have understood that training a neural network to identify exterior objects in Lemelson would have been done with “real data,” and not with simulated or “partial data” (i.e., actual waves received from objects other than those being detected). Pet. Reply 8 (citing Ex. 1023 ¶¶ 10–26). Initially, we note that Toyota introduces this argument for the first time in its Reply, along with a supporting Reply Declaration from Dr. Papanikolopoulos (Ex. 1023). A reply may only respond to arguments raised in the patent owner response. 37 C.F.R. § 42.23(b). Furthermore, “a reply that raises a new issue or belatedly presents evidence will not be considered.” Office Patent Trial Practice Guide, 77 Fed. Reg. 48,756, 48,767 (Aug. 12, 2014). Even if we consider Toyota’s newly proffered argument and evidence as responsive to AVS’s Patent Owner Response, we are not persuaded that Toyota has shown by a preponderance of the evidence that Lemelson’s “known inputs” are patterns of waves actually received from possible exterior objects, as required by the claims. Instead, we find credible the testimony of AVS’s expert, Dr. Koutsougeras, that one of ordinary skill in the art would have understood that Lemelson’s “known inputs” could include real or simulated data for training a neural network. See Ex. 2001 ¶¶ 55–63; see also Ex. 1022, 132:24–138:5, 157:12–159:14, 163:18–164:7 IPR2013-00419 Patent 6,772,057 B2 16 (deposition testimony of Dr. Koutsougeras). 6 This understanding is supported by Toyota’s counsel, who was asked at the hearing whether “known inputs” in Lemelson refers to “just real sensor data or is . . . understood as both” real data and simulated data. Tr. 27:25–28:1. He answered that “one of ordinary skill in the art would have understood [known inputs] as real sensor data, but it is not to the exclusion of simulated data.” Id. at 28:4–6 (emphasis added). In addition, Toyota’s expert, Dr. Papanikolopoulos, acknowledges in his deposition testimony that use of simulated data was a possibility for training pattern recognition algorithms. See Ex. 2002, 102:5–14 (stating that “[i]n this particular domain, you go to simulated data, or if you don’t have access to real data, to real images” for training pattern recognition systems to detect automobiles); see also id. at 48:2–9 (stating that using simulated data rather than images was a possibility). In his Reply Declaration, Dr. Papanikolopoulos opines that one of ordinary skill in the art at the time of the invention would have understood “known inputs” in Lemelson to refer to real data because an ordinarily skilled artisan “would have known that training with ‘real data’ would have yielded the best results for” the purpose of identifying exterior objects. Ex. 1023 ¶ 9. In essence, Dr. Papanikolopoulos’s supporting analysis indicates that one of ordinary skill in the art would have preferred real data over simulated or partial data for various applications. See id. ¶¶ 10–26. This preference, however, is not sufficient to show that Lemelson’s description of training with “known inputs” expressly discloses generating a 6 We refer to the page numbers of the deposition transcript rather than the exhibit page numbers. IPR2013-00419 Patent 6,772,057 B2 17 pattern recognition algorithm using waves actually received from possible exterior objects, as required by the claims. 7 Nor does Toyota’s evidence show by a preponderance of the evidence that Lemelson’s “known inputs” inherently, or necessarily, refer to waves actually received from possible objects, because the “known inputs” could refer to simulated data. We disagree with Toyota’s argument that Dr. Koutsougeras’s testimony should be given little weight because he has limited experience with pattern recognition in vehicles. See Pet. Reply 11–12. Dr. Koutsougeras testified that his “dissertation [was] on neural networks, particularly methods for training neural networks,” and he has taught classes and directed student theses on neural networks or for which neural networks were a substantial component. Ex. 1022, 19:19–20:24. We are not persuaded by Toyota’s argument that experience in training neural networks specifically for vehicle exterior monitoring systems is necessary to support testimony regarding an ordinarily skilled artisan’s understanding of Lemelson’s disclosure of training using “known inputs.” In addition, we are not persuaded by Toyota’s untimely citation in its Reply to a portion of Lemelson discussing “adaptive operation” and “online adjustment” of a neural network. See Pet. Reply 11 (citing Ex. 1002, 8:9– 10, Figs. 1, 2). This passage refers to alterations to the neural network after training has occurred. Thus, Toyota’s evidence of “real-world” data being 7 We are not persuaded by Toyota’s unsupported argument, improperly raised for the first time in its Reply, that “known inputs” in Lemelson are analogous to a genus, and “real data” is a claimed species, so that the “generated from” claim language is met by Lemelson’s disclosure of “known inputs.” See Pet. Reply 7. IPR2013-00419 Patent 6,772,057 B2 18 used to adjust operations of a neural network does not show that Lemelson discloses training with real data. Toyota also argues for the first time in its Reply that the “generated from” limitation is not a limitation for purposes of patentability because it is a process step within an apparatus claim and should not be given patentable weight. See Pet. Reply 3–4. We find this argument to be untimely. Toyota’s Petition does not treat the “generated from” language as if it is not a limitation, and Toyota makes no argument in its Petition that any claim contains product-by-process language. See Pet. 12–13. In any event, we are not persuaded by Toyota’s argument (Pet. Reply 4) that because the resulting pattern recognition algorithm is the same whether the algorithm is trained using real data or simulated data, the “generated from” language is not a limitation for purposes of patentability. See In re Thorpe, 777 F.2d 695, 697 (Fed. Cir. 1985) (“If the product in a product-by-process claim is the same as or obvious from a product of the prior art, the claim is unpatentable even though the prior product was made by a different process.”). Toyota’s proffered expert testimony and attorney argument suggest that a pattern recognition algorithm trained with real data differs from one that is trained using simulated data. See, e.g., Ex. 1023 ¶¶ 18, 21 (testimony from Dr. Papanikolopoulos that simulated data would have been ineffective at the time of the invention for training a neural network to distinguish between types of objects, resulting in “garbage in- garbage out”); see also id. ¶¶ 13–15 (testimony from Dr. Papanikolopoulos that training with partial data would have been ineffective). Accordingly, we do not find that the “generated from” language of claims 1, 41, and 56 is not a limitation for purposes of patentability. IPR2013-00419 Patent 6,772,057 B2 19 Based on the foregoing, Toyota’s Petition and supporting evidence fail to establish that the reference in Lemelson to training with “known inputs” discloses training with patterns of waves actually received from possible exterior objects. In addition, Toyota’s Reply fails to provide timely and persuasive evidence or argument that a person having ordinary skill in the art would have understood that training a neural computer using “known inputs” in Lemelson necessarily describes training with patterns of waves actually received from possible exterior objects. Accordingly, Toyota has not shown by a preponderance of the evidence that Lemelson discloses a “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects,” as recited in independent claims 1 and 56 and dependent claim 41. Therefore, Toyota has not demonstrated by a preponderance of the evidence that independent claims 1 and 56 and dependent claim 41 are anticipated by Lemelson. Toyota also has not demonstrated by a preponderance of the evidence that dependent claims 2–4 and 7–10, which depend from claim 1, and claims 59–61 and 64, which depend from claim 56, are anticipated by Lemelson. 4. Claims 40, 46, 48, and 49 Toyota contends that Lemelson discloses all of the limitations of independent claim 40 and dependent claims 46, 48, and 49. Pet. 11–15, 20– 21. AVS does not challenge Toyota’s evidence regarding the unpatentability of these claims. PO Resp. 4 n.1. We find that Toyota has shown by a preponderance of the evidence that Lemelson anticipates claims 40, 46, 48, and 49. IPR2013-00419 Patent 6,772,057 B2 20 Claim 40 requires a plurality of receivers arranged apart from one another to receive waves from different parts of the environment exterior of a vehicle and generate signals characteristic of the received waves. Lemelson describes multiple cameras that may be used for front, side, and rear viewing, as well as radar and lidar receivers. Ex. 1002, 5:67–6:8, 6:37– 38. Claim 40 also requires a processor that classifies, identifies, or locates an exterior object based on the generated signals, which is met by Lemelson’s IAC. Id. at 5:37–47. Finally, the operation of a system in Lemelson, such as the brakes or steering mechanism, is affected in response to the classification, identification, or location of the exterior object, as required by claim 40. Id. at 5:47–59. We also are persuaded, based on Toyota’s Petition, that Lemelson discloses the additional limitations in dependent claims 46, 48, and 49. See Pet. 14–15. Having reviewed Toyota’s arguments and evidence, we find that Toyota has demonstrated by a preponderance of the evidence that Lemelson anticipates claims 40, 46, 48, and 49. D. Obviousness over Lemelson and Borcherts Claims 30–34, 37–39, and 62 Toyota asserts that claims 30–34, 37–39, and 62 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Borcherts. Pet. 22–24. To support this assertion, Toyota relies on the analysis provided with respect to anticipation by Lemelson, additional analysis regarding Borcherts, and the declaration testimony of Dr. Papanikolopoulos. Id. AVS responds, relying on the declaration testimony of Dr. Koutsougeras. PO Resp. 29–32. IPR2013-00419 Patent 6,772,057 B2 21 Having considered the parties’ contentions and supporting evidence, we determine that Toyota has demonstrated by a preponderance of the evidence that claims 30, 32–34, and 37–39 are unpatentable for obviousness over Lemelson and Borcherts, but Toyota has not demonstrated by a preponderance of the evidence that claims 31 and 62 are unpatentable for obviousness over Lemelson and Borcherts. 1. Borcherts Borcherts describes a system for steering a vehicle automatically using images detected by a video camera. Ex. 1004, 2:56–60. Figure 1 of Borcherts, reproduced below, illustrates a vehicle with components of an automatic steering system: As shown in Figure 1, “[i]mage input device 12 [(e.g., a video camera)] may be mounted in combination with the rear view mirror assembly” and “monitors the road in front of the vehicle.” Id. at 2:56–60. Image processor 14 detects lanes on the road. Id. at 4:67–68. A lane centering algorithm then provides signals to steering controller 18, which controls IPR2013-00419 Patent 6,772,057 B2 22 steering actuator 16 to steer the vehicle in a desired direction so that it remains within the lane. Id. at 5:1–8. 2. Analysis Independent claim 30 is similar to independent claim 40, but instead of a plurality of receivers, it recites “at least one receiver arranged on a rear view mirror of the vehicle to receive waves from the environment exterior of the vehicle.” Lemelson describes mounting a video camera at the front end of the roof of a vehicle. Ex. 1004, 5:31–33. In view of Borcherts’s description of a monitoring system with a video camera that may be mounted “in combination with the rear view mirror assembly,” Toyota contends it would have been obvious to a person having ordinary skill in the art to attach Lemelson’s camera to the rear view mirror. Pet. 22–23 (citing Ex. 1004, 2:52–58, Fig. 1; Ex. 1016 ¶¶ 108–111). According to Toyota, one of ordinary skill in the art would have wanted to place the front camera inside the vehicle so that the sensing arrays did not get dirty from outside debris and would have recognized that a receiver on the rear view mirror would have a better view of obstacles in front of the vehicle than, for example, a receiver on the vehicle’s grill. Pet. 23 (citing Ex. 1016 ¶¶ 110– 111). AVS argues that Borcherts’s “receiver mounted ‘in combination’ with the rear view ‘assembly’ is not expressly ‘on’ the rearview mirror itself, nor is it inherently (i.e., necessarily) ‘on’ the rearview mirror.” PO Resp. 29. The asserted ground of unpatentability, however, is obviousness, not anticipation. Thus, the pertinent question is whether the claimed subject matter as whole would have been obvious to one of ordinary skill in the art, not whether a particular reference (Borcherts) expressly or inherently IPR2013-00419 Patent 6,772,057 B2 23 discloses a limitation. See Beckson Marine, Inc. v. NFM, Inc., 292 F.3d 718, 727 (Fed. Cir. 2002). We are persuaded that Borcherts’s description of a camera “in combination with” a rear view mirror, along with Figure 1 of Borcherts, which appears to illustrate a camera on a vehicle’s rear view mirror, would have suggested modifying Lemelson’s system to place a receiver on the rear view mirror. We also determine that Toyota has articulated sufficient reasoning with rational underpinning for combining the teachings of Borcherts and Lemelson. See KSR, 550 U.S. at 418; Pet. 23. Accordingly, Toyota has shown by a preponderance of the evidence that claim 30 is unpatentable for obviousness over Lemelson and Borcherts. We also are persuaded, based on Toyota’s Petition, that Lemelson discloses the additional limitations in dependent claims 32–34 and 37–39. See Pet. 23–24 (citing Pet. 13–14 for analysis of similar limitations in claims 2–4 and 8–10, respectively). Thus, Toyota has shown by a preponderance of the evidence that claims 32–34 and 37–39 are unpatentable for obviousness over Lemelson and Borcherts. Claim 31 depends from claim 30 and further recites a “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects.” Claim 62 includes the same limitation, by virtue of its dependency from claim 56. For the reasons discussed above with respect to anticipation of claims 1 and 56, Toyota has not shown that Lemelson discloses this limitation. Further, Toyota’s Petition does not rely on Borcherts to teach or suggest this limitation. See Pet. 23–24. Accordingly, Toyota has not demonstrated by a preponderance of the evidence that claims 31 and 62 are unpatentable for obviousness over Lemelson and Borcherts. IPR2013-00419 Patent 6,772,057 B2 24 E. Obviousness Grounds Based in Part on Asayama Claims 4, 34, 43, and 59 Toyota contends that claims 4, 43, and 59 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Asayama, and claim 34 is unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson, Borcherts, and Asayama. Pet. 24–26. Each of these claims depends directly from an independent claim already discussed, and further requires a transmitter for transmitting waves into the environment, whereby a receiver (or receivers) is arranged to receive waves transmitted by the transmitter and reflected by exterior objects. Toyota relies on Asayama for teaching the transmitter limitation in these claims. Id. Claims 4 and 59 depend from claims 1 and 56, respectively. For the reasons discussed above, Toyota has not shown that Lemelson discloses a “trained pattern recognition algorithm generated from data of possible exterior objects and patterns of received waves from the possible exterior objects,” as recited in claims 1 and 56. Further, Toyota’s Petition does not rely on Asayama to teach or suggest this limitation. See Pet. 24–25. Accordingly, Toyota has not demonstrated by a preponderance of the evidence that claims 4 and 59 are unpatentable for obviousness over Lemelson and Asayama. AVS does not challenge Toyota’s evidence regarding the unpatentability of claim 43, which depends from claim 40. PO Resp. 4 n.1. With respect to claim 34, which depends from claim 30, AVS argues only that Borcherts does not disclose a receiver arranged on a rear view mirror. PO Resp. 29. As discussed with respect to claim 30, we are persuaded that a IPR2013-00419 Patent 6,772,057 B2 25 person of ordinary skill in the art would have modified Lemelson’s system in view of Borcherts to place a receiver on the rear view mirror. We find that Asayama teaches the transmitter limitation of claims 34 and 43. Asayama describes an apparatus mounted on a vehicle for detecting distances to vehicles or other objects ahead of and behind a vehicle. Ex. 1005, 1:8–10, 4:12–14, 6:56–58. The apparatus includes an infrared light generator for projecting a beam of infrared light that is reflected off the surface of an object and received by a pair of image sensors. Id. at 7:6–22. Toyota also has articulated sufficient reasoning with rational underpinning for combining the teachings of Asayama with Lemelson (claim 43) and with Lemelson and Borcherts (claim 34). Pet. 24–26 (citing Ex. 1016 ¶¶ 121, 124). For these reasons, Toyota has demonstrated by a preponderance of the evidence that claim 43 is unpatentable for obviousness over Lemelson and Asayama and claim 34 is unpatentable for obviousness over Lemelson, Borcherts, and Asayama. F. Obviousness over Yamamura and Borcherts Claims 30, 32, and 37–39 Toyota contends that claims 30, 32, and 37–39 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Yamamura and Borcherts. Pet. 59– 60. This ground of unpatentability is similar to the obviousness ground based on Lemelson and Borcherts, with Toyota relying on Yamamura in place of Lemelson for teaching most of the claim limitations. Id. AVS argues only that Borcherts does not disclose a receiver arranged on a rear view mirror, as recited in independent claim 30. PO Resp. 29. IPR2013-00419 Patent 6,772,057 B2 26 As explained below, we determine that Toyota has demonstrated by a preponderance of the evidence that claims 30, 32, and 37–39 are unpatentable for obviousness over Yamamura and Borcherts. 1. Yamamura Yamamura describes a system for monitoring the environment outside a vehicle in order to detect preceding vehicles. Ex. 1013 ¶ 1. The system includes a laser or radar for measuring distance and direction to exterior objects. Id. ¶ 7. Based on those measurements, an image sensor, such as a video camera that uses CCDs, obtains images of the objects. Id. An image processor, which may be a computer, then applies “pattern recognition” for identifying objects as, for example, pedestrians, buildings, or traffic signs. Id. ¶¶ 7, 9. Additional distance measurement means and image sensors can be installed as necessary, “such as on the front, rear, left, or right of the vehicle.” Id. ¶ 11. Once a preceding vehicle is identified, the vehicle’s throttle, brakes, and steering wheel can be controlled to maintain a safe following distance. Id. ¶ 15. 2. Analysis We agree with Toyota that Yamamura teaches most of the limitations of independent claim 30. Pet. 59 (citing Pet. 53–55 for analysis of similar limitations in claim 1). Claim 30 recites at least one receiver arranged to receive waves from the environment outside the vehicle and generate a signal characteristic of the received waves. Yamamura discloses a distance measurement means, such as a laser or radar, as well as a video camera that obtains images of exterior objects. Ex. 1013 ¶ 7. Claim 30 also requires a processor that classifies, identifies, or locates an exterior object based on the generated signal, which is met by Yamamura’s image processor that IPR2013-00419 Patent 6,772,057 B2 27 identifies objects. Id. ¶¶ 7, 9. In addition, the operation of a system in Yamamura, such as the throttle, brakes, or steering wheel, is affected in response to the classification, identification, or location of the exterior object, as required by claim 30. Id. ¶15. As with the obviousness ground based on Lemelson and Borcherts, AVS argues that Borcherts does not disclose a receiver arranged on a rear view mirror, as recited in claim 30. PO Resp. 29. Similar to our determination with respect to Lemelson and Borcherts, we are persuaded that Borcherts’s description of a camera “in combination with” a rear view mirror, along with Figure 1 of Borcherts, which appears to illustrate a camera on a vehicle’s rear view mirror, would have suggested modifying Yamamura’s system to place a receiver on the rear view mirror. Toyota also has articulated sufficient reasoning with rational underpinning for combining the teachings of Borcherts and Yamamura—mounting Yamamura’s camera on the rear view mirror “would have helped keep the image sensor clean while still obtaining a good image of the road.” Pet. 60 (citing Ex. 1016 ¶¶ 175–180). Accordingly, we determine that Toyota has shown by a preponderance of the evidence that claim 30 is unpatentable for obviousness over Yamamura and Borcherts. We also are persuaded, based on Toyota’s Petition, that Yamamura discloses the additional limitations in dependent claims 32 and 37–39. See Pet. 59–60 (citing Pet. 55–56 for analysis of similar limitations in claims 2 and 8–10, respectively). Thus, Toyota has shown by a preponderance of the evidence that claims 32 and 37–39 are unpatentable for obviousness over Yamamura and Borcherts. IPR2013-00419 Patent 6,772,057 B2 28 III. CONCLUSION Based on the evidence and arguments, Toyota has demonstrated by a preponderance of the evidence that: (1) claims 40, 46, 48, and 49 are unpatentable under 35 U.S.C. § 102(e) as anticipated by Lemelson; (2) claims 30, 32–34, and 37–39 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Borcherts; (3) claim 43 is unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Asayama; (4) claim 34 is unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson, Borcherts, and Asayama; and (5) claims 30, 32, and 37–39 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Yamamura and Borcherts. Based on the evidence and arguments, Toyota has not demonstrated by a preponderance of the evidence that: (1) claims 1–4, 7–10, 41, 56, 59–61, and 64 are unpatentable under 35 U.S.C. § 102(e) as anticipated by Lemelson; (2) claims 31 and 62 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Borcherts; and (3) claims 4 and 59 are unpatentable under 35 U.S.C. § 103(a) for obviousness over Lemelson and Asayama. IV. ORDER Accordingly, it is: ORDERED that claims 30, 32–34, 37–40, 43, 46, 48, and 49 of U.S. Patent No. 6,772,057 are unpatentable; and IPR2013-00419 Patent 6,772,057 B2 29 FURTHER ORDERED that claims 1–4, 7–10, 31, 41, 56, 59–62, and 64 of U.S. Patent No. 6,772,057 have not been shown to be unpatentable. This is a final decision. Parties to the proceeding seeking judicial review of the decision must comply with the notice and service requirements of 37 C.F.R. § 90.2. IPR2013-00419 Patent 6,772,057 B2 30 FOR PETITIONER: Matthew Berkowitz Antony Pfeffer Thomas Makin mberkowitz@kenyon.com apfeffer@kenyon.com tmakin@kenyon.com FOR PATENT OWNER: Thomas J. Wimbiscus Scott P. McBride Christopher M. Scharff McAndrews Held & Malloy twimbiscus@mcandrews-ip.com smcbride@mcandrews-ip.com cscharff@mcandrews-ip.com Copy with citationCopy as parenthetical citation