Opinion
Case No. 2:15-cv-1523-JRG-RSP
09-27-2016
INFERNAL TECHNOLOGY, LLC ET AL., Plaintiffs, v. ELECTRONIC ARTS INC. ET AL., Defendants.
MEMORANDUM OPINION AND ORDER
Before the Court is the opening claim construction brief of Terminal Reality, Inc. and Infernal Technology, LLC (collectively "Plaintiffs") (Dkt. No. 77, filed on July 12, 2016), the response of Electronic Arts Inc. ("Defendant") (Dkt. No. 86, filed on July 27, 2016), and the reply of Plaintiffs (Dkt. No. 87, filed on August 3, 2016). The Court held a hearing on the issue of claim construction on August 23, 2016. Having considered the arguments and evidence presented by the parties at the hearing and in their briefing, having considered the intrinsic evidence, and having made subsidiary factual findings about the extrinsic evidence, the Court hereby issues this Claim Construction Memorandum and Order. See Phillips v. AWH Corp., 415 F.3d 1303, 1314 (Fed. Cir. 2005); Teva Pharm. USA, Inc. v. Sandoz, Inc., 135 S. Ct. 831, 841 (2015).
Citations to the parties' filings are to the filing's number in the docket (Dkt. No.) and pin cites are to the page numbers assigned through ECF.
Table of Contents
I. BACKGROUND ............................................................................................................... 3
II. LEGAL PRINCIPLES ..................................................................................................... 5
A. Claim Construction ................................................................................................. 5
B. Departing from the Ordinary Meaning of a Claim Term ........................................ 8
C. Functional Claiming and 35 U.S.C. § 112, ¶ 6 (pre-AIA) / § 112(f) (AIA) ......... 10
D. Definiteness Under 35 U.S.C. § 112, ¶ 2 (pre-AIA) / § 112(b) (AIA) ................. 12
III. PERSON OF ORDINARY SKILL IN THE ART ....................................................... 13
IV. AGREED CONSTRUCTIONS ...................................................................................... 13
V. CONSTRUCTION OF DISPUTED TERMS ............................................................... 14
A. "observer data of a simulated multi-dimensional scene" ...................................... 14
B. "lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data" ...................................................................................................................... 18
C. "light image data" ................................................................................................. 19
D. "for each of said plurality of light sources . . . light accumulation buffer" .......... 22
E. "combining at least a portion of said light accumulation buffer with said observer data" ....................................................................................................... 30
F. "displaying resulting image data to a computer screen" ....................................... 33
G. "outputting resulting image data" ......................................................................... 36
H. "precalculated matrix transformation look-up table" ........................................... 39
I. "selectively controlled source color data that can be changed over a period of time" ................................................................................................................. 42
VI. CONCLUSION ............................................................................................................... 45
I. BACKGROUND
Plaintiffs allege infringement of U.S. Patents No. 6,362,822 (the "'822 Patent) and No. 7,061,488 (the "'488 Patent") (collectively, the "Asserted Patents"). The '822 Patent is entitled "Lighting and Shadowing Methods and Arrangements for Use in Computer Graphic Simulations." The application leading to the '822 Patent was filed on March 12, 1999 and the patent issued on March 26, 2002. The '488 Patent is also entitled "Lighting and Shadowing Methods and Arrangements for Use in Computer Graphic Simulations." The application leading to the '488 Patent was filed on December 6, 2001 and the patent issued on June 13, 2006. The application that issued as the '488 Patent is a continuation of the application that issued as the '822 Patent.
In general, the Asserted Patents are directed to technology for handling lighting and shadowing in computer graphics. The technology can be generally understood with reference to Figures 2, 3, and 4 of the '822 Patent. Figure 2, reproduced here, depicts a simulated three-dimensional ("3D") scene (10). The scene includes at least one 3D object (12) that is represented by spatial data, such as polygons. And the scene is illuminated by one or more light sources (16, 18). The 3D scene is rendered in two dimensions ("2D") and the 2D image is suitable for display, such as on a computer screen. The 2D view of the 3D scene is from the observer's, or camera's,
The disclosures of the '822 Patent and the '488 Patent are substantially the same. As such, the Court cites the '822 Patent understanding that the cited material is also in the '488 Patent.
'822 Patent Figure 2
Image materials not available for display. (14) perspective and takes into account the light incident on the 3D object as viewed from the observer's perspective. The relative positions of the object, observer, and light sources are defined using a 3D coordinate system (20). '822 Patent col.1 ll.25-38, col.6 ll.28-57.
Figures 3 and 4, reproduced here, depict data structures and the image processing flow for producing an image for display. The scene is rendered into 2D from the observer's (camera's) view (step 102) and also from each light source's view (step 104). The 2D data for each view include image (e.g., color) and depth information at each point in the view's 2D coordinate system (data structure 51A, 51B, 51C, 51D, 51E, 51F). The 2D version of the scene from the observer's view is transformed from the observer's coordinate system to the light source's coordinate system (step 106) and for each region in the observer's view that is illuminated by the light source, the light image information is accumulated in a light source buffer (data structure 51G) (step 108). After the observer data is processed for each light source to generate the accumulated light data, the accumulated light data is combined with the observer data to generate the image for display (data structure 50; step 118). Id. at col.6 l.58 - col.9 l.22.
'822 Patent
Image materials not available for display.
Figure 3
Image materials not available for display.
Figure 4
The abstracts of the Asserted Patents are identical and provide:
The effects of lighting and resulting shadows within a computer simulated three-dimensional scene are modeled by rendering a light depth image and a light color image for each of the light sources. The light depth images are compared to a camera depth image to determine if a point within the scene is lighted by the various light sources. An accumulated light image is produced by combining those portions of the light color images determined to be lighting the scene. The resulting accumulated light image is then combined with a camera color image to produce a lighted camera image that can be further processed and eventually displayed on a computer display screen. The light color image can be static or dynamic. Transformations between different perspective and/or coordinate systems can be precalculated for fixed cameras or light sources. The various images and manipulations can include individual pixel data values, multiple-pixel values, polygon values, texture maps, and the like.
'822 Patent
1. A shadow rendering method for use in a computer system, the method comprising the steps of:
providing observer data of a simulated multi-dimensional scene;
providing lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting, data including light image data;
for each of said plurality of light sources, comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in a light accumulation buffer; and then
combining at least a portion of said light accumulation buffer with said observer data; and
displaying resulting image data to a computer screen.
'488 Patent
27. A computer-readable medium carrying at least one set of computer instructions configured to cause at least one processor to operatively render simulated shadows in a multi-dimensional simulated scene by performing the steps of:
providing observer data of a simulated multi-dimensional scene;
providing lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data;
for each of said plurality of light sources, complying at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in a light accumulation buffer; and then
combining at least a portion of said light accumulation buffer with said observer data; and
outputting resulting image data.
Exemplary Claim 1 of the '822 Patent and Claim 27 of the '488 Patent recite as follows:
II. LEGAL PRINCIPLES
A. Claim Construction
"It is a 'bedrock principle' of patent law that 'the claims of a patent define the invention to which the patentee is entitled the right to exclude.'" Phillips v. AWH Corp., 415 F.3d 1303, 1312 (Fed. Cir. 2005) (en banc) (quoting Innova/Pure Water Inc. v. Safari Water Filtration Sys., Inc., 381 F.3d 1111, 1115 (Fed. Cir. 2004)). To determine the meaning of the claims, courts start by considering the intrinsic evidence. Id. at 1313; C.R. Bard, Inc. v. U.S. Surgical Corp., 388 F.3d 858, 861 (Fed. Cir. 2004); Bell Atl. Network Servs., Inc. v. Covad Commc'ns Group, Inc., 262 F.3d 1258, 1267 (Fed. Cir. 2001). The intrinsic evidence includes the claims themselves, the specification, and the prosecution history. Phillips, 415 F.3d at 1314; C.R. Bard, Inc., 388 F.3d at 861. The general rule—subject to certain specific exceptions discussed infra—is that each claim term is construed according to its ordinary and accustomed meaning as understood by one of ordinary skill in the art at the time of the invention in the context of the patent. Phillips, 415 F.3d at 1312-13; Alloc, Inc. v. Int'l Trade Comm'n, 342 F.3d 1361, 1368 (Fed. Cir. 2003); Azure Networks, LLC v. CSR PLC, 771 F.3d 1336, 1347 (Fed. Cir. 2014) ("There is a heavy presumption that claim terms carry their accustomed meaning in the relevant community at the relevant time.") (vacated on other grounds).
"The claim construction inquiry. . . begins and ends in all cases with the actual words of the claim." Renishaw PLC v. Marposs Societa' per Azioni, 158 F.3d 1243, 1248 (Fed. Cir. 1998). "[I]n all aspects of claim construction, 'the name of the game is the claim.'" Apple Inc. v. Motorola, Inc., 757 F.3d 1286, 1298 (Fed. Cir. 2014) (quoting In re Hiniker Co., 150 F.3d 1362, 1369 (Fed. Cir. 1998)). First, a term's context in the asserted claim can be instructive. Phillips, 415 F.3d at 1314. Other asserted or unasserted claims can also aid in determining the claim's meaning, because claim terms are typically used consistently throughout the patent. Id. Differences among the claim terms can also assist in understanding a term's meaning. Id. For example, when a dependent claim adds a limitation to an independent claim, it is presumed that the independent claim does not include the limitation. Id. at 1314-15.
"[C]laims 'must be read in view of the specification, of which they are a part.'" Id. (quoting Markman v. Westview Instruments, Inc., 52 F.3d 967, 979 (Fed. Cir. 1995) (en banc)). "[T]he specification 'is always highly relevant to the claim construction analysis. Usually, it is dispositive; it is the single best guide to the meaning of a disputed term.'" Id. (quoting Vitronics Corp. v. Conceptronic, Inc., 90 F.3d 1576, 1582 (Fed. Cir. 1996)); Teleflex, Inc. v. Ficosa N. Am. Corp., 299 F.3d 1313, 1325 (Fed. Cir. 2002). But, "'[a]lthough the specification may aid the court in interpreting the meaning of disputed claim language, particular embodiments and examples appearing in the specification will not generally be read into the claims.'" Comark Commc'ns, Inc. v. Harris Corp., 156 F.3d 1182, 1187 (Fed. Cir. 1998) (quoting Constant v. Advanced Micro-Devices, Inc., 848 F.2d 1560, 1571 (Fed. Cir. 1988)); see also Phillips, 415 F.3d at 1323. "[I]t is improper to read limitations from a preferred embodiment described in the specification—even if it is the only embodiment—into the claims absent a clear indication in the intrinsic record that the patentee intended the claims to be so limited." Liebel-Flarsheim Co. v. Medrad, Inc., 358 F.3d 898, 913 (Fed. Cir. 2004).
The prosecution history is another tool to supply the proper context for claim construction because, like the specification, the prosecution history provides evidence of how the U.S. Patent and Trademark Office ("PTO") and the inventor understood the patent. Phillips, 415 F.3d at 1317. However, "because the prosecution history represents an ongoing negotiation between the PTO and the applicant, rather than the final product of that negotiation, it often lacks the clarity of the specification and thus is less useful for claim construction purposes." Id. at 1318; see also Athletic Alternatives, Inc. v. Prince Mfg., 73 F.3d 1573, 1580 (Fed. Cir. 1996) (ambiguous prosecution history may be "unhelpful as an interpretive resource").
Although extrinsic evidence can also be useful, it is "'less significant than the intrinsic record in determining the legally operative meaning of claim language.'" Phillips, 415 F.3d at 1317 (quoting C.R. Bard, Inc., 388 F.3d at 862). Technical dictionaries and treatises may help a court understand the underlying technology and the manner in which one skilled in the art might use claim terms, but technical dictionaries and treatises may provide definitions that are too broad or may not be indicative of how the term is used in the patent. Id. at 1318. Similarly, expert testimony may aid a court in understanding the underlying technology and determining the particular meaning of a term in the pertinent field, but an expert's conclusory, unsupported assertions as to a term's definition are entirely unhelpful to a court. Id. Generally, extrinsic evidence is "less reliable than the patent and its prosecution history in determining how to read claim terms." Id. The Supreme Court recently explained the role of extrinsic evidence in claim construction:
In some cases, however, the district court will need to look beyond the patent's intrinsic evidence and to consult extrinsic evidence in order to understand, for example, the background science or the meaning of a term in the relevant art during the relevant time period. See, e.g., Seymour v. Osborne, 11 Wall. 516, 546 (1871) (a patent may be "so interspersed with technical terms and terms of art that the testimony of scientific witnesses is indispensable to a correct understanding of its meaning"). In cases where those subsidiary facts are in dispute, courts will need to make subsidiary factual findings about that extrinsic evidence. These are the "evidentiary underpinnings" of claim construction that we discussed in Markman, and this subsidiary factfinding must be reviewed for clear error on appeal.Teva Pharm. USA, Inc. v. Sandoz, Inc., 135 S. Ct. 831, 841 (2015).
B. Departing from the Ordinary Meaning of a Claim Term
There are "only two exceptions to [the] general rule" that claim terms are construed according to their plain and ordinary meaning: "1) when a patentee sets out a definition and acts as his own lexicographer, or 2) when the patentee disavows the full scope of the claim term either in the specification or during prosecution." Golden Bridge Tech., Inc. v. Apple Inc., 758 F.3d 1362, 1365 (Fed. Cir. 2014) (quoting Thorner v. Sony Computer Entm't Am. LLC, 669 F.3d 1362, 1365 (Fed. Cir. 2012)); see also GE Lighting Solutions, LLC v. AgiLight, Inc., 750 F.3d 1304, 1309 (Fed. Cir. 2014) ("[T]he specification and prosecution history only compel departure from the plain meaning in two instances: lexicography and disavowal."). The standards for finding lexicography or disavowal are "exacting." GE Lighting Solutions, 750 F.3d at 1309.
Some cases have characterized other principles of claim construction as "exceptions" to the general rule, such as the statutory requirement that a means-plus-function term is construed to cover the corresponding structure disclosed in the specification. See, e.g., CCS Fitness, Inc. v. Brunswick Corp., 288 F.3d 1359, 1367 (Fed. Cir. 2002).
To act as his own lexicographer, the patentee must "clearly set forth a definition of the disputed claim term," and "clearly express an intent to define the term." Id. (quoting Thorner, 669 F.3d at 1365); see also Renishaw, 158 F.3d at 1249. The patentee's lexicography must appear "with reasonable clarity, deliberateness, and precision." Renishaw, 158 F.3d at 1249.
To disavow or disclaim the full scope of a claim term, the patentee's statements in the specification or prosecution history must amount to a "clear and unmistakable" surrender. Cordis Corp. v. Boston Sci. Corp., 561 F.3d 1319, 1329 (Fed. Cir. 2009); see also Thorner, 669 F.3d at 1366 ("The patentee may demonstrate intent to deviate from the ordinary and accustomed meaning of a claim term by including in the specification expressions of manifest exclusion or restriction, representing a clear disavowal of claim scope."). "Where an applicant's statements are amenable to multiple reasonable interpretations, they cannot be deemed clear and unmistakable." 3M Innovative Props. Co. v. Tredegar Corp., 725 F.3d 1315, 1326 (Fed. Cir. 2013).
C. Functional Claiming and 35 U.S.C. § 112 , ¶ 6 (pre-AIA) / § 112(f) (AIA)
Because the applications resulting in the Asserted Patents were filed before September 16, 2012, the effective date of the America Invents Act ("AIA"), the Court refers to the pre-AIA version of § 112.
A patent claim may be expressed using functional language. See 35 U.S.C. § 112, ¶ 6; Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1347-49 & n.3 (Fed. Cir. 2015) (en banc in relevant portion). Section 112, Paragraph 6, provides that a structure may be claimed as a "means . . . for performing a specified function" and that an act may be claimed as a "step for performing a specified function." Masco Corp. v. United States, 303 F.3d 1316, 1326 (Fed. Cir. 2002).
But § 112, ¶ 6 does not apply to all functional claim language. There is a rebuttable presumption that § 112, ¶ 6 applies when the claim language includes "means" or "step for" terms, and that it does not apply in the absence of those terms. Masco Corp., 303 F.3d at 1326; Williamson, 792 F.3d at 1348. The presumption stands or falls according to whether one of ordinary skill in the art would understand the claim with the functional language, in the context of the entire specification, to denote sufficiently definite structure or acts for performing the function. See Media Rights Techs., Inc. v. Capital One Fin. Corp., 800 F.3d 1366, 1372 (Fed. Cir. 2015) (§ 112, ¶ 6 does not apply when "the claim language, read in light of the specification, recites sufficiently definite structure" (quotation marks omitted) (citing Williamson, 792 F.3d at 1349; Robert Bosch, LLC v. Snap-On Inc., 769 F.3d 1094, 1099 (Fed. Cir. 2014))); Williamson, 792 F.3d at 1349 (§ 112, ¶ 6 does not apply when "the words of the claim are understood by persons of ordinary skill in the art to have sufficiently definite meaning as the name for structure"); Masco Corp., 303 F.3d at 1326 (§ 112, ¶ 6 does not apply when the claim includes an "act" corresponding to "how the function is performed"); Personalized Media Communications, L.L.C. v. International Trade Commission, 161 F.3d 696, 704 (Fed. Cir. 1998) (§ 112, ¶ 6 does not apply when the claim includes "sufficient structure, material, or acts within the claim itself to perform entirely the recited function . . . even if the claim uses the term 'means.'" (quotation marks and citation omitted)).
When it applies, § 112, ¶ 6 limits the scope of the functional term "to only the structure, materials, or acts described in the specification as corresponding to the claimed function and equivalents thereof." Williamson, 792 F.3d at 1347. Construing a means-plus-function limitation involves multiple steps. "The first step . . . is a determination of the function of the means-plus-function limitation." Medtronic, Inc. v. Advanced Cardiovascular Sys., Inc., 248 F.3d 1303, 1311 (Fed. Cir. 2001). "[T]he next step is to determine the corresponding structure disclosed in the specification and equivalents thereof." Id. A "structure disclosed in the specification is 'corresponding' structure only if the specification or prosecution history clearly links or associates that structure to the function recited in the claim." Id. The focus of the "corresponding structure" inquiry is not merely whether a structure is capable of performing the recited function, but rather whether the corresponding structure is "clearly linked or associated with the [recited] function." Id. The corresponding structure "must include all structure that actually performs the recited function." Default Proof Credit Card Sys. v. Home Depot U.S.A., Inc., 412 F.3d 1291, 1298 (Fed. Cir. 2005). However, § 112 does not permit "incorporation of structure from the written description beyond that necessary to perform the claimed function." Micro Chem., Inc. v. Great Plains Chem. Co., 194 F.3d 1250, 1258 (Fed. Cir. 1999).
For § 112, ¶ 6 limitations implemented by a programmed general purpose computer or microprocessor, the corresponding structure described in the patent specification must include an algorithm for performing the function. WMS Gaming Inc. v. Int'l Game Tech., 184 F.3d 1339, 1349 (Fed. Cir. 1999). The corresponding structure is not a general purpose computer but rather the special purpose computer programmed to perform the disclosed algorithm. Aristocrat Techs. Austl. Pty Ltd. v. Int'l Game Tech., 521 F.3d 1328, 1333 (Fed. Cir. 2008).
D. Definiteness Under 35 U.S.C. § 112 , ¶ 2 (pre-AIA) / § 112(b) (AIA)
Because the applications resulting in the Asserted Patents were filed before the effective date of the America Invents Act ("AIA"), the Court refers to the pre-AIA version of § 112.
Patent claims must particularly point out and distinctly claim the subject matter regarded as the invention. 35 U.S.C. § 112, ¶ 2. A claim, when viewed in light of the intrinsic evidence, must "inform those skilled in the art about the scope of the invention with reasonable certainty." Nautilus Inc. v. Biosig Instruments, Inc., 134 S. Ct. 2120, 2129 (2014). If it does not, the claim fails § 112, ¶ 2 and is therefore invalid as indefinite. Id. at 2124. Whether a claim is indefinite is determined from the perspective of one of ordinary skill in the art as of the time the application for the patent was filed. Id. at 2130. As it is a challenge to the validity of a patent, the failure of any claim in suit to comply with § 112 must be shown by clear and convincing evidence. Id. at 2130 n.10. "[I]ndefiniteness is a question of law and in effect part of claim construction." ePlus, Inc. v. Lawson Software, Inc., 700 F.3d 509, 517 (Fed. Cir. 2012).
When a term of degree is used in a claim, "the court must determine whether the patent provides some standard for measuring that degree." Biosig Instruments, Inc. v. Nautilus, Inc., 783 F.3d 1374, 1378 (Fed. Cir. 2015) (quotation marks omitted). Likewise, when a subjective term is used in a claim, "the court must determine whether the patent's specification supplies some standard for measuring the scope of the [term]." Datamize, LLC v. Plumtree Software, Inc., 417 F.3d 1342, 1351 (Fed. Cir. 2005); accord Interval Licensing LLC v. AOL, Inc., 766 F.3d 1364, 1371 (Fed. Cir. 2014) (citing Datamize, 417 F.3d at 1351).
In the context of a claim governed by 35 U.S.C. § 112, ¶ 6, the claim is invalid as indefinite if the claim fails to disclose adequate corresponding structure to perform the claimed functions. Williamson, 792 F.3d at 1351-52. The disclosure is inadequate when one of ordinary skill in the art "would be unable to recognize the structure in the specification and associate it with the corresponding function in the claim." Id. at 1352.
III. PERSON OF ORDINARY SKILL IN THE ART
Plaintiffs submit that a person of ordinary skill in the art at the time of the invention "would have a Bachelor's of Science degree in electrical engineering, computer science, software engineering, or equivalent, with at least two years of work experience in the field of three-dimensional computer graphics." Dkt. No. 77 at 12 (citing Ferraro Decl. ¶ 4, Dkt. No. 77-3 at 3).
Declaration of Richard F. Ferraro (Ferraro Decl.), Plaintiffs' Exhibit B, Dkt. No. 77-3.
Defendant neither proposes a definition of the person of ordinary skill in the art nor objects to Plaintiffs' proposed definition.
Accordingly, the Court finds that the person of ordinary skill in the art at the time of the invention would have a Bachelor's of Science degree in electrical engineering, computer science, software engineering, or equivalent, with at least two years of work experience in the field of three-dimensional computer graphics.
IV. AGREED CONSTRUCTIONS
The parties have agreed to the following constructions set forth in their Joint Patent Rule 4-5(d) Claim Construction Chart (Dkt. No. 88).
| Agreed Construction | |
|---|---|
| "providing"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | "making available" |
| "associated with"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | "related to" |
For all term charts in this order, the claims in which the term is found are listed with the term but: (1) only the highest level claim in each dependency chain is listed, and (2) only asserted claims identified in the parties' Joint Patent Rule 4-5(D) Claim Construction Chart (Dkt. No. 88) are listed.
Having reviewed the intrinsic and extrinsic evidence of record, the Court agrees with and hereby adopts the parties' agreed constructions.
V. CONSTRUCTION OF DISPUTED TERMS
A. "observer data of a simulated multi-dimensional scene"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "observer data of a simulatedmulti-dimensional scene"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | data representing a simulatedmulti-dimensional sceneviewed from an observer'sperspective, including at leastcolor data and depth data forthe scene from the observer'sperspective | polygon color and/or depthfrom an observer'sperspective |
The Parties' Positions
Plaintiffs submit that the "observer data" is the result of rendering a 3D object as a 2D image from the observer's perspective. Dkt. No. 77 at 13. This 2D observer (or camera) data, according to Plaintiffs, includes color data (e.g., "RGB pixel data values") and depth data. Id. Plaintiffs argue that this is the way the 2D data is described in the Asserted Patents and the way "observer data" is used in the claims. Id. at 13-14. Specifically, Plaintiffs contend: (1) the "observer data" is compared with "lighting data," the "lighting data" includes depth data, therefore the "observer data" includes depth data, and (2) the "observer data" is combined with the "light accumulation buffer," the data of the "light accumulation buffer" that is combined with the observer data is color data, therefore the "observer data' includes color data. Id. at 13-14 (citing Ferraro Decl. ¶¶ 15-16). Plaintiffs further submit that the "observer data" is not "polygon" data, as Defendant proposes. Rather, Plaintiffs contend, polygon data is in the 3D domain whereas "observer data" is the pixel data in the 2D domain. Id. at 14.
In addition to the claims themselves, Plaintiffs cite the following intrinsic and extrinsic evidence to support their position: Intrinsic evidence: '822 Patent col.3 ll.37-38, col.4 ll.24-26, col.7 ll.4-11, col.8 ll.39-41, figs.3, 4. Extrinsic evidence: Ferraro Decl. ¶¶ 15-16 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 6-7).
Defendant responds that "observer data" refers to polygon data and does not necessarily include both color and depth data. Dkt. No. 86 at 9. According to Defendant, the patents describe that "observer data" includes color and depth data "associated with a plurality of modeled polygons . . . as rendered from an observer's perspective." Id. at 9-10 (quoting '822 Patent col.3 ll.38-46). This color and depth data is "assigned to pixels when generating the 'observer data'" and "is the color and depth of the polygons in the scene visible at each pixel" as viewed before lighting is applied. Id. at 10. Defendant argues that Plaintiffs' proposed construction is improper because it does not reflect that the source of the observer data is the polygon. Id. And, according to Defendant, the requirement that observer data include both color and depth data is expressly recited in Claim 2 of the '822 Patent, and therefore should not be read into "observer data" because doing so would render Claim 2 superfluous. Id. at 11-12.
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '822 Patent, at [57] Abstract, col.3 ll.33-35, col.3 ll.38-46, col.6 ll.30-33, col.7 ll.5-6, col.9 ll.13-17, col.9 ll.43-44; Preliminary Response, IPR2016-00928 (excerpts) (Defendant's Ex. 1).
Exclusive Licensee Infernal Technology LLC's Preliminary Response, IPR2016-00928, Patent 6,362,822.
Plaintiffs reply that the patents describe that the observer data represents the modeled polygons "as rendered from an observer's perspective" and that "as rendered" means the polygons are represented as pixels in the 2D observer view. Dkt. No. 87 at 4. Plaintiffs further reply that if observer data does not include depth data, then the comparing step cannot be performed and Claim 1 of each patent would be inoperable. Id. Finally, Plaintiffs reply that the difference between Claim 1 and Claim 2 of the '822 Patent is that Claim 2 requires "a plurality of modeled polygons," not that it requires both observed color and depth data. Id. at 5.
Analysis
There are two issues raised by the dispute. First, whether "observer data" necessarily includes both color and depth data. Second, whether "observer data" necessarily represents polygon data. With respect to the first issue, the Court understands that "observer data" necessarily includes color but does not necessarily include depth data. With respect to the second issue, the Court understands that "observer data" represents the scene, as viewed from the observer's perspective, and therefore the color data represents the color of the objects defining the scene. But the Court does not understand that the simulated scene is necessarily represented in 3D as polygons.
"Observer data" does not necessarily include both color and depth data. The Court is not persuaded by Plaintiffs' argument that observer data necessarily includes depth data because depth data is necessarily used to determine if a modeled point within the scene is illuminated by a light source. So using depth data is not expressed in Claim 1 of the '822 Patent or in Claims 1 or 27 of the '488 Patent. In fact, such as comparison is described in the patents as a "further embodiment" of the basic shadow-rendering method. '822 Patent col.3 ll.19-35, col.3 ll.53-57. Further, Claim 7 of the '822 Patent recites such a depth comparison to determine if the modeled point is illuminated. Under these circumstances, the Court will not read a depth-comparison limitation into the "comparing" step of the claims and the Court discerns no other reason to require "observer data" to include depth data.
The Court does, however, determine that "observer data" includes at least an "observer image" which is set forth in the patents as including data regarding the color of objects in the scene. The Court understands that: (1) the claims at issue each include "combining" observer data with accumulated light image data to produce an image of the scene for display, and (2) an observer image is necessary to produce such an image. And the Court further understands that an observer image inherently includes at least data regarding the color of the modeled point (the point on the modeled object in the scene). Therefore, "observer data" necessarily includes at least color data.
The "observer data" is not necessarily polygon data. Objects in a multi-dimensional scene may be represented in a computer simulation (modeled) using polygons. But the patents explain that using polygons is typical or exemplary, not that polygons must be used. See, e.g., id. at col.1 ll.25-26 ("These 3D objects are typically modeled by one or more polygons." (emphasis added)), col.6 ll.32-33 ("3D object 12 is represented by spacial data, for example , a plurality of polygons." (emphasis added)). Further, to the extent Defendant contends that observer data includes 3D data for the object, rather than data about the 3D object as viewed from the observer's perspective, the Court disagrees. The patents explain that the "images" of the scene are 2D views of the scene. For example, the patents provide that
once the arrangement of these 3D objects with respect to one another has been determined for a particular viewpoint, the rendered scene of the 3D world is projected onto a two-dimensional (2D) frame that can be displayed to the user. The frame is typically a data base that contains pixel color information with regard to the display screen. ... [and] advanced algorithms are implemented in either hardware and/or software to further streamline the processing of the resulting 3D scene and 2D images.Id. at col.1 ll.35-53.
Accordingly, the Court construes "observer data of a simulated multi-dimensional scene" as follows:
• "observer data of a simulated multi-dimensional scene" means "data representing at least the color of objects in a simulated multi-dimensional scene as viewed from an observer's perspective."
B. "lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "lighting data associated witha plurality of simulated lightsources arranged to illuminatesaid scene, said lighting dataincluding light image data"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | The parties agree that it isunnecessary to construe thephrase "lighting dataassociated with a plurality ofsimulated light sourcesarranged to illuminate saidscene" further. | data relating to therepresentation of simulatedlight sources arranged toilluminate said scene,including light image data foreach light source• To the extent Plaintiffsconcede that lightimage data is requiredfor each light source,there is no further needto construe this term. |
The Parties' Positions
Plaintiffs submit that Defendant's proposed construction is improper because it includes "including light image data for each light source" which appears elsewhere in the claims. Dkt. No. 77 at 15.
Defendant responds that the claims require light image data for each light source. Dkt. No. 86 at 12 & n.4.
Plaintiffs reply that it "concedes that it is unnecessary to construe this phrase further" but note that the claims do not include "for each light source." Dkt. No. 87 at 5 n.9.
Analysis
At the hearing, the parties agreed that light image data is required for each light source and, as such, there is no further dispute as to this term.
C. "light image data"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "light image data"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | data representing the lightemitted by each of theplurality of light sources | pixels representing the lightemitted by each of theplurality of light sources |
The Parties' Positions
Plaintiffs submit that the Asserted Patents describe that pixels may be represented by data, not that "pixels" are coextensive with "data" and, therefore, it would be improper to construe "light image data" as "pixels." Dkt. No. 77 at 16. Plaintiffs further submit that the data may represent a single pixel and, therefore, it would be improper to require more than one pixel. Id.
In addition to the claims themselves, Plaintiffs cite the following intrinsic evidence to support their position: '822 Patent col.1 ll.31-32, col.7 ll.15-20, col.7 ll.29-33.
Defendant responds that an "image" is a two-dimensional array of pixels and therefore the "light image data" refers to the pixels, not just any data. Dkt. No. 86 at 13-14. Defendant argues that Plaintiff's proposed construction fails to give effect to the word "image" in the term and thereby also fails to account for the association between the light image data and the modeled points within the scene. Id. at 14-15.
In addition to the claims themselves, Defendant cites the following intrinsic and extrinsic evidence to support its position: Intrinsic evidence: '822 Patent col.6 ll.62-63, col.7 ll.15-17, col.7 ll.19-20, col.7 ll.23-31, col.8 ll.3-11, col.10 ll.62-66, figs.3, 4, 7C. Extrinsic evidence: Ferraro Decl. ¶¶ 10, 14 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 5-6).
Plaintiffs reply that the patents provide that a light image may be a number of pixels, but that they also provide that "other conventions and/or arrangements can also be used for storing and manipulating the data." Dkt. No. 87 at 5-6.
Plaintiffs cite further intrinsic evidence to support their position: '822 Patent col.7 ll.1-3.
Analysis
There are two issues raised by the dispute. First, whether "light image data" is necessarily in pixel form. Second, whether "light image data" necessarily includes a plurality of pixels. With respect to the first issue, the Court understands that while "light image data" necessarily represents scene lighting from the 2D perspective of the light source, it is not necessarily in pixel form. With respect to the second issue, the Court understands that "light image data" may represent a single pixel.
"Light image data" is not necessarily in pixel (or array) form. The patents explain that a "light image" is the view of the scene from the light source's perspective. See, e.g., '822 Patent col.7 ll.15-16 ("For light source #1, a light image 51 C is rendered of scene 10."), col.8 ll.34-44 ("In step 104, the view [of scene 10] from a light source is rendered. For example, in step 104, rendering the view from light source #1 produces light #1 image 51C and light #1 depth 51D."). And the patents further describe that while image data (observer or light-source) may be in the form of a 2D array with entries corresponding to the pixels of an output device, "other conventions and/or arrangements can also be used for storing and manipulating the data." Id. at col.6 l.58 - col.7 l.3. Indeed, the patents describe an embodiment where light image data is for polygons rather than for pixels:
If the camera is continually moving with respect to scene 10, rather than manipulating and accumulating light based on pixels, process 100 is adapted to manipulate groups of pixels and/or polygons. This allows for a more granular approach, in which a light image can be rendered for each polygon, and light accumulation buffers can be used to accumulate light intensities from the various light sources for each polygon, rather than for each screen pixel.Id. at col.11 ll.51-61.
The "light image data" does not necessarily include more than one pixel. As set forth above, the "light image data" is not necessarily in the form of pixels, therefore it need not include more than one pixel. Further, the patents explain that even when in pixel form, the lighting data may comprise data for only those pixels that changed from frame to frame, and that not all pixels must change from frame to frame. Id. at col.11 ll.15-27. From this, the Court concludes that "light image data" may reflect a change from frame to frame that consists of a change in value of a single pixel.
Accordingly, the Court construes "light image data" as follows:
• "light image data" means "for each of the plurality of light sources, data representing an image of the light emitted by the light source to illuminate the scene as viewed from the light source's perspective."
D. "for each of said plurality of light sources . . . light accumulation buffer"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "for each of said plurality oflight sources, comparing atleast a portion of saidobserver data with at least aportion of said lighting datato determine if a modeledpoint within said scene isilluminated by said lightsource and storing at least aportion of said light imagedata associated with saidpoint and said light source ina light accumulation buffer"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | Plaintiffs contend thatconstruing the entirety of thisclaim step [1.4] isinappropriate. Plaintiffscontend the following termsin this step should beconstrued as follows:• "at least a portion of":plain and ordinarymeaning (i.e. "at leastsome but potentially allof")• "a modeled point withinsaid scene": "a pixel orgroup of pixelsrepresenting a point on amodeled polygon within asimulated multi-dimensional scene"• "light accumulationbuffer": "the region ofmemory separate anddistinct from the framebuffer where such data isstored" | the cumulative light emittedonto the scene by each lightsource is stored in a lightaccumulation buffer beforethe data in the lightaccumulation buffer iscombined with the observerdata• "at least a portion of":"one or more valuesfrom"• "a modeled point withinsaid scene": "a point on apolygon within saidscene"• "light accumulationbuffer": "a memory forstoring the cumulativelight emitted onto eachpixel in the scene, fromthe observer'sperspective" |
The Parties' Positions
Plaintiffs submit the it would be improper to rewrite the entire the entirety of this term and that Defendant's proposed construction is improper as it removes and changes limitations within the term and introduces redundancy into the claims. Dkt. No. 77 at 17-18. Specifically, Plaintiffs contend that Defendant's proposed construction is improper as it: (1) removes "comparing" and "to determine" from the term, (2) introduces a "combined" limitation that is expressed elsewhere in the claims, and (3) changes that which is combined with the observer data from "at least a portion of said light accumulation buffer" to "data in the light accumulation buffer." Id. at 18-19. With respect to specific phrases within the term, Plaintiffs submit as follows:
"At least a portion of": Plaintiffs submit that this term has its plain and ordinary meaning and that it would be improper to limit it to "one or more values from," as Defendant proposes. Id. at 19-20. Specifically, Plaintiffs argue that "at least a portion of [data]" has a meaning that is distinct from "one or more values from [data]." Id.
"A modeled point within said scene": Plaintiffs submit that a "modeled point" is a pixel or pixels representing a point on the polygon that is modeled. Id. at 20-21. According to Plaintiffs, the Asserted Patents describe that the invention determines if a pixel or group of pixels is illuminated by a light source, thus the modeled point, which the claimed process determines whether is illuminated, is a pixel or group of pixels. Id. And, Plaintiffs contend, the patents describe that 3D polygons are modeled by being converted to points (pixels) in a 2D image—polygons do not have points. Id. at 22.
"Light accumulation buffer": Plaintiffs submit that "light accumulation buffer" in "storing at least a portion of said light image data . . . in a light accumulation buffer" refers to memory in which data is stored. Id. at 23. This, according to Plaintiffs, is distinct from "light accumulation buffer" in "combining at least a portion of said light accumulation buffer with said observer data," which refers to what is stored in that memory. Id. Plaintiffs contend that "light accumulation buffer," when referring to a memory location in the patents, denotes memory "which accumulates light falling on a pixel" and that is separate and distinct from the "frame buffer" which stores the combination of light-accumulation data and observer data. Id. at 24-26. Plaintiffs argue that the separate and distinct nature of the two buffers is evinced by: (1) the description of the two buffers as separate and distinct in the exemplary embodiments, id. at 24- 25, (2) the separate recitation of the two buffers in Claim 11 of the '822 Patent, id. at 26, and (3) the patents' distinguishing of prior art based on a separate and distinct light accumulation buffer, id. at 26 (citing '822 Patent col.7 ll.13-14, col.7 ll.49-53). Finally, Plaintiffs submit that the light accumulation buffer stores only the light information for pixels that are illuminated, not for "each pixel in the scene," as Defendant suggests. Id. at 26-27.
In addition to the claims themselves, Plaintiffs cite the following intrinsic and extrinsic evidence to support their position: Intrinsic evidence: '822 Patent, at [57] Abstract, col.2 ll.25-30, col.3 ll.25-29, col.3 ll.40-43, col.3 ll.53-61, col.4 ll.6-20, col.4 ll.30-38, col.5 ll.5-7, col.5 l.66 - col.6 l.3, col.6 ll.33-37, col.7 ll.13-14, col.7 ll.43-61, col.8 ll.34-37, col.8 l.56 - col.9 l.22, col.11 ll.22-27, col.11 ll.36-43, figs.1, 2, 3, 4. Extrinsic evidence: Ferraro Decl. ¶¶ 20-24, 26 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 8-10).
Defendant responds that in addition to its contentions regarding the meanings of the constituent phrases in this term, it contends that the "for each of said plurality . . ." step must be performed before the combining step. Dkt. No. 86 at 16-17. Specifically, Defendant contends that light from every light source must be accumulated in the light accumulation buffer before any data in that buffer is combined with observer data. Id. With respect to specific phrases within the term, Defendant responds as follows:
"At least a portion of": Defendant responds that "at least a portion of" modifies various data sets in the claims, each of these data sets is a set of values, and, therefore, "at least a portion of" data means at least one value from the data set. Id. at 17-19. Defendant argues that Plaintiff's proposal threatens to improperly allow that any number less than or equal to any of the values in the data is "at least a portion of" the data. Id. at 19.
"A modeled point within said scene": Defendant responds that the patents describe that 3D objects within a scene are modeled with polygons and, therefore, a "modeled point" refers to a point on a polygon. Id. at 20. And the pixels in the image data are not modeled, they are associated with the modeled polygons. Id. at 20-21. Defendant further responds that the "comparing" that is done "to determine if a modeled point within said scene is illuminated" includes transforming 3D data—and specifically is not limited to, or even encompasses—comparing 2D data. Id. at 21-22.
"Light accumulation buffer": Defendant responds that the light accumulation buffer is not necessarily distinct from the frame buffer and that the light accumulation buffer stores the cumulative light emitted onto each pixel in the scene. Id. at 23-24. According to Defendant, a separate light accumulation buffer is simply a feature of an exemplary embodiment and the Asserted Patents allow that such a feature is optional. Id. at 23 (citing '822 Patent col.5 l.66 - col.6 l.3, col.6 ll.61-63, col.7 l.1-3). And, Defendant contends, the purported distinction between the inventions of the Asserted Patents and the prior art is not that the light accumulation buffer is separate from the frame buffer, but rather that the invention accumulates light rather than accumulating shadows. Id. at 24 (citing '822 Patent col.7 ll.49-53). Defendant also contends that while the light accumulation buffer stores "at least a portion of" of the light image data, it accumulates all of the light image data for that portion of the data that illuminates the portions of the scene visible to the observer. Id. at 24-26. Finally, Defendant agrees with Plaintiffs the "light accumulation buffer" in "storing . . . in a light accumulation buffer" refers to memory whereas "light accumulation buffer" in "combining . . . light accumulation buffer with said observer data" refers to values stored in the memory. Id. at 25 n.6.
In addition to the claims themselves, Defendant cites the following intrinsic and extrinsic evidence to support its position: Intrinsic evidence: '822 Patent col.1 ll.25-27, col.2 ll.25-28, col.3 ll.27-31, col.3 ll.38-40, col.3 ll.55-57, col.4 ll.16-17, col.4 ll.21-38, col.4 ll.41-43, col.4 ll.50-63, col.5 ll.6-9, col.5 l.66 - col.6 l.3, col.6 ll.30-33, col.6 ll.53-56, col.6 ll.61-63, col.6 l.67 - col.7 l.61, col.8 ll.6-8, col.8 ll.22-23, col.8 ll.45-67, col.9 ll.3-5, col.9 ll.8-17, col.9 l.42, col.9 ll.44-54, col.9 ll.59-62, col.10 ll.33-35, col.11 ll.22-25, figs.3, 4, 5, 7A. Extrinsic evidence: Ferraro Decl. ¶¶ 10, 12-13, 16-17, 23 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 5-7, 9); Goodin Decl. ¶¶ 15-22 (Ex. B to Corrected Joint Claim Construction and Prehearing Statement, Dkt. No. 76-2 at 5-8).
Declaration of Richard Goodin in Support of Defendant Electronic Arts, Inc.'s Claim Constructions. --------
Plaintiffs reply that the sequence of steps is expressly addressed in the language of the claims—the comparing and storing steps are performed for each light source before the combining step is performed. Dkt. No. 87 at 6. With respect to specific phrases within the term, Plaintiffs reply as follows:
"At least a portion of": Plaintiffs reply that "portions" and "values" are not synonymous in the patents, thus Defendant's proposed construction improperly equates the terms. Id. at 6-7.
"A modeled point within said scene": Plaintiffs reply that the patents are clear that the "comparing step determines which pixels or groups of pixels are illuminated." Id. at 7. Plaintiffs further reply that comparing the pixel data comes after the 3D polygon data is converted to the 2D image data. Id. And Plaintiffs reply that the patents describe a 2D transformation prior to comparing the pixel data. Id. at 7-8 (citing '822 Patent col.9 ll.31-44, fig.4).
"Light accumulation buffer": Plaintiffs reply that the patents consistently describe that the light accumulation buffer is separate from the frame buffer and that the different functions of the two buffers also evince that they are separate: the light accumulation buffer stores "RGB pixel data for the accumulated light falling on a pixel or group of pixels" and the frame buffer stores "a finally-rendered image for display." Id. at 9 (citing '822 Patent col.7 ll.47-49, col.9 ll.13-22). Finally, Plaintiffs reply that Defendant's proposed construction does not state that only light on pixels visible to the observer is accumulated—rather, that proposed construction states "the cumulative light emitted onto each pixel in the scene." Id. at 10.
Plaintiffs cite further intrinsic evidence to support their position:'822 Patent col.7 ll.15-28, col.8 ll.42-44, col.9 ll.31-44.
Analysis
There are five issues raised by the dispute. First, whether the comparing and storing steps necessarily are completed before the combining step begins. Second, whether "at least a portion of [data]" is necessarily "one or more values from [data]." Third, whether a "modeled point within said scene" is necessarily a point on a polygon used to model an object within a scene. Fourth, whether the "comparing" step to determine whether a modeled point within a scene is necessarily performed on 3D data. Fifth, whether the "light accumulation buffer" is necessarily separate from the frame buffer. With respect to the first issue, the comparing and storing steps are completed before the combining step begins. With respect to the second issue, the Court understands that while "at least a portion of [data]" encompasses "one or more values from [data]," it is not so limited. With respect to the third issue, "a modeled point within said scene" is a point on an object in the simulated scene as represented in the observer data. With respect to the fourth issue, the comparing step is accomplished using observer data and lighting data, both of which are 2D views of the scene. With respect to the fifth issue, although the light is accumulated in the "light accumulation buffer" before being combined with the observer data for display, the "light accumulation buffer" is not necessarily separate from the frame buffer that ultimately holds the result of that combination.
Order of the steps. From a plain reading of the claim language, the Court understands that the comparing and storing steps are completed for each light source before the combining step begins.
"At least a portion of": The Court is not persuaded that "at least a portion of," when referring to data, necessarily means at least one value from the data, as Defendant contends. Importantly, it is unclear what constitutes a "value" in Defendant's proposed construction. That said, the Court does not understand that the plain and ordinary meaning of "at least a portion of" data encompasses any number that is a fractional component of one number within the set. Something that is "at least a portion of" data must be tied to the data. For example, the Court understands that "comparing at least a portion of said observer data with at least a portion of said lighting data" is not satisfied by comparing two values unless one value is tied to the observer data and the other value is tied to the lighting data. But this does not preclude fractional comparisons, such as determining whether the observer data is a fractional component of the lighting data.
"A modeled point within said scene": As set forth in the above section on "light image data" the Court does not understand that scene-view image data is necessarily in the form of pixels (or arrays of data corresponding to display-device pixels). Further, as set forth in the above section on "observer data," the Court does not understand that objects in the simulated scene are necessarily modeled using polygons. That said, the Court understands that the "modeled point within said scene" refers to a point on a modeled 3D object within the simulated scene. But this does not mean that the "comparing" step necessarily is comparing 3D data. Rather, the claim language expresses that the comparison is between observer data and lighting data, both of which represent 2D views of the scene.
"Light accumulation buffer": Although the "light accumulation buffer" is depicted in certain exemplary embodiments as separate from the frame buffer, the Court does not understand the patents to require that the "light accumulation buffer" is separate from the "frame buffer." In fact, the patents describe that a single data structure may serve more than one purpose. For example, the "camera image" data structure at one point in the process holds just the camera image data and at another point in the process holds the combination of the camera image data and the accumulated light image data. Id. at col.9 ll.13-17 ("pixel data value from camera image 51A is multiplied by a corresponding pixel data value from light accumulation buffer 51G and the results are stored in camera image 51A"), col.9 l.44 ("CAMERA IMAGE (SPx, SPy) *= ACCUM (SPx, SPy)"). Thus, the patents expressly teach that a single data structure may serve more than one function. The Court discerns no contrary teaching that would require the frame buffer and light accumulation buffer be different data structures, logically or physically. That said, the Court reiterates that light accumulation data is not combined with the observer data until the comparing and storing steps are complete.
Accordingly, the Court construes this term as follows:
• order of the comparing, storing, and combining steps: the comparing and storing steps are completed before beginning the combining step;
• "at least a portion of" is not limited to "one or more values from," has its plain and ordinary meaning, and needs no further construction;
• "a modeled point within said scene" means "a point on a modeled object within said scene";
• "light accumulation buffer" means "memory for storing the light image data for cumulative light falling on a region in the observer image corresponding to the modeled point."
E. "combining at least a portion of said light accumulation buffer with said observer data"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "combining at least a portionof said light accumulationbuffer with said observerdata"• '822 Patent Claim 1• '488 Patent Claims 1, 27 | Plain and ordinary meaning.• "light accumulationbuffer": "stored datarepresenting theaccumulated light fallingon a pixel or group ofpixels" | multiplying one or more pixelvalues in the lightaccumulation buffer by thecorresponding polygon colorvalue(s) in the observer data |
The Parties' Positions
Plaintiffs submit that this term plainly refers to the production of composite image data by combining the light accumulation buffer and observer data. Dkt. No. 77 at 27-28. According to Plaintiffs, "combining" is not limited to "multiplying," as Defendant suggests, but rather includes other forms of combining, such as adding. Id. at 28. "Multiplying" is simply an exemplary embodiment described in the Asserted Patents and is expressly recited in Claim 21 of the '822 Patent. Id. Thus, Plaintiffs argue, construing "combining" as "multiplying" improperly limits the claims to an exemplary embodiment. Id. Plaintiffs further submit that the observer data does not included "polygon color value(s)," as suggested by Defendant, as the values in the observer data are 2D pixel values, not 3D polygon values. Id. Plaintiff also submits that "light accumulation buffer," as used in this term, refers to the data representing the accumulated light falling on a pixel rather than the buffer itself. Id. at 28-29.
In addition to the claims themselves, Plaintiffs cite the following intrinsic evidence to support their position: '822 Patent col.4 ll.35-37, col.4 ll.60-62, col.7 ll.4-6, col.7 ll.44-53, col.8 ll.57-60.
Defendant responds that the only method of "combining" described in the Asserted Patents is multiplying and therefore "combining" should be construed as multiplying. Dkt. No. 86 at 26-27. Defendant argues that construing "combining" to include other forms of combining, such as addition, would render the claims invalid for lack of the requisite written description support for or enablement of such combining. Id. at 27-28.
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '822 Patent col.9 ll.14-17, col.9 l.44, fig.4.
Plaintiffs reply that the patents state that in addition to the described embodiments, "[t]hose skilled in the art will recognize that other conventions and/or arrangements can also be used for storing and manipulating the data ," so it would be improper to limit "combining" to the exemplary multiplying. Dkt. No. 87 at 10 (quoting '822 Patent col.7 ll.1-3 (modification and emphasis by Plaintiffs)).
Plaintiffs cite further intrinsic evidence to support their position: '822 Patent col.7 ll.1-3.
Analysis
There are two issues raised by the dispute. First, whether "combining" is necessarily multiplying. Second, whether the observer data that is combined with the accumulated light data is necessarily the polygon color value(s) in the observer data. With respect to the first issue, "combining" is not limited to "multiplying." With respect to the second issue, "said observer data" is not limited to corresponding polygon color value(s) in the observer data."
While the patents describe an exemplary embodiment in which light accumulation data and observer data are combined by multiplying the data, the Court does not understand that combining these two pieces of data is limited to that embodiment. See Phillips v. AWH Corp., 415 F.3d 1303, 1323 (Fed. Cir. 2005) (en banc) ("we have expressly rejected the contention that if a patent describes only a single embodiment, the claims of the patent must be construed as being limited to that embodiment"); see also Thorner v. Sony Comput. Entm't Am. LLC, 669 F.3d 1362, 1366 (Fed. Cir. 2012) ("It is likewise not enough that the only embodiments, or all of the embodiments, contain a particular limitation. We do not read limitations from the specification into claims; we do not redefine words. Only the patentee can do that."); SRI Int'l v. Matsushita Elec. Corp., 775 F.2d 1107, 1121 (Fed. Cir. 1985) (en banc) ("The law does not require the impossible. Hence, it does not require that an applicant describe in his specification every conceivable and possible future embodiment of his invention."). In fact, the patents expressly describe other ways in which data may be combined. For example, light accumulation data may be combined with light image data by adding the two items of data. Id. at col.9 l.42 ("ACCUM (SPx, SPy) += LIGHT IMAGE (LPx, LPy)"). The Court discerns no reason to limit "combining" to multiplying.
Likewise, the Court discerns no reason to limit the "observer data" that is combined with the accumulated light image data to "corresponding polygon color value(s) in the observer data." To begin, the Court notes, and the parties agree, that "observer data" is not limited to color data. Indeed, as set forth above, Defendant argues that the observer data may include only depth data. While the Court understands that "observer data" necessarily includes data regarding the color of objects in the simulated scene, that does not mean that combining accumulated light image data with observer data necessarily involves combining accumulated light image data with the polygon color values. As with construing "combining" as "multiplying," construing this term to require combination of "corresponding polygon color value(s) in the observer data" would improperly narrow the term.
With respect to Defendant's claim-construction argument based on the purported lack of written description or enablement support for a "combining" that encompasses more than "multiplying," the Court declines to engage in such an invalidity analysis. As the en banc Federal Circuit stated in Phillips,
While we have acknowledged the maxim that claims should be construed to preserve their validity, we have not applied that principle broadly, and we have certainly not endorsed a regime in which validity analysis is a regular component of claim construction. Instead, we have limited the maxim to cases in which the court concludes, after applying all the available tools of claim construction, that the claim is still ambiguous.415 F.3d at 1327 (citations and quotation marks omitted). Here, as in Phillips, the requested validity analysis is inappropriate.
Accordingly, the Court construes the term as follows:
• "combining at least a portion of said light accumulation buffer with said observer data" means "combining at least a portion of the data in the light accumulation buffer with said observer data."
F. "displaying resulting image data to a computer screen"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "displaying resulting imagedata to a computer screen"• '822 Patent Claim 1 | causing the resulting imagedata to appear on a computerscreen | displaying the pixel dataresulting from combining theportion of the lightaccumulation buffer with theobserver data |
The Parties' Positions
Plaintiffs submit that this term requires that the image be displayed to a computer screen rather than on a computer screen, and therefore the claim "does not require the actual display on a computer screen." Dkt. No. 77 at 29-30.
In addition to the claims themselves, Plaintiffs cite the following intrinsic evidence to support their position: '822 Patent col.4 ll.9-19, col.7 ll.54-59.
Defendant responds that the meaning of "displaying" is well understood and does not need to be construed and that the "resulting image data" is the image data that results from combining at least a portion of the light accumulation buffer with the observer data. Dkt. No. 86 at 28. With respect to "displaying," Defendant contends that Plaintiffs' proposed construction was disclaimed in prosecution of the '488 Patent when the patentee argued that the "displaying" of the '822 Patent is different than "generating an output signal to a display" in that the latter does not involve "displaying or having a display as part of the system or method." Id. at 29 (quoting '488 Patent File Wrapper October 21, 2002 Amendment at 2, Dkt. No. 86-3 at 3 (modification marks omitted)). With respect to "resulting image data," Defendant contends that a plain reading of the claim shows that "resulting" refers back to the "combining" step and that the patents state the image data is comprised of pixels. Id.
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '488 Patent File Wrapper August 28, 2002 Office Action (Defendant's Ex.2, Dkt. No. 86-3 at 19-22), October 21, 2002 Amendment (Defendant's Ex. 2, Dkt. No. 86-3 at 2-15).
Plaintiffs reply that "displaying . . . to a computer screen" does not require displaying on a computer screen. Dkt. No. 87 at 10-11. Plaintiffs further reply that the statements from the prosecution history of the '488 Patent distinguish the '822 Patent's claims from the '488 Patent's claims on the ground that the '822 Patent's claim require display to a computer screen whereas the '448 Patent's claims do not require a computer screen. Id. at 11.
Plaintiffs cite further intrinsic evidence to support their position: '488 Patent File Wrapper October 21, 2002 Amendment (Defendant's Ex. 2, Dkt. No. 86-3 at 2-15).
Analysis
There are two issues raised by the dispute. First, whether "displaying . . . to a computer screen" requires displaying on a computer screen. Second, whether the "resulting image data" is the result of "combining at least a portion of said light accumulation buffer with said observer data." With respect to the first issue, the Court understands that the resulting image data is displayed on a computer screen. With respect to the second issue, the "resulting image data" is the result of the "combining" step.
The Court understands "displaying . . . to a computer screen" in light of the prosecution history to mean "displaying . . . on a computer screen." Specifically, the patentee explained this term in the course of prosecuting the '488 Patent as "displaying [the image] upon some type of computer screen." '488 Patent File Wrapper October 21, 2002 Amendment at 2-3 (Defendant's Ex. 2, Dkt. No. 86-3 at 3-4) (emphasis added). While the Court does not understand this to be a disclaimer, as Defendant argues, the Court understands the phrase the same way the patentee explained the phrase—displaying to a screen means displaying on a screen.
The Court understands "displaying" as used here to include that which is necessary to place the resulting image data in condition for presentation on a computer screen. For example, the patents describe that the frame buffer "includes RGB data of the final rendered 2D image of scene 10" and that "[t]he data in the frame buffer 50 is provided to output device 56 for display to the user." '822 Patent col.7 ll.54-61. The patents further explain that the resulting image "is further rendered in accord with other processes, as needed, and eventually stored in frame buffer 50 and displayed as a frame of data." Id. at col.9 ll.12-22. Thus, the Court understands that "displaying" encompasses the further rendering contemplated by the patents to place the resulting image data in condition for display on a computer screen.
Accordingly, construes the term as follows:
• "displaying the resulting image data to a computer screen" means "displaying the image data resulting from combining at least a portion of the light accumulation buffer with the observer data on a computer screen."
G. "outputting resulting image data"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "outputting resulting imagedata"• '488 Patent Claims 1, 27 | transmitting the resultingimage data from one locationto another | outputting for display thepixel values resulting fromcombining the portion of thelight accumulation bufferwith the observer data to bedisplayed |
The Parties' Positions
Plaintiffs submit that the Asserted Patents describe "that the output device may be a display, or may be some other device or method," such as a printer. Dkt. No. 77 at 31-32 (citing '822 Patent col.6 ll.7-15, col.6 ll.24-27; '488 Patent col.6 ll.26-29). Plaintiffs further submit that "outputting," as used in the art, means "transmitting or sending to." Id. at 31. Thus, Plaintiffs conclude, "outputting the resulting image data" encompasses sending the data to various locations, such as to computer screen or a removable disk. Id. Plaintiffs argue that Defendant's proposed construction is improper as it conflates "outputting" and "displaying" and effectively removes the distinction between Claim 1 of the '822 Patent—which requires "displaying"—and Claim 1 of the '488 Patent—which requires "outputting." Id.
In addition to the claims themselves, Plaintiffs cite the following intrinsic and extrinsic evidence to support their position: Intrinsic evidence: '822 Patent col.6 ll.7-15, col.6 ll.24-27, col.7 ll.58-59; '488 Patent col.6 ll.26-29. Extrinsic evidence: Ferraro Decl. ¶¶ 28-29 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 10); Microsoft Computer Dictionary (5th ed. 2002), "output" (Plaintiffs' Ex. 1, Dkt. No. 77-5).
Defendant responds that the patentee explained during the prosecution of the '488 Patent that the claims of that patent "relate to systems and methods which generate an output signal to a display." Dkt. No. 86 at 30 (quoting '488 Patent File Wrapper October 21, 2002 Amendment at 2, Dkt. No. 86-3 at 3). Defendant argues that its proposed construction gives effect to that explanation. Id. Defendant further responds that each output device listed in the patents is a device for displaying an image, and none are akin to a removable disk. Id.
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '822 Patent col.6 ll.24-27; '488 Patent File Wrapper October 21, 2002 Amendment (Defendant's Ex. 2, Dkt. No. 86-3 at 2-15).
Plaintiffs reply that Defendant's construction does not give the proper effect to the prosecution history of the '488 Patent, where the patentee explained the difference between the claims of the '822 Patent and the claims of the '488 Patent is
Thus, while there are many similarities between these claims, the present claims are more particularly focused on generating image data and then outputting it rather than both generating the image data and displaying it upon some type of computer screen ... Thus the claims of the [] present invention are more narrowly focused on systems and methods for generating and outputting a particular type of data than those of the claims of the '822 Patent.Dkt. No. 87 at 11 (quoting '488 Patent File Wrapper October 21, 2002 Amendment at 3, Dkt. No. 86-3 at 4 (first modification and emphasis added by Plaintiffs)).
Plaintiffs cite further intrinsic evidence to support their position: '488 Patent File Wrapper October 21, 2002 Amendment (Defendant's Ex. 2, Dkt. No. 86-3 at 2-15).
Analysis
There are two issues raised by the dispute. First, whether "outputting resulting image data" requires outputting that image "for display." Second, whether the "resulting image data" is the result of "combining at least a portion of said light accumulation buffer with said observer data." With respect to the first issue, the "outputting resulting image data" is not necessarily "for display." With respect to the second issue, the "resulting image data" is the result of the "combining" step.
The term "output" in the Asserted Patents is distinct from "display." For example, the patents provide that "[a]t least one output device 56 is connected to bus 46 and configured to display or otherwise output data from processor 42 to the user." '822 Patent col.6 ll.22-23; '488 Patent col.6 ll.24-26. That is, "output" encompasses "display" but is not limited to "display." The prosecution history does not evince a clear and unmistakable surrender of all "output" that is not "for display." While the prosecution history contains a statement that the '488 Patent claims "relate to systems and methods which generate an output signal to a display" it also contains statements that the "claims would be more narrowly focused ... on some type of device for generating graphics signals used within a computer system" and that the "claims are more particularly focused on generating image data then outputting it rather than both generating the image data and displaying it upon some type of computer screen." '488 Patent File Wrapper October 21, 2002 Amendment at 2-33, Dkt. No. 86-3 at 3-4. These statements, taken together in the context of the patents' disclosure, may reasonably be interpreted to mean that "outputting resulting image data" means that once the light accumulation data and observer data have been combined, the resulting image data is output following the recited shadow-rendering steps, ultimately for presentation to the user, but not necessarily immediately for display. Thus, the prosecution history does not rise to a clear and unmistakable disclaimer of scope other than "outputting for display." 3M Innovative Props. Co. v. Tredegar Corp., 725 F.3d 1315, 1326 (Fed. Cir. 2013) ("Where an applicant's statements are amenable to multiple reasonable interpretations, they cannot be deemed clear and unmistakable.").
Accordingly, the Court construes the term as follows:
• "outputting resulting image data" means "outputting for presentation to a user the image data resulting from combining at least a portion of the light accumulation buffer with the observer data."
H. "precalculated matrix transformation look-up table"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "precalculated matrixtransformation look-up table"• '822 Patent Claim 8• '488 Patent Claims 8, 34 | Plain and ordinary meaning• i.e. "a table containingcoordinates transformedfrom an observer'sperspective to a lightsource's perspective thatare calculated before theyare needed" | a table with entries pairingeach pixel in the observer'sperspective with a pixel in alight source view |
The Parties' Positions
Plaintiffs submit that a transformation table in the art is a mathematical construct that is used to transform data and in the Asserted Patents the transformation table is used to transform data from one coordinate system to another coordinate system. Dkt. No. 77 at 32. Plaintiffs further submit that a "precalculated" transformation table, in the context of the patents, is one that is calculated at least a frame in advance. Id. at 33. Plaintiffs argue that Defendant's proposed construction is improper because it gives no effect to "precalculated." Id. at 33-34.
In addition to the claims themselves, Plaintiffs cite the following intrinsic and extrinsic evidence to support their position: Intrinsic evidence: '822 Patent col.7 ll.62-66, col.8 ll.3-33. Extrinsic evidence: Ferraro Decl. ¶¶ 32-35 (Plaintiffs' Ex. C, Dkt. No. 77-3 at 11-12).
Defendant responds that the transformation tables of the Asserted Patents operate to transform 3D data. Dkt. No. 86 at 31. Defendant further responds that the tables are expressly "look-up" tables and thus must correlate pixel location in the observer view with pixel location in the light-source view. Id. at 31-32. And Defendant agrees with Plaintiffs that "precalculated," in the context of the patents, means at least a frame in advance. Id. at 32-33.
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '822 Patent col.7 l.62 - col.8 l.2, col.8 ll.9-11, col.8 ll.48-56, figs.3, 6.
Plaintiffs reply that the patents describe transformation tables that transform 2D data, and are therefore not restricted to transforming 3D data, as Defendant suggests. Dkt. No. 87 at 12.
Analysis
There is one issue raised by the dispute, namely, whether the transformation is necessarily a 3D to 2D transformation. The Court understands that the transformation is not necessarily a 3D to 2D transformation, but rather is expressly recited as the transformation of the observer data from the observer's coordinate system to the light source's coordinate system—which, in an exemplary embodiment, is a 2D to 2D transformation.
The Asserted Patents explain generally that converting from one coordinate system to another is a transformation. See, e.g., '822 Patent, at [57] Abstract ("Transformations between different perspective and/or coordinate systems can be precalculated for fixed cameras or light sources."), col.6 ll.53-56 ("Known matrix algebra techniques are used to convert or transform between world space coordinate system 20 and the unique coordinates from the perspectives of light source 16, light source 18, and camera 14."), col.7 ll.20-23 ("An exemplary pixel 60', which corresponds to pixel 60 in camera image 51A, is depicted following transformation as having light source #1 coordinates L1,Px and L1,Py."). And the patents specifically describe a transformation table that "is used, in accordance with certain embodiments of the invention to provide for rapid transformation of pixels between camera coordinates and light source #1 coordinates." Id. at col.7 ll.63-66. Thus, this table is used to convert from one 2D coordinate system (the observer's) to another 2D coordinate system (the light source's). The transformation table is explicitly described as converting from a 2D system to a 2D system. Thus, the "precalculated matrix transformation look-up table" should not be understood to necessarily involve transformation of 3D coordinates.
Further, the Court declines to read in Defendant's suggested "pairing" limitation. First, it is unclear what "pairing" entails. To the extent Defendant intends "pairing" to mean that the table has both observer coordinates and light-source coordinates, the Court disagrees. The patents describe an exemplary table that includes the same number of entries as there are points in the camera (observer) and light-source views. Id. at col.7 l.66 - col.8 l.2. A table with pairs of entries, one for observer coordinates and one for corresponding light-source coordinates, would have twice as many entries. In this particular example, the table entries are "preferably arranged to directly correspond[] to the sequential memory addresses of the pixel data values in camera image 51A." Id. Thus, the exemplary table allows a look-up of the light-source coordinates corresponding to the observer coordinates without "pairing."
Accordingly, the Court construes this term as follows:
• "precalculated matrix transformation look-up table" means "look-up table of lighting data coordinates that correspond to observer data coordinates where the lighting data coordinates are calculated at least a full frame in advance of the step of converting."
I. "selectively controlled source color data that can be changed over a period of time"
| Disputed Term | Plaintiffs' ProposedConstruction | Defendant's ProposedConstruction |
|---|---|---|
| "selectively controlled sourcecolor data that can bechanged over a period oftime"• '822 Patent Claim 9• '488 Patent Claims 9, 35 | a portion of the source colordata that can be changed tosimulate dynamicallychanging light | multiple light imagesreflecting the change in lightemitted by a source over time |
The Parties' Positions
Plaintiffs submit that the Asserted Patents describe changing light-source data, such as "source color data," to simulate light sources that change over time, such as light reflected from rippling water. Dkt. No. 77 at 34. Plaintiffs argue that Defendant's proposed construction improperly reads out "data" from the term and improperly requires "multiple" images whereas the patents describe that a single image "can include a dynamically changing graphical image." Id. at 35 (quoting '822 Patent col.10 l.63 - col.11 l.7).
In addition to the claims themselves, Plaintiffs cite the following intrinsic evidence to support their position: '822 Patent col.3 l.64 - col.4 l.5, col.10 l.63 - col.11 l.7.
Defendant responds that the plain meaning of "selectively controlled" requires actual control, and not just capability. Dkt. No. 86 at 33. Defendant further responds that all data in memory may be changed thus Plaintiffs' proposed construction provides no meaningful distinction between the claims in which this term appears and the claims from which those claims depend. Id. And Defendant responds that because this term appears in method claims that must be performed to be infringed, interpreting this term as mere capability leaves it unclear whether the claims are infringed if the source data can be changed, but is not. Id. at 33-34. This lack of clarity would render the claims indefinite. Id. at 34 (citing IPXL Holdings, LLC v. Amazon.com, Inc., 430 F.3d 1377, 1384 (Fed. Cir. 2005)).
In addition to the claims themselves, Defendant cites the following intrinsic evidence to support its position: '822 Patent col.10 l.66 - col.11 l.13.
Plaintiffs reply that the patents describe that "a light image can include a dynamically changing graphical image" and therefore it would be improper to require multiple light images. Dkt. No. 87 at 13. Plaintiffs further reply that the claim language is expressly directed to capability, in that the selectively controlled source color data can be changed and therefore it would be improper to require actual control. Id.
Analysis
There are two issues raised by the dispute. First, whether the source color data must change over time. Second, whether changing source color data necessarily requires multiple light images. With respect to the first issue, the source color data need not change over time, so long as it can change and is controlled to change or not change. With respect to the second issue, the change in source color data need not be represented as multiple light images.
This term is best understood in the context of the surrounding claim language. For example, Claim 9 of the '822 Patent, reproduced below and annotated by the Court, provides that the term is reflecting a characteristic of the source color data that is manipulated in the claimed method. The term is not a separate "changing" step in the process. Thus, this claim language means that the shadow-rendering method provides lighting data that includes source color data that can be selected and controlled in order to change it over a period of time. And that period of time is set forth in the claims. This recited characteristic of lighting data does not raise an indefiniteness issue as dealt with in IPXL Holdings, LLC v. Amazon.com, Inc., 430 F.3d 1377 (Fed. Cir. 2005). There, the Federal Circuit held that an apparatus claim that included a limitation requiring the user to actually use the apparatus was indefinite for mixing method and apparatus claims. Id. at 1384. Here, in contrast, the claims simply recite characteristics of the data that is manipulated according to the claimed method. That is, the data that is provided per the "providing lighting data" must include changeable source color data.
The patents further explain that changing the lighting data does not require multiple images. Such a change may be effected, for example, by including a "dynamically changing graphical image" in a light image. '822 Patent col.10 l.66 - col.11 l.1.
'822 Patent
1. A shadow rendering method for use in a computer system, the method comprising the steps of:
providing observer data of a simulated multi-dimensional scene;
providing lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data;
for each of said plurality of light sources, comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in a light accumulation buffer; and then
combining at least a portion of said light accumulation buffer with said observer data; and
displaying resulting image data to a computer screen.
2. The method as recited in claim 1, wherein said observer data includes observed color data and observed depth data associated with a plurality of modeled polygons within said scene as rendered from an observer's perspective.
4. The method as recited in claim 2, wherein said lighting data includes source color data associated with at least one of said light sources and source depth data associated with said plurality of modeled polygons within said scene as rendered from a plurality of different light source's perspectives,
9. The method as recited in claim 4, wherein at least a portion of said source color data is selectively controlled source color data that can be changed over a period of time during which at least the step of displaying resulting image data to said computer screen is repeated a plurality of times.
Accordingly, the Court construes this term as follows:
• "selectively controlled source color data that can be changed over a period of time" means "source color data that the computer can
change over a period of time."
VI. CONCLUSION
The Court adopts the above constructions set forth in this opinion for the disputed and agreed terms of the Asserted Patents. The parties are ordered that they may not refer, directly or indirectly, to each other's claim construction positions in the presence of the jury. Likewise, the parties are ordered to refrain from mentioning any portion of this opinion, other than the actual definitions adopted by the Court, in the presence of the jury. Any reference to claim construction proceedings is limited to informing the jury of the definitions adopted by the Court.
SIGNED this 27th day of September, 2016.
/s/_________
ROY S. PAYNE
UNITED STATES MAGISTRATE JUDGE