Opinion
CIVIL ACTION NO. 10-11597-RGS
11-05-2012
EVERYSCAPE, INC. v. ADOBE SYSTEMS, INC.
MEMORANDUM AND ORDER
ON CLAIM CONSTRUCTION
STEARNS, D.J.
In this intellectual property dispute, plaintiff EveryScape, Inc., alleges that defendant Adobe Systems, Inc., infringes United States Patents Nos. 7,327,374 (the '374 patent) and 7,593,022 (the '022 patent). Adobe, by way of a counterclaim, alleges that EveryScape infringes United States Patents Nos. 6,411,742 (the '742 patent) and 7,095,905 (the '905 patent). Before the court are the parties' briefs on claim construction. The court heard argument, pursuant to Markman v. Westview Instruments, Inc., 517 U.S. 370 (1996), on October 31, 2012.
BACKGROUND
EveryScape Patents
The clone brush, also known as the clone stamp, is "[o]ne of the most powerful and widely-used tools in photo editing." '374 patent, col. 1, ll. 18-20. The clone brush "cop[ies] and past[es] from one region of an image to another via a brush interface. It is often used to remove undesirable portions of an image." Id. col. 1, ll. 20-22. Numerous image editing software packages, including Adobe Photoshop, offer a clone brush feature. Id. col 1, ll. 55-58.
Because the '374 and '022 patents share the same specification, only citations to the '374 patent are included in this opinion.
The '374 and '022 patents are both entitled "Structure-Preserving Clone Brush," and list Byong Mok Oh and Fredo Durand as the co-inventors. The two patents are directed to "tools and techniques for clone brushing pixels in an image while accounting for inconsistencies in apparent depth and orientation within the image." '374 patent, abstract. The patents improve on the traditional clone brush by making adjustments for perspective foreshortening. Id. col. 2, ll. 9-14. Perspective foreshortening is the phenomenon that causes portions of an image closer to the viewer's eye to appear larger than portions further away. A traditional clone brush, when copying a portion of an image more distant from the eye to a position closer (or vice versa) produces a result in which the copied portion appears disproportionate in size to its new surroundings. See id. col. 4, ll. 58-61 & Fig. 2b.
The '374 and '022 patents solve this problem by mapping both the source and destination regions of the clone operation from the image plane to an intermediate representation (defined as the "world plane"), using a homography that is determined by the perspective of the image. See id. col. 5, ll. 1-6. The world plane is a "perspective correcting" representation of the source (and destination) region of the image. See id. col. 5, ll. 8-9 & 44-46. The desired image manipulation (translation of pixels from the source region to the destination region) takes place in the world plane, and then the result is mapped, using an inverse homography, to the selected destination portion of the image. Id. col. 5, ll. 11-18.
Two different homographies are used where the source and destination portions of the image have different perspectives. See id. col. 5, ll. 19-31.
The '374 patent was issued on February 5, 2008, and has 19 claims. The '022 patent was issued on September 22, 2009, and has 32 claims. Claim 1 of the '374 patent is representative:
1. A clone-brushing method of painting in a 2D image, the method comprising:(emphasis added). The parties dispute the construction of two claim terms - "homography" and "source position."
a) specifying a first world plane in the 2D image;
b) providing a source position and a destination position in the 2D image;
c) identifying a destination region in the 2D image relative to the destination position;
d) determining a source region in the 2D image corresponding to the destination region in the 2D image including:
defining a transformation that maps the destination position relative to the first world plane to the source position relative to the first world plane using a homography defined by the first world plane,e) transforming 2D image information of the source region relative to the first world plane to 2D image information of the destination region; and
and identifying pixels in the source region of the 2D image corresponding to pixels in the destination region of the 2D image using the transformation and the homography;
f) painting in the 2D image by copying the transformed 2D image information to the destination region.
ADOBE PATENTS
A camera has a limited field of view. See '742 patent. col. 1, ll. 15-18; '905 patent, col. 1, ll. 14-17. "It is sometimes necessary to capture an image of a view that is larger than can be captured within the field of view of a camera. Multiple overlapping images of segments of the view are taken and then the images are joined together, or 'merged,' to form a composite image, known as a panoramic image." '742 patent, col. 1, ll. 18-23. See also '905 patent, col. 1, ll. 17-22.
The '742 and '905 patents are both entitled "Merging Images to Form a Panoramic Image" and list John Peterson as the inventor. Both patents are directed to methods for blending and/or merging images of segments of a view to form a panoramic image. '742 patent, abstract; '905 patent, abstract. The patents disclose algorithms that first determine the positions of images relative to each other, and then "seamlessly" merge the pairs of overlapping of images. '742 patent, col. 1, ll 34-45; id. col. 2, ll. 40-43; '905 patent, col. 1, ll. 33-42; id. col. 2, l. 66 - col. 3, l. 1.
The '742 patent was issued on June 25, 2002, and has 18 claims. Claim 1 is representative:
1. A method for merging images, comprising:(emphasis added.) The parties dispute the construction of three claim terms - "each image," "all other images," and "grouping the images into pairs, wherein an image is grouped into a pair with each image identified as overlapping the image." The parties stipulate that the claim term "position of the images relative to each other" should be construed as "for each image, position of the image relative to another image." The parties further stipulate that the claim term "transition band" should be construed as "band at the changeover between two images in a pair."
receiving images, each image representing a segment of a view;
receiving position information specifying positions of the images relative to each other;
for each image and based on the position information, identifying all other images that overlap the image;
grouping the images into pairs, wherein an image is grouped into a pair with each image identified as overlapping the image;
defining a transition band for each pair of images;
for each image in a pair of images, assigning a masking value for each pixel of the image, wherein the masking values specify complete visibility for each pixel in an area of the image that does not overlap the other image of the pair, the masking values
specify partial visibility for pixels in the transition band, and the masking values specify complete invisibility for the remainder of the pixels in the image; and
merging the images using the calculated masking values.
The parties also stipulate that "each image" in this stipulated construction should have the same meaning as the court's construction of the term.
The court finds that these stipulated constructions are consistent with the '742 patent's use of these terms and adopts these constructions.
The '905 patent was issued on August 22, 2006, and has 27 claims. Claim 1 is representative:
1. A method of merging images of segments of a view, comprising:(emphasis added). The parties the dispute the construction of two claim terms used in the '905 patent - "contents of the images," and "mask out part of the first image."
receiving a first image representing a first segment of the view and a second image representing a second segment of the view;
determining the position of the second segment of the view relative to the first segment of the view without the aid of positioning information provided by a human operator;
blending the first image with the second image based solely on the content of the images and the determined position of the second segment relative to the first segment to merge the first image and the second image into a panoramic image of the view, wherein the blending comprises:
dividing the second image into a first portion and a second portion based on the position of the second segment relative to the first segment; and
compositing the first portion of the second image on the first image at a relative position of the second segment relative to the first segment to produce the panoramic image, the compositing of the first portion of the second image causing the first portion to mask out a part of the first image.
DISCUSSION
Claim construction is a question of law for the determination of the court. See Markman v. Westview Instruments, Inc., 517 U.S. 370, 388-389 (1996). Claim terms are generally given the ordinary and customary meaning that would have been assigned to the terms by a person of ordinary skill in the art in question at the time of the invention. Phillips v. AWH Corp., 415 F.3d 1303, 1312-1313 (Fed. Cir. 2005) (en banc) (citations omitted). "[T]he person of ordinary skill in the art is deemed to read the claim term . . . in the context of the entire patent, including the specification." Id. at 1313.
The patent specification "'is always highly relevant to the claim construction analysis. Usually it is dispositive; it is the single best guide to the meaning of a disputed term.'" Id. at 1315, quoting Vitronics Corp. v. Conceptronic, Inc., 90 F.3d 1576, 1582 (Fed. Cir. 1996). Because the purpose of the specification is to "teach and enable those of skill in the art to make and use the invention and to provide the best mode for doing so," Phillips, 415 F.3d at 1323, it is "entirely appropriate for a court, when conducting claim construction, to rely heavily on the written description for guidance as to the meaning of the claims." Id. at 1317.
In addition to the patent's specification, the prosecution history "can often inform the meaning of the claim language by demonstrating how the inventor understood the invention and whether the inventor limited the invention in the course of prosecution, making the claim scope narrower than it would otherwise be." Id. (citations omitted). Although not as reliable as the patent and its prosecution history, the court is also authorized to consider extrinsic evidence "if the court deems it helpful in determining the true meaning of language used in the patent claims." Id. at 1318. Ultimately, "[t]he construction that stays true to the claim language and most naturally aligns with the patent's description of the invention [in the specification] will be, in the end, the correct construction." Id. at 1316 (citation omitted).
1. '374 and '022 patents - "homography"
EveryScape contends that "homography" should be construed as "a linear mapping between a world plane and an image plane." EveryScape relies on a passage in the specification of the '374 and '220 patents that it argues defines "homography":
The traditional clone brush, as noted above, is based on relative translation in image space. In the case of foreshortened geometry, it is desirable for the translation to take place in world space, as shown in FIG. 3. In an embodiment of the invention, this is accomplished by defining the world plane using a homography - a linear 2D-to-2D mapping between the world and image planes. Once the homography is determined, points on the image plane can be mapped to the world plane, and vice versa.'374 patent, col. 4, l. 64 - col. 5, l. 6 (emphasis added).
Adobe, for its turn, contends that "homography" should be construed as "a 2D projective transformation," and relies on a different passage in the specification: "As illustrated in FIG. 5, points on the world plane are projections of points on the image plane, and this linear mapping between the two planes is defined through a homography, i.e. a 2D projective transformation." Id. col. 5, ll. 50-53 (emphasis added). Adobe argues that the use of "i.e.," "signals an intent to define the word to which it refers." Edward Lifesciences LLC v. Cook Inc., 582 F.3d 1322, 1334 (Fed. Cir. 2009). Adobe further points out that when distinguishing prior art, the patentee defined "homography" as "a 2D projective mapping" during the prosecution of the '374 patent. See Oct. 2, 2006 Resp. to Office Action at 17. Finally, Adobe argues that EveryScape's proposed construction is mathematically incorrect because linear transformation cannot achieve the perspective correction required by the patents. Adobe Br. at 11-13.
Adobe also argues that EveryScape's proposed construction describes only a specific embodiment.
EveryScape counters that the "em dash" in its preferred specification passage is also indicative of definitional language. Moreover, the use of "i.e." does not necessarily preface a definition where there is "support in the intrinsic evidence for a construction of the disputed claim other than the construction linked to 'i.e.'." Pfizer, Inc. v. Teva Pharms., USA, Inc., 429 F.3d 1364, 1373 (Fed. Cir. 2005) (citation omitted). EveryScape argues that its proposed construction of "homography" is consistent with the specification's use of the term. It points out that Adobe's preferred specification passage also describes a "homography" as a "linear mapping between the two planes." '374 patent, col. 5, ll. 51-52. Furthermore, the provisional patent application from which the '374 and '022 patents claim priority states that "a homography determines a linear 2D-to-2D mapping between the image and world planes, where the world plane is projected onto the imagine plane with respect to perspective camera o." U.S. Provisional Patent Appl. No. 60/466,628 (EveryScape Prov. Appl.) at 79. Finally, EveryScape argues that the use of the word "linear" is mathematically accurate because a projective transformation is a non-singular linear transformation. EveryScape Reply Br. at 13.
This dispute is ultimately about what a "homography" is and, in turn, what a "homography" does. Nothing in the intrinsic record suggests that "homography" is being used in these patents differently from its customary meaning in the field of computer graphics, which in turn relies on algebraic geometry. This is most evident in the provisional patent application, which appears to be co-inventor Dr. Oh's Ph.D. thesis and presentation. Citing to Semple and Kneebone's classic 1952 text on algebraic geometry, Dr. Oh's thesis defines "homography" as "a 2D projective transformation." EveryScape Prov. Appl. at 78. Thus, consistent with the specification, a "homography" is customarily understood as "a 2D projective transformation."
What a "homography" does is to "determine[] a linear 2D-to-2D mapping between the image and world planes." EveryScape Prov. Appl. at 79. See also '374 patent, col. 5, ll. 51-53 (a "homography" "define[s]" a "linear mapping between the two planes"). As explained in the specification, such a linear mapping means "points on the image plane can be mapped to the world plane, and vice versa." Id. col. 5, ll. 5-6. See also id. col. 5, ll. 55-59 (points on the image plane are mapped to the points on the world plane using a homography, and points from the world plane are mapped back to the image plane using an inverse homography). Because the image manipulation (pixel translation) occurs on the world plane before being inversely mapped back to the image plane, see id. col. 5, ll. 11-18, it makes intuitive sense that there must be a strict correspondence between points on the image plane and points on the world plane for the clone brush to work.
Adobe is correct that a "homography" is not a linear transformation with respect to two-dimensional coordinates. Because the goal of the patents is to use a "homography" to correct perspective foreshortening, see '374 patent col. 5, ll. 8-9 & 44-46, such a transformation does not generally preserve parallel lines. See id. col. 6, 28-29 ("Parallel lines on the world plane intersect at a vanishing point on the image plain."). Co-inventor Professor Durand agrees with Adobe that, mathematically, a transformation that does not preserve parallel lines is not a linear transformation with respect to two-dimensional coordinates. Durand Depo., 208:13-23. (Adobe Claim Construction Br. Ex. 6). EveryScape is also correct that a projective transformation, such as a "homography," may be linear with respect to other reference points. See EveryScape Reply Br. at 13. However, the specification is clear about what a linear mapping is in the context of the patents, and the court will not adopt a needlessly complex concept of linearity.
Despite the parties' adverse positions, what a "homography" is enables what a "homography" does. As Figure 4-5 of the provisional application makes clear, a "homography" is a transformation from an image plane to a world plane from the perspective of a camera. See EveryScape Prov. Appl. at 79; EveryScape Reply Br. at 13. A point on the image plane is transformed to a point in the world plane by projecting - drawing an imaginary straight line from the camera, through the image plane at the point, to the world plane. See id. See also '374 patent, col. 5, ll. 50-51 ("[P]oints on the world plane are projections of points on the image plane."). As a result of the projection, each point on the image plane corresponds to a unique point on the world plane, and vice versa, resulting in the "linear mapping" between the two planes. See '374 patent, col. 5, ll. 1-6. Because the projective transformation and the resultant linear mapping are both integral aspects of a "homography," the appropriate construction should incorporate both characteristics. Therefore, a "homography" is construed as "a 2D projective transformation that determines a linear mapping between an image and a world plane."
At the Markman hearing, Adobe argued that because claim 1 of the '374 patent states that "a homography [is] defined by the first world plane," a construction that references an "image plane" - a term not found in the claim - improperly imports a limitation. However, claim 1 also states that a first world plane is specified in the image, and thus a world plane does not exist independent of the image. Adobe also suggested that a homography may be performed between two world planes. However, because a world plane is a perspective-correcting representation of the image, it would be unnecessary to perform a homography to correct perspective between two world planes.
2. '374 and '022 patents - "source position"
EveryScape contends that "source position" should be construed as "a position that is selected by a user independently of a destination position." EveryScape argues that its proposed construction reflects how the clone brush tool is traditionally used - the user selects a source position, and independently selects a destination position. See id. col. 4, ll. 47-58. EveryScape also argues that this construction reflects how the claimed clone brush is used in the specification. Figures 8 and 10 both indicate the selection of a source position as a separate step from the selection of a destination position. See id. col. 8, ll. 45-47 & col. 9, ll. 36-38. The specification also indicates that the source position is selected by a user. See id. col. 7, ll. 47-50 & col. 9, ll. 47-50.
EveryScape further maintains that its proposed construction is consistent with the asserted claims and prosecution history. The claimed invention uses the "source position" and destination position to determine a translation relationship, and then uses that relationship to calculate the source region (from which image information is copied) relative to the selected destination region. See, e.g., id. claim 1 ("[D]etermining a source region in the 2D image corresponding to the destination region in the 2D image including: defining a transformation that maps the destination position relative to the first world plane to the source position relative to the first world plane using a homography defined by the first world plane, and identifying pixels in the source region of the 2D image corresponding to pixels in the destination region of the 2D image using the transformation and the homography."); Aug. 9, 2006 Resp. to Office Action at 2 (adding the claim limitation "wherein the source region in the 2D image is determined by a transformation that maps the destination position to the source position . . ."). Because this translation relationship necessarily requires the "source position" to be different from the destination position, the "source position" must be selected independently from the destination position.
Adobe also contends that "source position" does not need to be construed and should be given its ordinary meaning because the patentee did not given it any special meaning in the intrinsic record and it is not a technical term. See Phillips, 415 F.3d at 1312-1313. Adobe objects to EveryScape's proposed construction because it improperly imports limitations from the specification into the claim term. See id. at 1323. The asserted claims do not require either a user's selection of a "source position," or that the "source position" be independently selected from the destination position. In fact, claim 1 of the '374 patent suggests that the source and destination positions may be selected together by requiring "providing a source position and a destination position in the 2D image." (emphasis added).
Adobe further argues that the dependent claims indicate that the "source position" is not necessarily the result of a user interaction, or selected separately from the destination. Claim 17 of the '374 patent discloses a system that "interact[s] with a user to provide a source position . . . ." Thus, claim 1, which does not claim any user interaction, should not have a user limitation added. See Phillips, 415 F.3d at 1314 (limitations elsewhere claimed should not be added where not claimed). Similarly, claim 12 of the '374 patent claims "snapped destination position," which is determined by a computer with reference in part to the "source position." See '374 patent, col. 10, ll. 39-64. Therefore, the "source position" and destination position are not necessarily independent.
Adobe argues additionally that the prosecution history cited by EveryScape does not in fact support its proposed construction because the added limitation did not survive to the issued patent. The limitation at best suggests that the source position is different from the destination position, but it says nothing about a user independently selecting one and then the other. Adobe finally points to the prosecution history where EveryScape appeared to acquiesce to the patent examiner's interpretation that the claims covered prior art where the source and destination positions are both automated. See Dec. 15, 2006 Office Action at 2-3; Apr. 5, 2006 Office Action at 6.
The parties agree that the source position must be different from the destination position for the clone brush operation to be of any use.
In the alternative, Adobe proposes that "source position" should be construed as "a position from which image information is copied," as its ordinary meaning conveys. EveryScape objects to this proposal, arguing that although the source position (and the destination position) supplies the translation that is used to perform the clone brush, the claims do not require any actual copying of image information from the source position (as opposed to the source region) to the destination position.
Although EveryScape's objection to Adobe's proposed construction is valid,its proposed construction also suffers from several defects. Of greatest significance, EveryScape's proposal does not capture the "source" aspect of a "source position." The "source position" is the "source" because the source region from which image information is copied is determined relative to the source position using the translation relationship between the source and destination positions. See '374 patent, col. 4, ll. 51-56 (role of "source position" in traditional clone brush); & col. 5, ll. 11-18 & Fig. 3 (same role of "source position" in claimed invention). In addition, the plain language of claim 1 of the '374 claim does not require that a "source position" be selected by a user, or be selected apart from the destination position. Although a user's independent selection may be the predominant way to specify a source position, as Adobe argues, the other claims of the '374 patent (as well as prior art) suggest other possible modes of selection. Therefore, these limitations are not necessary to a "source position" and should not be imported into the its meaning. Rather, Adobe's alternative proposal, "a position relative to which image information is copied," most aptly captures the role of a "source position" in the invention. Therefore, to resolve the dispute over claim scope, see O2 Micro Intern. Ltd. v. Beyond Innovation Tech. Co., Ltd., 521 F.3d 1351, 1362 (Fed. Cir. 2008), the court will construe "source position" as "a position different from the destination position, relative to which image information is copied."
Adobe's proposal is also overly broad - not every "position from which image information is copied" is a "source position," as the source region may contain many "positions from which image information is copied."
3. '742 patent - "each image," "all other images," and "grouping the images into pairs, wherein an image is grouped into a pair with each image identified as overlapping the image"
Adobe contends that the three "image" terms should be given their ordinary meaning because they are plain English words, not terms of art, and the patentee did not give them a special meaning. EveryScape does not disagree, but argues that the meaning of these terms should be clarified to reflect that the "images" refer to "received images" consistent with claim 1 of the '742 patent, which specifies as a first step, the "receiving [of] images, each image representing a segment of a view." Thus, subsequent references to images logically refer to a received image. EveryScape proposes that "each image" should be construed as "each received image"; "all other images" should be construed as "all other received images relating to the view"; and "grouping the images into pairs, wherein an image is grouped into a pair with each image identified as overlapping the image" should be construed as "for each received image, grouping the received image into a pair with each other received image that overlaps it."
Adobe does not dispute that the image terms refer to their antecedent basis, Adobe Reply Br. at 18, but contends that adding "received" alters the meaning of the "all other images" and "grouping . . . .". Adobe argues that EveryScape's insertion of "received" into these terms "could suggest that all images received by the system must be processed into a single panorama, irrespective of user intent." Id. (emphasis original). The invention is not so limited, and "could operate on multiple panoramic images in parallel, including potentially multiple panoramic images of the same view." Id.
At the hearing, Adobe supported its argument by using as an example multiple sets of images of the same view at different exposures. Adobe is correct that claim 1 of the '742 patent is not restricted to creating one panorama. However, merging multiple panoramas is neither facilitated or limited by the "image" terms. Presumably, the "position information specifying positions of the images relative to each other" will specify when images belong to multiple panoramas. The "identifying" step will only identify overlapping pairs of images based on the position information.
The court agrees with Adobe that these three "image" claim terms are common English terms and have not been given special meanings in the patents. The court will therefore construe "each image," "all other images," and "grouping the images into pairs, wherein an image is grouped into a pair with each image identified as overlapping the image" according to their ordinary meaning.
4. '905 patent - "contents of the images"
Adobe contends that this term should be construed according to its ordinary meaning. EveryScape does not disagree, and proposes that the ordinary meaning is "the pixels of the first and second images" because the specification describes blending two images together based on their pixel content. See '905 patent, col. 7, ll. 5-12.Because the meaning "contents of the images" is plain, the court will construe "contents of the images" according to its ordinary meaning.
Adobe objects to EveryScape's construction because the contents of an image consist of more than the sum of its pixels. Abode also objects to EveryScape's proposal because it is unduly limiting. Claim 1 of the '905 patent recites "blending the first image with the second image based solely on the contents of the images . . . ." Adobe argues that EveryScape's proposed construction excludes a blending process based in part on a third overlapping image and also excludes other types of image representations besides pixels. Adobe is correct that the '905 patent does disclose blending of two images based in part on a third image. See' 905 patent, col. 2, ll. 1-16; id. claim 7.
5. '905 patent - "mask out part of the first image"
Adobe contends that this term should be construed according to its ordinary meaning. EveryScape proposes to clarify the meaning of "mask out" based on the specification. The specification of the '905 patent describes assigning a mask value of 0 to those pixels in the first image that are "masked out" by portions of the second image. See '905 patent, col. 7, ll. 8-12. Consequently, EveryScape proposes to construe "mask out part of the first image" as "set the mask value of the pixels in the part of the first image corresponding to the first portion of the second image to be '0'." Adobe objects to EveryScape's proposed construction because it imports limitations from a particular embodiment into the claim. See Intervet Am., Inc. v. Kee-Vet Labs., Inc., 887 F.2d 1050, 1053 (Fed. Cir. 1989) ("[I]nterpreting what is meant by a word in a claim 'is not to be confused with adding an extraneous limitation appearing in the specification, which is improper.'") (emphasis in original, citation omitted).
Adobe is correct that setting the mask value of a pixel is only a particular way of "masking out" a portion of an image. The specification uses the term "mask out" more broadly to describe a portion of one image obstructing an overlapping portion of another image. "Although all the pixels of the first image are set visible, some of the pixels of the first image may be obstructed or masked out by visible portions of subsequent images . . . ." '905 patent, col. 6, ll. 41-44 (emphasis added). Indeed, the purpose of setting the mask value of pixels in a second image is to obstruct the overlapping portion of a first image. "The section of the second image with a mask value of 1 is first composited on the first image, thereby obstructing the part of the first image that is to the right of the dividing line." Id. col. 7, ll. 25-28 (emphasis added). Adobe's propose alternative construction - "blank out or inhibit part of the first image" - more accurately captures the concept of "masking out." However, it is not necessary to resort to external definitions as the specification has provided an internal definition of the term. The court will construe "mask out part of the first image" to mean "obstruct part of the first image."
Adobe's position at the Markman hearing, that "masking out" also encompasses a blending mask, appears to be inconsistent with its proposed construction of "blank out," which suggests a total obstruction. However, it is unnecessary to resolve this issue (if it is one) in light of the court's construction.
--------
ORDER
The claim terms at issue will be construed for the jury and any other purpose in this litigation in a manner consistent with the above rulings of the court.
SO ORDERED.
Richard G. Stearns
UNITED STATES DISTRICT JUDGE