Google Inc.v.Grandeye Ltd.

Patent Trial and Appeal Board

Nov 25, 2014

09228760 (P.T.A.B. Nov. 25, 2014)

Trials@uspto.gov Paper 34
571-272-7822 Entered: November 25, 2014

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

GOOGLE INC.,
Petitioner,

v.

GRANDEYE LTD.,
Patent Owner.
____________

Case IPR2013-00547
Patent 6,243,099 B1
____________

Before JAMESON LEE, DAVID C. McKONE, and
PATRICK M. BOUCHER, Administrative Patent Judges.

BOUCHER, Administrative Patent Judge.

FINAL WRITTEN DECISION
35 U.S.C. § 318(a) and 37 C.F.R. § 42.73

IPR2013-00547
Patent 6,243,099 B1

2
I. INTRODUCTION
A. Background
On August 30, 2013, Google Inc. (“Petitioner”) filed a Petition
(Paper 1, “Pet.”) to institute an inter partes review of claims 1, 3–5, 11, and
13 of U.S. Patent No. 6,243,099 B1 (Ex. 1001, “the ’099 patent”) pursuant
to 35 U.S.C. § 311–319. Grandeye Ltd. (“Patent Owner”) filed a
Preliminary Response (Paper 15, “Prelim. Resp.”) to the Petition on
November 7, 2013. Pursuant to 35 U.S.C. § 314, the Board instituted trial
on February 5, 2014, as to claims 1, 3–5, 11, and 13. Paper 16 (“Dec.”).
During the trial, Patent Owner timely filed a Patent Owner Response
(Paper 22, “PO Resp.”), and Petitioner timely filed a Reply to the Patent
Owner Response (Paper 25, “Reply”). An oral hearing was held on
September 5, 2014. Paper 33 (“Tr.”).
We have jurisdiction under 35 U.S.C. § 6(c). This Decision is a final
written decision under 35 U.S.C. § 318(a) as to the patentability of the
challenged claims. Based on the record before us, Petitioner has
demonstrated by a preponderance of the evidence that claims 1, 3–5, 11, and
13 are unpatentable.

B. Related Proceedings
In addition to this proceeding, Petitioner filed petitions for inter partes
review of, and we instituted trial on, certain claims of U.S. Pat. Nos.
7,542,035 B1 (Ex. 1003, “the ’035 patent”) and 8,077,176 (Ex. 1002, “the
IPR2013-00547
Patent 6,243,099 B1

3
’176 patent”) in IPR2013-00548 and IPR2013-00546, respectively. Paper 6,
3. The ’176 patent is a continuation of the ’035 patent, which is a
continuation of the ’099 patent. Ex. 1002 at [63]; Ex. 1003 at [63].
View 360 Solutions LLC (“View 360”), a purported licensee of the
’035, ’099, and ’176 patents, has sued Petitioner for infringement of each of
those patents in View 360 Solutions LLC v. Google, Inc., Case No. 1:12-cv-
1352 (N.D.N.Y.). Pet. 1–2; Paper 6, 3. Patent Owner has asserted the ’035
and ’099 patents against others in Grandeye Ltd. v. Sentry 360 Security, Inc.,
Case No. 1:11-cv-02188 (N.D. Ill.).

C. Grounds of Unpatentability
Petitioner relies on the following prior-art references.

Photo VR Wen-kae Tsao et al., Photo VR: A System of Rendering
High Quality Images for Virtual Environments Using
Sphere-like Polyhedral Environment Maps, THE SECOND
WORKSHOP ON REAL-TIME AND MEDIA SYSTEMS (RAMS
’96) 397–403 (July 30–31, 1996) (Ex. 1007);

QuickTime® VR Shenchang Eric Chen, QuickTime
®
VR — An Image-
Based Approach to Virtual Environment Navigation
(ACM, 1995) (Ex. 1012).
We instituted this proceeding based on the asserted grounds that
claims 1, 3, 11, and 13 are anticipated under 35 U.S.C. §§ 102(a) and 102(b)
by Photo VR; and that claims 4 and 5 would have been obvious under 35
U.S.C. § 103(a) over Photo VR and QuickTime
®
VR. Dec. 30.
IPR2013-00547
Patent 6,243,099 B1

4
D. The ’099 Patent
The ’099 patent “relates generally to a method and corresponding
apparatus for viewing images.” Ex. 1001, col. 1, ll. 12–13. For instance, a
virtual pictosphere may be created using a conventional three-dimensional
graphics system that results from “texture mapping” the visible world onto a
sphere. Id. at col. 4, ll. 58–61. Different viewpoints enable different types
of perspective views when rendered with the primitives of a conventional
three-dimensional graphics system. For example, a linear perspective view
is achieved with a viewpoint at the center of the sphere, while a circular
perspective view is achieved with a viewpoint on the surface of the sphere
with a view direction towards the center. Id. at col. 4, l. 61 – col. 5, l. 3.
Figures 5 and 6 of the ’099 patent, reproduced below, are illustrative:

Figures 5 and 6 show projections of a portion of the visible world onto a
plane with a linear perspective view and a circular perspective view,
respectively.
In an illustrative example, the ’099 patent describes the mapping of
two fisheye images to adjoining hemispheres to generate spherical image
data. Id. at col. 8, ll. 39–55. A user interactively may move the viewpoint to
IPR2013-00547
Patent 6,243,099 B1

5
different positions that include the center of the sphere and to points very
near the inside of the sphere, thereby achieving the different perspective
views. Id. at col. 8, ll. 47–52. The surface of the sphere also may be rotated
to simulate looking around within the sphere. Id. at col. 8, ll. 52–55.
Although this illustration is provided with two adjoining hemispheres, the
’099 patent more generally contemplates mapping with respect to polyhedral
approximations of spheres described in the ’099 patent as “p-spheres.”
See id. at col. 5, ll. 50–61.
Claim 1, reproduced below, is illustrative of the claims at issue:
1. A method of modeling of the visible world using full-
surround image data, said method comprising:
selecting a view point within a p-surface; and
texture mapping full-surround image data onto said p-
surface such that the resultant texture map is substantially
equivalent to projecting full-surround image data onto the p-
surface from said view point to thereby generate a texture
mapped p-surface.

II. ANALYSIS
A. Claim Construction
The Board interprets claims using the broadest reasonable
construction in light of the specification of the patent in which they appear.
37 C.F.R. § 42.100(b); see also Office Patent Trial Practice Guide, 77 Fed.
Reg. 48,756, 48,766 (Aug. 14, 2012). Under that construction, claim terms
are given their ordinary and customary meaning as would be understood by
IPR2013-00547
Patent 6,243,099 B1

6
one of ordinary skill in the art in the context of the entire patent disclosure.
In re Translogic Tech., Inc., 504 F.3d 1249, 1257 (Fed. Cir. 2007).

1. Claim Terms Previously Construed
In the Decision to Institute, we construed claim terms as reproduced in
the table below. Dec. 7–13.
Claim Term Construction in the Decision to Institute
“full-surround image data”
1
data which samples the points P [defined as
“[t]he visible world” (Ex. 1001, col. 5, l. 30)].
This data encodes, explicitly or implicitly, the
association of a color value with a given
direction from a given point of projection.
“p-surface” a computer graphics representation of any
surface with a well-defined inside and outside,
where there exists at least one point x inside
(neither intersecting, nor lying outside) the
surface which may be connected to every point
of the surface with a distinct line segment, no
portion of which said line segment lies outside
the surface or intersects the surface at a point
not an endpoint.
“texture mapping” applying image data to a surface
“texture-mapped p-surface” p-surface onto which image data have been
applied
“projecting the full-
surround image data onto
the p-surface”
generating a new image by moving image
pixels along rays from the view point to the p-
surface

1
At oral hearing, Patent Owner confirmed that the claims use “full-surround
image data,” “full-surround data,” and “image data” synonymously.
Tr. 81:16–82:2.
IPR2013-00547
Patent 6,243,099 B1

7
During trial, Patent Owner accepted our constructions of “p-surface”
and “texture mapping,” but disputed our constructions of “full-surround
image data” and “projecting the [full-surround] image data onto the p-
surface,” and proposed constructions for “view point” and “point of
projection.” PO Resp. 12–17. In its Reply, Petitioner accepts our
construction of “full-surround image data” and “projecting the [full-
surround] image data onto the p-surface” (Reply 4–5, 10–11), opposes
Patent Owner’s constructions of “view point” and “point of projection” (id.
at 6–7, n.6), and does not contest our constructions of “p-surface” and
“texture mapping.”

2. “full-surround image data”
Independent claims 1 and 11 each recite “texture mapping full-
surround image data onto [a] p-surface” (emphasis added). The
specification includes the following description:
(4) FULL-SURROUND IMAGE DATA: data which
samples the points P. This data encodes, explicitly or
implicitly, the association of a color value with a given
direction from a given point of projection. It should be
mentioned at this point that full-surround image data is
useful in many fields of entertainment because, when
delivered to many viewers, it enables the construction of
an independent viewing system defined below.
Ex. 1001, col. 5, ll. 42–49 (emphasis added). In the Decision to Institute, we
found the italicized language in the description above to be an express
IPR2013-00547
Patent 6,243,099 B1

8
definition of “full-surround image data” and concluded that the term should
be construed in accordance with this definition. Dec. 7–8.
Patent Owner proposes construing “full-surround image data” to
mean:
data sampling points P of the visible world encoding, explicitly
or implicitly, the association of a color value C for each
sampled point in P observed in a given direction [ray V] from a
given point of projection (VP), the sampled points sufficient to
provide in a standard [3D] computer graphics system a display
in which changing the direction of view provides a human
viewer the impression of being present at the point of projection
from which the color was observed for the sampled point in P.

PO Resp. 14 (emphasis and brackets in original). Patent Owner contends
that the unitalicized portion of its proposed construction is the same as our
construction except that it adds letters and terms from “predicate conditions”
described in the specification and from Figures 2, 3, 4A, and 4B of the ’099
patent. Id. Specifically, Patent Owner argues that full-surround image data
encode a unique association of one point (P) and color (C) with a given ray
(V) from a given viewpoint (VP). Id.
According to Patent Owner (id. at 10–11), the claims must be read in
light of what it calls three “embodiment-independent predicate conditions”
disclosed in the specification:
The method and corresponding apparatus according to the
present invention are predicated on the following starting, i.e.,
given, conditions:
IPR2013-00547
Patent 6,243,099 B1

9
(1) the set of all rays V from a given point VP, as illustrated
in FIG. 1;
(2) a set of points P not including VP, each point in P being
contained by one and only one ray in V, as illustrated in
FIG. 2; and
(3) the set of color values C, each color in C being associated
with one and only one ray in V, and also thereby
associated with the point in P contained by said ray.
Ex. 1001, col. 5, ll. 17–28. Patent Owner argues that these conditions apply
to “the present invention” rather than to the recited embodiments, that they
lead up to the definitions set forth in the patent, and, thus, that they are
incorporated into the definitions. PO Resp. 11–12. Petitioner argues that
these additions are inconsistent with the express definition of “full-surround
image data.” Reply 5.
We agree with Patent Owner that the conditions reproduced above
define the set of points P as each being associated with a single ray from a
given view point VP and that each color in the set C similarly is associated
with a single ray.
2
Also, we are not persuaded by Petitioner’s argument that
these conditions are inconsistent with the express definition of “full-
surround image data.” Accordingly, we modify our construction to reflect
that full-surround image data encode the associations of unique color values
with given directions.

2
We note that C is not a color; it is a set of colors. Similarly, V is not a ray;
it is a set of rays.
IPR2013-00547
Patent 6,243,099 B1

10
As stated above, Patent Owner further proposes that full-surround
image data include “the sampled points sufficient to provide in a standard
[3D] computer graphics system a display in which changing the direction of
view provides a human viewer the impression of being present at the point
of projection from which the color was observed for the sampled point in P.”
PO Resp. 14 (emphasis omitted). To that end, Patent Owner argues, id., that
the specification provides the following additional definitions:
(10) INDEPENDENT VIEWING SYSTEM: an interactive
viewing system in which multiple viewers can freely,
independently of one another, and independently of the
source of the image data, pan that image data in all
directions with the effect that each viewer feels like they
are “inside” of that imagery, or present at the location
from which the imagery was produced, recorded, or
transmitted; and
(11) STANDARD COMPUTER GRAPHICS SYSTEM: a
computer graphics system which supports linear
perspective viewing, including the changing of the focal
length or the altering of the view angle, the apparent
rotation of viewed objects, and/or the apparent changing
of direction of vision, and the texture mapping of image
data onto objects within the class of p-surface.
Ex. 1001, col. 6, ll. 34–47. According to Patent Owner, these are additional
links in a “definitional chain” that we should incorporate into our
construction of full-surround image data. PO Resp. 14–16.
As noted above, the description of full-surround image data includes
that “[i]t should be mentioned at this point that full-surround image data is
useful in many fields of entertainment because, when delivered to many
IPR2013-00547
Patent 6,243,099 B1

11
viewers, it enables the construction of an independent viewing system
defined below.” Ex. 1001, col. 5, ll. 45–49. Patent Owner argues that this
language invokes the definition of “independent viewing system”
(definition 10). PO Resp. 15, n.3. Patent Owner further argues that an
independent viewing system is implemented on a standard computer graphic
system, invoking the definition of that term (definition 11). Id. At the
hearing, Patent Owner conceded that a standard computer graphics system,
the subject matter of definition 11 (Ex. 1001, col. 6, ll. 41–47), is not a
requirement of full-surround image data. Tr. 70:23–71:23. Rather, “this
was a construction that [Patent Owner] did in a different case.” Id. at 71:17–
18. Accordingly, we decline to read the requirement of a standard computer
graphics system into “full-surround image data.”
Regarding an independent viewing system, the subject matter of
definition 10 (Ex. 1001, col. 6, ll. 34–40), Patent Owner also has conceded
that this is not a requirement of “full-surround image data.” Tr. 71:21–
73:20. Moreover, we conclude that the patent’s description of full-surround
image data enabling the construction of an independent viewing system
(Ex. 1001, col. 5, ll. 45–49) is an example of a result that can be achieved
using full-surround image data, not an additional restriction on the scope of
the term. See In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1369 (Fed.
Cir. 2004) (citation omitted) (“We have cautioned against reading
limitations into a claim from the preferred embodiment described in the
specification, even if it is the only embodiment described, absent clear
IPR2013-00547
Patent 6,243,099 B1

12
disclaimer in the specification.”). Accordingly, we are not persuaded by
Patent Owner’s argument.
In sum, “full-surround image data” is “data which samples the
points P. This data encodes, explicitly or implicitly, the association of a
single color value with a given direction from a given point of projection.”

3. “projecting the full-surround image data onto the p-surface”
In our Decision to Institute, we construed “projecting the full-
surround image data onto the p-surface” as “generating a new image by
moving image pixels along rays from the view point to the p-surface.” Dec.
12–13. Patent Owner argues that the image data recited in the claim term
are not just any data, but are specifically “full-surround image data.” PO
Resp. 16–17. We agree with this contention, and, accordingly, clarify our
construction of “projecting the full-surround image data onto the p-surface”
to reflect that the image pixels are full-surround image data: “generating a
new image by moving image pixels of the full-surround image data along
rays from the view point to the p-surface.”

4. “view point” and “point of projection”
The Specification of the ’099 patent defines “view point” and “point
of projection” together:
(3) MAGIC POINT, VIEWPOINT, OR POINT OF
PROJECTION: Point VP. Please note, no matter how
points P are projected, their appearance will remain the
IPR2013-00547
Patent 6,243,099 B1

13
same when viewed from point VP. This latter concept
may best be understood by referring to FIGS. 4A and 4B.
Ex. 1001, col. 5, ll. 37–41. Patent Owner proposes construing “view point”
and “point of projection” to mean “[a] ‘Point VP’ such that ‘no matter how
points P are projected, their appearance will remain the same when viewed
from point VP.’” PO Resp. 12. In other words, Patent Owner contends that
the entire statement quoted above constitutes a definition of “view point”
and “point of projection.” Id.
Petitioner contends that no construction is necessary, but that, if we do
construe these terms, they should mean “Point VP.” Reply 7, n.6.
3

Petitioner does not explain why the remainder of the language of
definition (3), defining magic point, viewpoint, or point of projection, should
be omitted from our construction.
We understand the language “[p]lease note, no matter how points P
are projected, their appearance will remain the same when viewed from
point VP” (Ex. 1001, col. 5, ll. 38–40) as expressing a necessary
consequence of the interrelationship of the ’099 patent’s embodiment-
independent predicate conditions and definitions of “POINTS P,” “A

3
Petitioner argues that Patent Owner’s proposed construction is “belated.”
Id. We disagree. Patent Owner is required to set forth its arguments
addressing Petitioner’s grounds for unpatentability in its Patent Owner
response, which it did. See 37 C.F.R. § 42.120(a). Patent Owner’s
Preliminary Response was not mandatory, see 37 C.F.R. § 42.107, and, thus,
Patent Owner was not required to set forth its claim construction positions in
that filing.
IPR2013-00547
Patent 6,243,099 B1

14
PROJECTION OF P,” and “MAGIC POINT, VIEWPOINT, OR POINT OF
PROJECTION.” See Ex. 1001, col. 5, ll. 17–41. As such, that language is
properly considered definitional. Accordingly, we adopt Patent Owner’s
proposed construction of “view point” and “point of projection” as “a point
VP such that no matter how points P are projected, their appearance will
remain the same when viewed from point VP.”

B. Petitioner’s Motion to Exclude
Patent Owner supports its Response with an Expert Declaration of
James H. Oliver, Ph.D. (Ex. 2013, “Oliver Decl.”). Petitioner moves to
exclude ¶¶ 33, 41–44, 50, 55–57, and 67–72; Figures 5 and 6; and
Appendices B–D of the Oliver Declaration.
4
Paper 27, 3. Petitioner
contends that these portions of the Oliver Declaration rely on the
consideration of two software applications that are inadmissible because
they lack authentication and relevance. Id. at 1–2.
First, Petitioner contends that Dr. Oliver’s testimony at ¶¶ 41–44, 55,
and 57; Figure 5; and Appendix C of Exhibit 2013 relied upon software
purportedly corresponding to the system described in Photo VR. Id. at 3–4,
6–7. Petitioner contends that the date stamp of the software is in 1999, after
the 1996 date of Photo VR, and argues that Dr. Oliver did not confirm

4
Petitioner also moves to exclude portions of a Supplemental Oliver
Declaration (Ex. 1032) “to the extent Patent Owner files and/or seeks to
enter Patent Owner’s Response to Google’s Objections.” Paper 27 at 3, n.1.
Patent Owner did not file the identified paper.
IPR2013-00547
Patent 6,243,099 B1

15
adequately that this software is the same software discussed in Photo VR.
Id. at 6–7. Accordingly, Petitioner argues, this software and the portions of
Dr. Oliver’s testimony that rely upon it are irrelevant. Id. at 6.
Second, Petitioner contends that Dr. Oliver’s testimony at ¶¶ 33, 50,
56, and 67–72; Figure 6; and Appendices B and D of Exhibit 2013 relied
upon software code purportedly corresponding to the source code deposited
with the Patent Office with the application for the ’099 patent. Id. at 7–9.
Petitioner contends that this source code actually was modified by a third
party after the effective filing date of the ’099 patent. Id. Accordingly,
Petitioner argues, this software and the portions of Dr. Oliver’s declaration
that rely upon it are irrelevant. Id. at 9.
Petitioner further argues that any relevance of these two pieces of
software is outweighed by its potential to confuse and mislead. Id.
Petitioner also contends that this software and the testimony that relies upon
it lack authentication and are hearsay. Id. at 10–12.
Patent Owner responds that it does not seek to admit the Photo VR or
’099 patent software itself. Paper 30, 2. Patent Owner argues that
Dr. Oliver considered this software as part of his review of the technology at
issue in the proceeding. Id. Patent Owner points out, id. at 6, that the bases
for an expert’s opinion need not be admissible “[i]f experts in the particular
field would reasonably rely on those kinds of facts or data in forming an
opinion on the subject.” FED. R. EVID. 703.
IPR2013-00547
Patent 6,243,099 B1

16
As to the ’099 patent software, Patent Owner argues that Dr. Oliver
determined that it was functionally identical to that disclosed in the patent
application (and consistent with the patent’s disclosure) except that it was
modified slightly to work with a current operating system. Paper 30, 2–3, 6–
8. As to the Photo VR software, Patent Owner argues that it corroborated
the understanding Dr. Oliver gained from the Photo VR reference itself. Id.
at 3. According to Patent Owner, Dr. Oliver had the knowledge and
experience necessary to authenticate the software by comparing its functions
with what they purport to be. Id. at 6 (citing FED. R. EVID. 901(b)(3) (listing
“[a] comparison with an authenticated specimen by an expert witness” as an
example of evidence that satisfies the authentication requirement)).
We are not persuaded that Dr. Oliver’s testimony should be excluded.
Rather, an expert reasonably could rely upon later versions of software to
gain a general understanding of the technology at issue in documents that
describe earlier versions of such software. Petitioner has not persuaded us
that Patent Owner’s purported failure to establish an identity between the
software Dr. Oliver actually reviewed and the software referenced in
Photo VR and the ’099 patent makes such reliance unreasonable in this
instance. Petitioner’s arguments instead go toward the weight we should
give Dr. Oliver’s testimony. Cf. Smith v. Ford Motor Co., 215 F.3d 713,
718 (Fed. Cir. 2000) (“The soundness of the factual underpinnings of the
expert’s analysis and the correctness of the expert’s conclusions based on
that analysis are factual matters to be determined by the trier of fact.”).
IPR2013-00547
Patent 6,243,099 B1

17
C. Anticipation by Photo VR
Petitioner contends that claims 1, 3, 11, and 13 are anticipated by
Photo VR. Pet. 35–44, 49–59.

1. Photo VR
Photo VR is prior art to each of the challenged claims under 35 U.S.C.
§ 102(b) because its publication date of July 30–31, 1996, precedes the
effective filing date of the ’099 patent by more than one year. Photo VR is
also prior art to those claims under 35 U.S.C. § 102(a) because Patent Owner
does not allege an invention date earlier than the publication date of
Photo VR.
Photo VR is directed to panoramic view rendering “by generating a
sphere-like polyhedral environment map from photo-realistic images and
using the generated maps to render the scene by techniques of computer
graphics.” Ex. 1007, 397 (col. 1). Photo VR illustrates its method in the
context of rendering a scene in a room by positioning a camera in a “proper
position, such as the center of the room,” from which images of the entire
view are taken and arranged as a “sphere-like polyhedron consisting of
textured trapezoids.” Id. at 397 (col. 2). Because the center of projection of
the images moves slightly as the camera is panned, Photo VR assumes that
the objects in the scene are sufficiently far from the camera that the effects
of any movement are negligible. Id. at 398 (col. 1). “Thus all optic axes of
the images can be regarded as intersected at the [center of projection].” Id.
IPR2013-00547
Patent 6,243,099 B1

18
The photographic images are registered to polygons of a polyhedron.
Id. at 398 (col. 2). For example, images can be registered to trapezoids or
triangles of a sphere-like polyhedron. Id. at 399 (col. 1). After the images
are registered, a texture mapped polyhedron is generated. Id. at 398 (col. 2).
The figure from page 398 of Photo VR, reproduced below, illustrates an
example:

The figure from page 398 illustrates the generation of a “texture mapped
sphere-like polyhedron” by ray-casting of original images onto polygons
arranged in the space by their registrations. Id. at 398 (col. 2).
Photo VR also describes an “object viewer,” as illustrated in a
drawing from page 400, reproduced below:
IPR2013-00547
Patent 6,243,099 B1

19

The drawing from page 400 illustrates that the object viewer allows a
user to “interactively observe an object from different views in real time.”
Id. at 400 (col. 1). That is, images “A,” “B,” and “C” in the drawing show
images taken of the same “object” from different views. Id.

2. Anticipation of Claims 1, 3, 11, and 13

We have reviewed the evidence presented by Petitioner, including the
claim charts in the Petition (Pet. 35–59) and the Declaration of John R.
Grindon, D.Sc. (Ex. 1005). Notwithstanding Patent Owner’s arguments, we
are persuaded that Photo VR anticipates claims 1, 3, 11, and 13. For
example, Petitioner has shown sufficiently that Photo VR discloses
modeling of the visible world using “full-surround image data” in that it
acquires images of “the whole view from the camera position.” Ex. 1007,
397 (col. 2). Likewise, Petitioner has shown that the sphere-like polyhedron
disclosed by Photo VR is a “p-surface” (in fact, a “p-sphere”) because it
provides a computer-graphics representation of a surface having a well-
defined inside and outside, with at least one point inside the surface capable
IPR2013-00547
Patent 6,243,099 B1

20
of connection to every point of the surface with a distinct line segment,
wherein no portion of the line segment lies outside the surface or intersects
the surface at a point that is not an endpoint. Pet 17, 39. Petitioner also has
shown that Photo VR discloses “texture mapping” of image data onto the p-
surface because the mapping it describes applies image data generated by
taking images from the camera position, e.g., at the center of a room (i.e.,
from a view point) onto polygons defined on the p-surface. Ex. 1007, 397
(col. 2). We address Patent Owner’s arguments below.

a. Photo VR Discloses “full-surround image data”
Patent Owner contends that a single image, such as one of the images
acquired by the camera disclosed in Photo VR, is not full-surround image
data. PO Resp. 20–21. Petitioner does not contend, however, that a single
image constitutes full-surround image data, nor did we make a preliminary
finding to that effect in our Decision to Institute. See Pet. 17, 39; Dec. 7–8;
Tr. 87:4–8. Rather, as explained in Section II.A.2, above, full-surround
image data are data which sample the points P.
Patent Owner further contends that full-surround image data must
have a single common view point from which the points P of the visible
world are observed. PO Resp. 18–19. By contrast, Patent Owner argues,
Photo VR discloses acquiring images from a camera with a moving center of
projection. Id. (citing Ex. 1007, 398 (col. 1)). Patent Owner points out that
Photo VR states that “it is almost impossible for some images to be
IPR2013-00547
Patent 6,243,099 B1

21
registered without objects on the border being duplicated or lost in adjacent
images because the camera was moved during panning.” PO Resp. 19
(quoting Ex. 1007, 400 (col. 2)).
Petitioner responds that Photo VR does disclose a single common
view point from which the points P are observed. Reply 6–8. As Petitioner
points out, Photo VR discloses that the camera is positioned in the same
place (e.g., the center of the room) for each of the photographs in a set and
that, for the acquired images, “all optic axes of the images can be regarded
as intersected at the [center of projection].” Id. at 7–8 (citing Ex. 1007, 398
(col. 1)). According to Petitioner, the purported deficiency to which Patent
Owner cites simply is an acknowledgment in Photo VR that there are
practical challenges to capturing multiple images from a common view
point. Reply 8. Patent Owner’s declarant, Dr. Oliver, admits that the
objective of Photo VR is to maintain the same center of projection for each
of the images. Ex. 2015, 113:4–21. Once the images are captured,
Petitioner argues, the generated texture map in Photo VR has a common
view point. Reply 8.
We agree with Petitioner. Photo VR regards each of the acquired
images as having a common center of projection. Ex. 1007, 398 (col. 1). As
a result, Photo VR explains, when the images acquired with slightly different
centers of projection are registered to the same center of projection, some
objects on the borders of the images may be duplicated or lost. Id. at 400
(col. 2). Thus, although the quality of some of the images might be
IPR2013-00547
Patent 6,243,099 B1

22
degraded, all of the acquired data are registered to the same center of
projection.
This is not meaningfully different from the embodiment described in
the ’099 patent, for example, in which a texture map is “built from two
pictures, respectively, taken with a fisheye lens,” together comprising a
“pictosphere.” Ex. 1001, col. 7, ll. 28–31. Because the camera must move
to take the second image, there will be a difference, however subtle, between
the centers of projection of the acquired images. Patent Owner directs us to
U.S. Patent No. 5,903,782 (Ex. 1019, “the ’782 patent”), incorporated by
reference into the ’099 patent (see Ex. 1001, col. 8, ll. 33–38), as disclosing
a technique for taking multiple photographs from exactly the same view
point. Tr. 56:22–57:9. In the embodiment disclosed in the ’782 patent, a
first image is taken, using a fisheye lens, after which the camera is rotated
180 degrees and a second image is taken. Ex. 1019, col. 3, ll. 31–35.
Because the camera moves, the view points of the two images cannot be in
exactly the same place. As Patent Owner admits, there will be some
“tolerance.” Tr. 57:7–23. Moreover, Patent Owner concedes that full-
surround image data are not limited to data acquired using fisheye lenses.
Tr. 58:15–22.
Patent Owner also contends that Photo VR’s registered image data are
not full-surround image data because the individual images, when mapped to
polygons, may overlap one another. PO Resp. 19–20. According to
Photo VR, a texture mapped sphere-like polyhedron is generated by ray
IPR2013-00547
Patent 6,243,099 B1

23
casting of the original images onto the polygons of the polyhedron according
to the registrations of the images, as shown in the figure from page 398 of
Photo VR (reproduced above). Ex. 1007, 398 (col. 2). Prior to Photo VR,
where images overlapped, typical ray casting involved using the color from
the first image hit by the ray. Id. at 399 (col. 1). In Photo VR, in contrast,
the colors for all images hit by the ray are averaged, using a weighted
average to determine the color for that ray. Id. Patent Owner argues that
this is not full-surround image data because there is no unique
correspondence of color, ray, and viewpoint. PO Resp. 19–20.
In response, Petitioner argues that an averaged color value is a single
unique color value corresponding to a ray and viewpoint. Reply 9–10.
Petitioner points to the testimony of Patent Owner’s declarant, Dr. Oliver,
that the averaged color value is a single result approximating two colors.
Id. (citing Ex. 2015, 122:17–123:6). We are persuaded by Petitioner that the
resultant color disclosed by Photo VR, which is a weighted average of two
overlapping colors, is a single color value corresponding to a ray from a
view point (here, the center of projection). Ex. 1007, 399 (col. 1). The data
that ultimately are texture mapped to the polyhedron contain averaged
IPR2013-00547
Patent 6,243,099 B1

24
values in places where the originally acquired data would have overlapped.
5

Patent Owner concedes that, except for the fact that the original images were
not acquired from the same view point (an argument we reject above), the
data actually mapped onto Photo VR’s polyhedron would be considered full-
surround image data. Tr. 67:20–68:13.
In sum, we are persuaded that Petitioner has shown that Photo VR
discloses that the data texture mapped to the polyhedron encodes, explicitly
or implicitly, the associations of unique color values with given directions
from a given point of projection, i.e., are full-surround image data.

b. Photo VR Discloses “projecting the full-surround
image data onto the p-surface from [a] view point”

Patent Owner contends that Photo VR does not disclose texture
mapping full-surround image data to generate a texture map “substantially

5
In response to our questioning at oral hearing, Patent Owner argued that
the color-averaging process itself is part of Photo VR’s texture-mapping
procedure and that this texture mapping is not performed on full-surround
image data because full-surround image data only result after the color
averaging. Tr. 68:14–69:2. This argument was not made previously by
Patent Owner. In any case, we are not persuaded by this argument.
Photo VR discloses texture mapping as performed with standard “rendering
packages, libraries or graphic hardware accelerators.” Ex. 1007, p. 397,
col. 2. Such standard rendering procedures are distinct from the color-
averaging process in Photo VR’s overall procedure. Thus, even if the color-
averaging process of Photo VR is a texture mapping, Photo VR discloses a
distinct texture mapping that is performed on full-surround image data. See
Tr. 88:9–89:17.
IPR2013-00547
Patent 6,243,099 B1

25
equivalent to projecting the full-surround image data onto the p-surface from
[a] view point,” as recited in independent claims 1 and 11. PO Resp. 21–22.
Specifically, Patent Owner argues that the data projected in Photo VR is not
full-surround image data. For the reasons given in Section II.C.2.a, we
disagree with this argument.
Patent Owner also argues that Photo VR describes specular lighting as
important but that Photo VR does not handle specular lighting. PO Resp.
21, n.6. Patent Owner relates this observation to an argument that
Photo VR’s method of capturing and projecting linear images introduces
distortion, contending that the claims of the ’099 patent require “distortion-
free results.” Id. at 21. The claims do not recite “distortion-free results.”
Moreover, as we explain in Section II.A.2, the term “full-surround image
data” does not require “the sampled points sufficient to provide in a standard
3D computer graphics system a display in which changing the direction of
view provides a human viewer the impression of being present at the point
of projection from which the color was observed for the sampled point in P,”
as argued by Patent Owner. See PO Resp. 14. Thus, we are not persuaded
that the claims require distortion-free results. Accordingly, Petitioner has
shown that Photo VR discloses “projecting the full-surround image data onto
the p-surface from a point of projection.”

IPR2013-00547
Patent 6,243,099 B1

26
c. Photo VR Discloses Selecting a View Point and a Direction of View
Independent claims 1 and 11 require “selecting a view point within a
p-surface.” Dependent claims 3 and 13 require that “said selecting step
comprises selecting the view point and a direction of view.” Patent Owner
argues that these limitations are not disclosed in Photo VR because the
“Photo VR ‘Object Viewer’ applies to [an] object, not [a] p-surface.”
PO Resp. 26–29. Patent Owner also argues that the Object Viewer
described in Photo VR is “inapposite” because it “does not appear to have
any connection with a texture-mapped p-surface and should be disregarded.”
Id. at 22.
It is unclear what Patent Owner contends is lacking in Photo VR when
compared with the claims. Nevertheless, as Petitioner points out, Photo VR
describes several instances of a user selecting a view point. Pet. 17, 37, 42,
52, 58; Reply 11–13. For example, Figures 5(1)–5(8) (Ex. 1007, p. 403)
illustrate images rendered at various viewing angles. Photo VR describes a
user selecting a direction of view using the Object Viewer. Ex. 1007, 400
(col. 1). The testimony of Patent Owner’s declarant, Dr. Oliver, further
implies that a user of the Photo VR system selects a direction of view from a
view point on a model. Ex. 2013 ¶ 57 (“user selects a viewpoint near the
center of projection”); id. (“[Photo VR] suggests navigating through
adjacent overlapping spherical environment maps”). We also agree with
Petitioner that Photo VR describes the Object Viewer as a component of the
same Windows 95–based system that contains the other described
IPR2013-00547
Patent 6,243,099 B1

27
components, such as the module that generates texture-mapped polyhedrons.
Id. at p. 400, cols. 1–2. Thus, Patent Owner has not persuaded us that the
Object Viewer is “inapposite.” Accordingly, Petitioner has shown that
Photo VR discloses “allowing a user to select a direction of view from a
view point on the model.”

d. Claims 1, 3, 11, and 13 Are Anticipated by Photo VR

In sum, based on our consideration of the evidence presented by
Petitioner and the arguments detailed above, we are persuaded that Petitioner
has proved by a preponderance of the evidence that independent claims 1, 3,
11, and 13 are anticipated by Photo VR.

D. Obviousness Over Photo VR and QuickTime
®
VR
Petitioner contends that claim 4 (which depends from independent
claim 1) and claim 5 (which depends from claim 4) would have been
obvious under 35 U.S.C. § 103(a) over Photo VR and QuickTime
®
VR. Pet.
45–48. As discussed above, Petitioner relies on Photo VR to meet all
features of independent claim 1. It relies on QuickTime
®
VR to account for
the additional features recited in claims 4 and 5.

1. QuickTime
®
VR
QuickTime
®
VR discloses the use of 360-degree cylindrical
panoramic images to compose a virtual environment that allows users to
IPR2013-00547
Patent 6,243,099 B1

28
pan, zoom, and navigate a scene. Ex. 1012, 29 (col. 1); 32 (col. 1).
QuickTime
®
VR explicitly states that one “constraint with the current
panoramic player is its limitation in looking straight up or down due to the
use of cylindrical panoramic images. This limitation can be removed if
other types of environment maps, such as cubic or spherical maps, are
used.” Id. at 36 (col. 2) (emphasis added). Conversely, Photo VR explicitly
notes that its spherical panoramic rendering of a real-world environment “is
different from the QuickTime VR approach,” a consequence both of its use
of spherical rendering and of its use of texture mapping instead of image
warping. Ex. 1007 at 397 (col. 1).

2. Combination of Photo VR and QuickTime
®
VR
Photo VR states that
[systems employing a cylindrical environment map] have a
common problem: the environment map used cannot cover the
top view and the bottom view, so the vertical viewing angle is
limited. In the following, we will propose a better method that
renders the scene by sphere-like polyhedral environment maps,
and thus the problem is solved.

Ex. 1007, 397 (col. 1). Petitioner reasons that Photo VR thus “explicitly
sought to improve” upon QuickTime
®
VR’s lack of mapping images to the
top and bottom of the cylinder. Pet. 27. This reasoning is supported by the
testimony of Petitioner’s declarant, Dr. Grindon. Ex. 1005 ¶ 125. Petitioner
asserts that both references are reasonably pertinent to the same problem and
IPR2013-00547
Patent 6,243,099 B1

29
that they are in the same field of endeavor of “image processing/computer
graphics.” Reply 13–14. We are persuaded that Petitioner has demonstrated
that one of skill in the art would have combined the teachings of Photo VR
and QuickTime
®
VR.
Patent Owner contends that “[a]n important consideration in
combining references is: which is the primary reference,” and that “[n]o
rationale is given by Petitioner for why one of ordinary skill in the art,
presented with QuickTime VR, would have any reason to look to Photo
VR.” PO Resp. 24–25. We are not persuaded by these contentions, as we
do not consider the order in which prior art is applied in a challenge to a
claim to be significant. See, e.g., In re Mouttet, 686 F.3d 1322, 1333
(Fed. Cir. 2012) (“[W]here the relevant factual inquiries underlying an
obviousness determination are otherwise clear, characterization by the
examiner of prior art as ‘primary’ and ‘secondary’ is merely a matter of
presentation with no legal significance.” (citation omitted)). In light of
Petitioner’s explanation that Photo VR explicitly refers to QuickTime
®
VR
and teaches an improvement over it, we conclude that Petitioner has
provided a reason, with rational underpinning, to make the combination.
Patent Owner also draws a distinction between image processing and
computer graphics, contending that those fields were separate at the time of
the invention. PO Resp. 10–11. But Patent Owner has not explained how
this distinction relates to combining the teachings of Photo VR and
QuickTime
®
VR, and has not contended that those references are drawn
IPR2013-00547
Patent 6,243,099 B1

30
from different ones of those fields. Instead, Patent Owner argues that
“accepted wisdom in image processing in 1997 was that computational
resources were limited, and (in general) should not be wasted,” and that
additional image processing steps such as described in the ’099 patent would
have been counterintuitive. Id. at 25. We are not persuaded by this
argument, which is presented generically, without specifically addressing the
content of Photo VR and QuickTime
®
VR. In particular, we are persuaded
by Petitioner’s observation that Photo VR discusses QuickTime
®
VR and
proposes improvements over it, an observation that Patent Owner does not
rebut adequately.

3. Obviousness of Claims 4 and 5
Claim 4 recites “further comprising displaying a predetermined
portion of said texture mapped p-surface” and claim 5 recites “wherein a
viewer is allowed to interactively alter at least one of focal length or an
angle of view relative to said textured mapped p-surface to thereby vary the
displayed portion of said texture mapped p-surface.”
QuickTime
®
VR discloses a panoramic moviemaking process that
stitches together a mosaic of overlapping photographs, including the ability
to link panoramas together by registering their viewing directions. Ex. 1012,
34 (col. 1). This linking allows users to activate links by clicking on hot
spots marked on the panoramic images. Id. at 35 (col. 1). In addition to
allowing users to pan, zoom, and navigate a scene (id. at 32 (col. 1)),
IPR2013-00547
Patent 6,243,099 B1

31
QuickTime
®
VR discloses that “[a]nother use of the orientation-independent
movie is in interactive TV. A movie can be broadcast in a 360-degree
format, perhaps using multiple channels. Each TV viewer can freely control
the camera angle locally while watching the movie.” Ex. 1012, 35 (col. 2).
We are persuaded that the combination of Photo VR with this
disclosure teaches displaying a predetermined portion of the texture-mapped
p-surface, as recited in claim 4. In addition, we are persuaded that this
disclosure teaches allowing a viewer interactively to alter an angle of view
relative to the texture-mapped p-surface to vary the displayed portion,
thereby disclosing the limitation of claim 5.
Patent Owner asserts that these limitations are not disclosed by
Photo VR or QuickTime
®
VR “singly or in any combination.” PO Resp.
25–26. But Patent Owner does not provide sufficient reasoning that
addresses these findings.
Thus, based on our consideration of the evidence presented by
Petitioner and the arguments detailed above, we are persuaded that Petitioner
has proved by a preponderance of the evidence that claims 4 and 5 would
have been obvious over Photo VR and QuickTime
®
VR.

III. CONCLUSION
Petitioner has demonstrated by a preponderance of the evidence that
(1) claims 1, 3, 11, and 13 are anticipated under 35 U.S.C. §§ 102(a) and
IPR2013-00547
Patent 6,243,099 B1

32
102(b) by Photo VR; and (2) claims 4 and 5 would have been obvious under
35 U.S.C. § 103(a) over Photo VR and QuickTime
®
VR.

IV. ORDER
For the reasons given, it is
ORDERED that, based on a preponderance of the evidence, claims 1,
3–5, 11, and 13 of U.S. Patent No. 6,243,099 B1 are held to be unpatentable;
FURTHER ORDERED that Petitioner’s motion to exclude ¶¶ 33, 41–
44, 50, 55–57, and 67–72; Figures 5 and 6; and Appendices B–D of Exhibit
2013 is denied; and
FURTHER ORDERED that, because this is a final written decision,
parties to this proceeding seeking judicial review of our decision must
comply with the notice and service requirements of 37 C.F.R. § 90.2.

IPR2013-00547
Patent 6,243,099 B1

33
PETITIONER:

Cono A. Carrano
Ruben Munoz
ccarrano@akingump.com
rmunoz@akingump.com

PATENT OWNER:

Stephen Chow
Hsuanyeh Chang
Robert Groover
Seth Horwitz
schow@burnslev.com
hchang@burnslev.com
groover@technopatents.com
shorwitz6576@gmail.com