Google Inc.v.Grandeye Ltd.

Patent Trial and Appeal Board

Nov 25, 2014

10602666 (P.T.A.B. Nov. 25, 2014)

Trials@uspto.gov Paper 34
571-272-7822 Entered: November 25, 2014

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

GOOGLE INC.,
Petitioner,

v.

GRANDEYE LTD.,
Patent Owner.
____________

Case IPR2013-00548
Patent 7,542,035 B2
____________

Before JAMESON LEE, DAVID C. McKONE, and
PATRICK M. BOUCHER, Administrative Patent Judges.

BOUCHER, Administrative Patent Judge.

FINAL WRITTEN DECISION
35 U.S.C. § 318(a) and 37 C.F.R. § 42.73

IPR2013-00548
Patent 7,542,035 B2

2
I. INTRODUCTION
A. Background
On August 30, 2013, Google Inc. (“Petitioner”) filed a Petition
(Paper 1, “Pet.”) to institute an inter partes review of claims 1, 7, 13, 22, 33,
44, 53, 54, 69, 73, 78, 80, 82–84, 86, 89, 98, 102, 105, 113, 115, 118–121,
and 123 of U.S. Patent No. 7,542,035 B2 (Ex. 1003, “the ’035 patent”)
pursuant to 35 U.S.C. § 311–319. Grandeye Ltd. (“Patent Owner”) filed a
Preliminary Response (Paper 15, “Prelim. Resp.”) to the Petition on
November 7, 2013. Pursuant to 35 U.S.C. § 314, the Board instituted trial
on February 5, 2014, as to claims 1, 22, 53, 54, 69, 73, 78, 80, 82–84, 86, 89,
98, 102, 105, 113, 115, 118–121, and 123. Paper 16 (“Dec.”); the Board did
not institute trial as to claims 7, 13, 33, or 44.
During the trial, Patent Owner timely filed a Patent Owner Response
(Paper 22, “PO Resp.”), and Petitioner timely filed a Reply to the Patent
Owner Response (Paper 25, “Reply”). An oral hearing was held on
September 5, 2014. Paper 33 (“Tr.”).
We have jurisdiction under 35 U.S.C. § 6(c). This Decision is a final
written decision under 35 U.S.C. § 318(a) as to the patentability of the
challenged claims. Based on the record before us, Petitioner has
demonstrated by a preponderance of the evidence that claims 1, 22, 53, 54,
69, 73, 78, 80, 82–84, 86, 89, 98, 102, 105, 113, 115, 118–121, and 123 are
unpatentable.

IPR2013-00548
Patent 7,542,035 B2

3
B. Related Proceedings
In addition to this proceeding, Petitioner filed petitions for inter partes
review of, and we instituted trial on, certain claims of U.S. Pat. Nos.
6,243,099 B1 (Ex. 1001, “the ’099 patent”) and 8,077,176 (Ex. 1002, “the
’176 patent”) in IPR2013-00547 and IPR2013-00546, respectively. Paper 6,
3. The ’176 patent is a continuation of the ’035 patent, which is a
continuation of the ’099 patent. Ex. 1002 at [63]; Ex. 1003 at [63].
View 360 Solutions LLC (“View 360”), a purported licensee of the
’035, ’099, and ’176 patents, has sued Petitioner for infringement of each of
those patents in View 360 Solutions LLC v. Google, Inc., Case No. 1:12-cv-
1352 (N.D.N.Y.). Pet. 1–2; Paper 6, 3. Patent Owner has asserted the ’035
and ’099 patents against others in Grandeye Ltd. v. Sentry 360 Security, Inc.,
Case No. 1:11-cv-02188 (N.D. Ill.).

C. Grounds of Unpatentability
Petitioner relies on the following prior-art references.

Photo VR Wen-kae Tsao et al., Photo VR: A System of Rendering
High Quality Images for Virtual Environments Using
Sphere-like Polyhedral Environment Maps, THE SECOND
WORKSHOP ON REAL-TIME AND MEDIA SYSTEMS (RAMS
’96) 397–403 (July 30–31, 1996) (Ex. 1007);

QuickTime® VR Shenchang Eric Chen, QuickTime
®
VR — An Image-
Based Approach to Virtual Environment Navigation
(ACM, 1995) (Ex. 1012).
IPR2013-00548
Patent 7,542,035 B2

4
We instituted this proceeding based on the asserted grounds that
claims 1, 22, 53, 54, 69, 73, 78, 80, 82–84, 89, 98, 102, 105, 113, 115, and
118–121 are anticipated under 35 U.S.C. §§ 102(a) and 102(b) by Photo VR;
and that claims 84, 86, 120, 121, and 123 would have been obvious under 35
U.S.C. § 103(a) over Photo VR and QuickTime
®
VR. Dec. 32.

D. The ’035 Patent
The ’035 patent “relates generally to a method and corresponding
apparatus for viewing images.” Ex. 1003, col. 1, ll. 49–50. For instance, a
virtual pictosphere may be created using a conventional three-dimensional
graphics system that results from “texture mapping” the visible world onto a
sphere. Id. at col. 6, ll. 21–24. Different viewpoints enable different types
of perspective views when rendered with the primitives of a conventional
three-dimensional graphics system. For example, a linear perspective view
is achieved with a viewpoint at the center of the sphere, while a circular
perspective view is achieved with a viewpoint on the surface of the sphere
with a view direction towards the center. Id., col. 6, ll. 24–33. Figures 5 and
6 of the ’035 patent, reproduced below, are illustrative:

IPR2013-00548
Patent 7,542,035 B2

5
Figures 5 and 6 show projections of a portion of the visible world onto a
plane with a linear perspective view and a circular perspective view,
respectively.
In an illustrative example, the ’035 patent describes the mapping of
two fisheye images to adjoining hemispheres to generate spherical image
data. Id. at col. 9, ll. 3–18. A user interactively may move the viewpoint to
different positions that include the center of the sphere and to points very
near the inside of the sphere, thereby achieving the different perspective
views. Id. at col. 9, ll. 11–16. The surface of the sphere also may be rotated
to simulate looking around within the sphere. Id. at col. 9, ll. 16–18.
Although this illustration is provided with two adjoining hemispheres, the
’035 patent more generally contemplates mapping with respect to polyhedral
approximations of spheres described in the ’035 patent as “p-spheres.”
See id. at col. 7, ll. 11–23.
Claim 1, reproduced below, is illustrative of the claims at issue:
1. A method of modeling of the visible world using full-
surround image data, said method comprising:
selecting a view point within a p-surface, wherein the p-
surface comprises polygons approximating a partial sphere;
selecting a direction of view within the p-surface;
texture mapping full-surround image data onto said p-
surface such that the resultant texture map is substantially
equivalent to projecting full-surround image data onto the p-
surface from said view point to thereby generate a texture
mapped p-surface; and
displaying a predetermined portion of said texture
mapped p-surface.
IPR2013-00548
Patent 7,542,035 B2

6
II. ANALYSIS
A. Claim Construction
The Board interprets claims using the broadest reasonable
construction in light of the specification of the patent in which they appear.
37 C.F.R. § 42.100(b); see also Office Patent Trial Practice Guide, 77 Fed.
Reg. 48,756, 48,766 (Aug. 14, 2012). Under that construction, claim terms
are given their ordinary and customary meaning as would be understood by
one of ordinary skill in the art in the context of the entire patent disclosure.
In re Translogic Tech., Inc., 504 F.3d 1249, 1257 (Fed. Cir. 2007).

1. Claim Terms Previously Construed
In the Decision to Institute, we construed claim terms as reproduced in
the table below. Dec. 8–13.
Claim Term Construction in the Decision to Institute
“full-surround image data”
1
data which samples the points P [defined as
“[t]he visible world” (Ex. 1003, col. 6, l. 58)].
This data encodes, explicitly or implicitly, the
association of a color value with a given
direction from a given point of projection.
“p-surface” a computer graphics representation of any
surface with a well-defined inside and outside,
where there exists at least one point x inside
(neither intersecting, nor lying outside) the
surface which may be connected to every point

1
At oral hearing, Patent Owner confirmed that the claims use “full-surround
image data,” “full-surround data,” and “image data” synonymously.
Tr. 81:16–82:2.
IPR2013-00548
Patent 7,542,035 B2

7
of the surface with a distinct line segment, no
portion of which said line segment lies outside
the surface or intersects the surface at a point
not an endpoint.
“texture mapping” applying image data to a surface
“texture-mapped p-surface” p-surface onto which image data have been
applied
“projecting the [full-
surround] image data onto
the p-surface”
generating a new image by moving image
pixels along rays from the view point to the p-
surface

During trial, Patent Owner accepted our constructions of “p-surface”
and “texture mapping,” but disputed our constructions of “full-surround
image data” and “projecting the [full-surround] image data onto the p-
surface,” and proposed constructions for “view point” and “point of
projection.” PO Resp. 12–18. In its Reply, Petitioner accepts our
construction of “full-surround image data” and “projecting the [full-
surround] image data onto the p-surface” (Reply 4–5, 10–11), opposes
Patent Owner’s constructions of “view point” and “point of projection” (id.
at 6–7, n. 6), and does not contest our constructions of “p-surface” and
“texture mapping.”

2. “full-surround image data”
Independent claims 1, 22, and 53 each recite “texture mapping full-
surround image data onto [a] p-surface” (emphasis added). The
specification includes the following description:
IPR2013-00548
Patent 7,542,035 B2

8
(4) FULL-SURROUND IMAGE DATA: data which
samples the points P. This data encodes, explicitly or
implicitly, the association of a color value with a given
direction from a given point of projection. It should be
mentioned at this point that full-surround image data is
useful in many fields of entertainment because, when
delivered to many viewers, it enables the construction of
an independent viewing system defined below.
Ex. 1003, col. 7, ll. 3–10 (emphasis added). In the Decision to Institute, we
found the italicized language in the description above to be an express
definition of “full-surround image data” and concluded that the term should
be construed in accordance with this definition. Dec. 8–9.
Patent Owner proposes construing “full-surround image data” to
mean:
data sampling points P of the visible world encoding, explicitly
or implicitly, the association of a color value C for each
sampled point in P observed in a given direction [ray V] from a
given point of projection (VP), the sampled points sufficient to
provide in a standard [3D] computer graphics system a display
in which changing the direction of view provides a human
viewer the impression of being present at the point of projection
from which the color was observed for the sampled point in P.

PO Resp. 14 (emphasis and brackets in original). Patent Owner contends
that the unitalicized portion of its proposed construction is the same as our
construction except that it adds letters and terms from “predicate conditions”
described in the specification and from Figures 2, 3, 4A, and 4B of the ’035
patent. Id. Specifically, Patent Owner argues that full-surround image data
IPR2013-00548
Patent 7,542,035 B2

9
encode a unique association of one point (P) and color (C) with a given ray
(V) from a given viewpoint (VP). Id.
According to Patent Owner (id. at 10–11), the claims must be read in
light of what it calls three “embodiment-independent predicate conditions”
disclosed in the specification:
The method and corresponding apparatus according to the
present invention are predicated on the following starting, i.e.,
given, conditions:
(1) the set of all rays V from a given point VP, as illustrated
in FIG. 1;
(2) a set of points P not including VP, each point in P being
contained by one and only one ray in V, as illustrated in
FIG. 2; and
(3) the set of color values C, each color in C being associated
with one and only one ray in V, and also thereby
associated with the point in P contained by said ray.
Ex. 1003, col. 6, ll. 46–56. Patent Owner argues that these conditions apply
to “the present invention” rather than to the recited embodiments, that they
lead up to the definitions set forth in the patent, and, thus, that they are
incorporated into the definitions. PO Resp. 11. Petitioner argues that these
additions are inconsistent with the express definition of “full-surround image
data.” Reply 5.
We agree with Patent Owner that the conditions reproduced above
define the set of points P as each being associated with a single ray from a
given view point VP and that each color in the set C similarly is associated
IPR2013-00548
Patent 7,542,035 B2

10
with a single ray.
2
Also, we are not persuaded by Petitioner’s argument that
these conditions are inconsistent with the express definition of “full-
surround image data.” Accordingly, we modify our construction to reflect
that full-surround image data encode the associations of unique color values
with given directions.
As stated above, Patent Owner further proposes that full-surround
image data include “the sampled points sufficient to provide in a standard
[3D] computer graphics system a display in which changing the direction of
view provides a human viewer the impression of being present at the point
of projection from which the color was observed for the sampled point in P.”
PO Resp. 14. To that end, Patent Owner argues, id., that the specification
provides the following additional definitions:
(10) INDEPENDENT VIEWING SYSTEM: an interactive
viewing system in which multiple viewers can freely,
independently of one another, and independently of the
source of the image data, pan that image data in all
directions with the effect that each viewer feels like they
are “inside” of that imagery, or present at the location
from which the imagery was produced, recorded, or
transmitted; and
(11) STANDARD COMPUTER GRAPHICS SYSTEM: a
computer graphics system which supports linear
perspective viewing, including the changing of the focal
length or the altering of the view angle, the apparent
rotation of viewed objects, and/or the apparent changing

2
We note that C is not a color; it is a set of colors. Similarly, V is not a ray;
it is a set of rays.
IPR2013-00548
Patent 7,542,035 B2

11
of direction of vision, and the texture mapping of image
data onto objects within the class of p-surface.
Ex. 1003, col. 7, l. 65–col. 8, l. 12. According to Patent Owner, these are
additional links in a “definitional chain” that we should incorporate into our
construction of full-surround image data. PO Resp. 14–16.
As noted above, the description of full-surround image data includes
that “[i]t should be mentioned at this point that full-surround image data is
useful in many fields of entertainment because, when delivered to many
viewers, it enables the construction of an independent viewing system
defined below.” Ex. 1003, col. 7, ll. 6–10. Patent Owner argues that this
language invokes the definition of “independent viewing system”
(definition 10). PO Resp. 15, n.3. Patent Owner further argues that an
independent viewing system is implemented on a standard computer graphic
system, invoking the definition of that term (definition 11). Id. At the
hearing, Patent Owner conceded that a standard computer graphics system,
the subject matter of definition 11 (Ex. 1003, col. 8, ll. 6–12), is not a
requirement of full-surround image data. Tr. 70:23–71:23. Rather, “this
was a construction that [Patent Owner] did in a different case.” Id. at 71:17–
18. Accordingly, we decline to read the requirement of a standard computer
graphics system into “full-surround image data.”
Regarding an independent viewing system, the subject matter of
definition 10 (Ex. 1003, col. 7, l. 65–col. 8, l. 5), Patent Owner also has
conceded that this is not a requirement of “full-surround image data.”
Tr. 71:21–73:20. Moreover, we conclude that the patent’s description of
IPR2013-00548
Patent 7,542,035 B2

12
full-surround image data enabling the construction of an independent
viewing system (Ex. 1003, col. 7, ll. 6–10) is an example of a result that can
be achieved using full-surround image data, not an additional restriction on
the scope of the term. See In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359,
1369 (Fed. Cir. 2004) (“We have cautioned against reading limitations into a
claim from the preferred embodiment described in the specification, even if
it is the only embodiment described, absent clear disclaimer in the
specification.”). Accordingly, we are not persuaded by Patent Owner’s
argument.
In sum, “full-surround image data” is “data which samples the points
P. This data encodes, explicitly or implicitly, the association of a single
color value with a given direction from a given point of projection.”

3. “projecting the [full-surround] image data onto the p-surface”
In our Decision to Institute, we construed “projecting the [full-
surround] image data onto the p-surface” as “generating a new image by
moving image pixels along rays from the view point to the p-surface.” Dec.
12–13. Patent Owner argues that the image data recited in the claim term
are not just any data, but are specifically “full-surround image data.” PO
Resp. 16–17. We agree with this contention, and, accordingly, clarify our
construction of “projecting the [full-surround] image data onto the p-
surface” to reflect that the image pixels are full-surround image data:
IPR2013-00548
Patent 7,542,035 B2

13
“generating a new image by moving image pixels of the full-surround image
data along rays from the view point to the p-surface.”

4. “view point” and “point of projection”
The Specification of the ’035 patent defines “view point” and “point
of projection” together:
(3) MAGIC POINT, VIEWPOINT, OR POINT OF
PROJECTION: Point VP. Please note, no matter how
points P are projected, their appearance will remain the
same when viewed from point VP. This latter concept
may best be understood by referring to FIGS. 4A and 4B.
Ex. 1003, col. 6, l. 65 – col. 7, l. 2. Patent Owner proposes construing “view
point” and “point of projection” to mean “[a] ‘Point VP’ such that ‘no matter
how points P are projected, their appearance will remain the same when
viewed from point VP.’” PO Resp. 12. In other words, Patent Owner
contends that the entire statement quoted above constitutes a definition of
“view point” and “point of projection.”
Petitioner contends that no construction is necessary, but that, if we do
construe these terms, they should mean “Point VP.” Reply 7, n.6.
3

3
Petitioner argues that Patent Owner’s proposed construction is “belated.”
PO Resp. 12. We disagree. Patent Owner is required to set forth its
arguments addressing Petitioner’s grounds for unpatentability in its Patent
Owner response, which it did. See 37 C.F.R. § 42.120(a). Patent Owner’s
Preliminary Response was not mandatory, see 37 C.F.R. § 42.107, and, thus,
Patent Owner was not required to set forth its claim construction positions in
that filing.
IPR2013-00548
Patent 7,542,035 B2

14
Petitioner does not explain why the remainder of the language of
definition (3), defining magic point, viewpoint, or point of projection, should
be omitted from our construction.
We understand the language “[p]lease note, no matter how points P
are projected, their appearance will remain the same when viewed from
point VP” (Ex. 1003, col. 6, ll. 66 – col. 7, l. 1) as expressing a necessary
consequence of the interrelationship of the ’035 patent’s embodiment-
independent predicate conditions and definitions of “POINTS P,” “A
PROJECTION OF P,” and “MAGIC POINT, VIEWPOINT, OR POINT OF
PROJECTION.” See Ex. 1003, col. 6, l. 46 – col. 7, l. 2. As such, that
language is properly considered definitional. Accordingly, we adopt Patent
Owner’s proposed construction of “view point” and “point of projection” as
“a point VP such that no matter how points P are projected, their appearance
will remain the same when viewed from point VP.”

B. Petitioner’s Motion to Exclude
Patent Owner supports its Response with an Expert Declaration of
James H. Oliver, Ph.D. (Ex. 2028, “Oliver Decl.”). Petitioner moves to
exclude ¶¶ 33, 41–44, 50, 55–57, and 67–72; Figures 5 and 6; and
Appendices B–D of the Oliver Declaration.
4
Paper 27, 3. Petitioner

4
Petitioner also moves to exclude portions of a Supplemental Oliver
Declaration (Ex. 1036) “to the extent Patent Owner files and/or seeks to
enter Patent Owner’s Response to Google’s Objections.” Paper 27 at 3, n.1.
Patent Owner did not file the identified paper.
IPR2013-00548
Patent 7,542,035 B2

15
contends that these portions of the Oliver Declaration rely on the
consideration of two software applications that are inadmissible because
they lack authentication and relevance. Id. at 1–2.
First, Petitioner contends that Dr. Oliver’s testimony at ¶¶ 41–44, 55,
and 57; Figure 5; and Appendix C of Exhibit 2028 relied upon software
purportedly corresponding to the system described in Photo VR. Id. at 3–4,
6–7. Petitioner contends that the date stamp of the software is in 1999, after
the 1996 date of Photo VR, and argues that Dr. Oliver did not confirm
adequately that this software is the same software discussed in Photo VR.
Id. at 6–7. Accordingly, Petitioner argues, this software and the portions of
Dr. Oliver’s testimony that rely upon it are irrelevant. Id. at 6.
Second, Petitioner contends that Dr. Oliver’s testimony at ¶¶ 33, 50,
56, and 67–72; Figure 6; and Appendices B and D of Exhibit 2028 relied
upon software code purportedly corresponding to the source code deposited
with the Patent Office with the application for the ’035 patent. Id. at 7–9.
Petitioner contends that this source code actually was modified by a third
party after the effective filing date of the ’035 patent. Id. Accordingly,
Petitioner argues, this software and the portions of Dr. Oliver’s declaration
that rely upon it are irrelevant. Id. at 9.
Petitioner further argues that any relevance of these two pieces of
software is outweighed by its potential to confuse and mislead. Id.
Petitioner also contends that this software and the testimony that relies upon
it lack authentication and are hearsay. Id. at 10–12.
IPR2013-00548
Patent 7,542,035 B2

16
Patent Owner responds that it does not seek to admit the Photo VR or
’035 patent software itself. Paper 30, 2. Patent Owner argues that
Dr. Oliver considered this software as part of his review of the technology at
issue in the proceeding. Id. Patent Owner points out, id. at 6, that the bases
for an expert’s opinion need not be admissible “[i]f experts in the particular
field would reasonably rely on those kinds of facts or data in forming an
opinion on the subject.” FED. R. EVID. 703.
As to the ’035 patent software, Patent Owner argues that Dr. Oliver
determined that it was functionally identical to that disclosed in the patent
application (and consistent with the patent’s disclosure) except that it was
modified slightly to work with a current operating system. Paper 30, 2–3, 6–
8. As to the Photo VR software, Patent Owner argues that it corroborated
the understanding Dr. Oliver gained from the Photo VR reference itself. Id.
at 3. According to Patent Owner, Dr. Oliver had the knowledge and
experience necessary to authenticate the software by comparing its functions
with what they purport to be. Id. at 6 (citing FED. R. EVID. 901(b)(3) (listing
“[a] comparison with an authenticated specimen by an expert witness” as an
example of evidence that satisfies the authentication requirement)).
We are not persuaded that Dr. Oliver’s testimony should be excluded.
Rather, an expert reasonably could rely upon later versions of software to
gain a general understanding of the technology at issue in documents that
describe earlier versions of such software. Petitioner has not persuaded us
that Patent Owner’s purported failure to establish an identity between the
IPR2013-00548
Patent 7,542,035 B2

17
software Dr. Oliver actually reviewed and the software referenced in Photo
VR and the ’035 patent makes such reliance unreasonable in this instance.
Petitioner’s arguments instead go toward the weight we should give
Dr. Oliver’s testimony. Cf. Smith v. Ford Motor Co., 215 F.3d 713, 718
(7
th
Cir. 2000) (“The soundness of the factual underpinnings of the expert’s
analysis and the correctness of the expert’s conclusions based on that
analysis are factual matters to be determined by the trier of fact.”).

C. Anticipation by Photo VR
Petitioner contends that claims 1, 22, 53, 54, 69, 73, 78, 80, 82–84,
89, 98, 102, 105, 113, 115, and 118–121 are anticipated by Photo VR.
Pet. 33–41, 49–51, 53–60.

1. Photo VR
Photo VR is prior art to each of the challenged claims under 35 U.S.C.
§ 102(b) because its publication date of July 30–31, 1996, precedes the
effective filing date of the ’035 patent by more than one year. Photo VR is
also prior art to those claims under 35 U.S.C. § 102(a) because Patent Owner
does not allege an invention date earlier than the publication date of
Photo VR.
Photo VR is directed to panoramic view rendering “by generating a
sphere-like polyhedral environment map from photo-realistic images and
using the generated maps to render the scene by techniques of computer
IPR2013-00548
Patent 7,542,035 B2

18
graphics.” Ex. 1007, 397 (col. 1). Photo VR illustrates its method in the
context of rendering a scene in a room by positioning a camera in a “proper
position, such as the center of the room,” from which images of the entire
view are taken and arranged as a “sphere-like polyhedron consisting of
textured trapezoids.” Id. at 397 (col. 2). Because the center of projection of
the images moves slightly as the camera is panned, Photo VR assumes that
the objects in the scene are sufficiently far from the camera that the effects
of any movement are negligible. Id. at 398 (col. 1). “Thus all optic axes of
the images can be regarded as intersected at the [center of projection].” Id.
The photographic images are registered to polygons of a polyhedron.
Id. at 398 (col. 2). For example, images can be registered to trapezoids or
triangles of a sphere-like polyhedron. Id. at 399 (col. 1). After the images
are registered, a texture mapped polyhedron is generated. Id. at 398 (col. 2).
The figure from page 398 of Photo VR, reproduced below, illustrates an
example:

IPR2013-00548
Patent 7,542,035 B2

19
The figure from page 398 illustrates the generation of a “texture mapped
sphere-like polyhedron” by ray-casting of original images onto polygons
arranged in the space by their registrations. Id. at 398 (col. 2).
Photo VR also describes an “object viewer,” as illustrated in a
drawing from page 400, reproduced below:

The drawing from page 400 illustrates that the object viewer allows a
user to “interactively observe an object from different views in real time.”
Id. at 400 (col. 1). That is, images “A,” “B,” and “C” in the drawing show
images taken of the same “object” from different views. Id.

2. Anticipation of Claims 1, 22, 53, 54, 69, 73, 78, 80,
82–84, 89, 98, 102, 105, 113, 115, and 118–121

We have reviewed the evidence presented by Petitioner, including the
claim charts in the Petition (Pet. 33–60) and the Declaration of John R.
Grindon, D.Sc. (Ex. 1005). Notwithstanding Patent Owner’s arguments, we
are persuaded that Photo VR anticipates claims 1, 22, 53, 54, 69, 73, 78, 80,
82–84, 89, 98, 102, 105, 113, 115, and 118–121. For example, Petitioner
IPR2013-00548
Patent 7,542,035 B2

20
has shown sufficiently that Photo VR discloses modeling of the visible
world using “full-surround image data” in that it acquires images of “the
whole view from the camera position.” Ex. 1007, 397 (col. 2). Likewise,
Petitioner has shown that the sphere-like polyhedron disclosed by Photo VR
is a “p-surface” (in fact, a “p-sphere”) because it provides a computer-
graphics representation of a surface having a well-defined inside and
outside, with at least one point inside the surface capable of connection to
every point of the surface with a distinct line segment, wherein no portion of
the line segment lies outside the surface or intersects the surface at a point
that is not an endpoint. Pet 18–19, 38. Petitioner also has shown that Photo
VR discloses “texture mapping” of image data onto the p-surface because
the mapping it describes applies image data generated by taking images from
the camera position, e.g., at the center of a room (i.e., from a view point)
onto polygons defined on the p-surface. Ex. 1007, 397 (col. 2). We address
Patent Owner’s arguments below.

a. Photo VR Discloses “full-surround image data”
Patent Owner contends that a single image, such as one of the images
acquired by the camera disclosed in Photo VR, is not full-surround image
data. PO Resp. 20–21. Petitioner does not contend, however, that a single
image constitutes full-surround image data, nor did we make a preliminary
finding to that effect in our Decision to Institute. See Pet. 17, 38; Dec. 8–9;
IPR2013-00548
Patent 7,542,035 B2

21
Tr. 87:4–8. Rather, as explained in Section II.A.2, above, full-surround
image data are data which sample the points P.
Patent Owner further contends that full-surround image data must
have a single common view point from which the points P of the visible
world are observed. PO Resp. 18–19. By contrast, Patent Owner argues,
Photo VR discloses acquiring images from a camera with a moving center of
projection. Id. at 19 (citing Ex. 1007, 398 (col. 1)). Patent Owner points out
that Photo VR states that “it is almost impossible for some images to be
registered without objects on the border being duplicated or lost in adjacent
images because the camera was moved during panning.” PO Resp. 19
(quoting Ex. 1007, 400 (col. 2)).
Petitioner responds that Photo VR does disclose a single common
view point from which the points P are observed. Reply 6–8. As Petitioner
points out, Photo VR discloses that the camera is positioned in the same
place (e.g., the center of the room) for each of the photographs in a set and
that, for the acquired images, “all optic axes of the images can be regarded
as intersected at the [center of projection].” Id. at 7–8 (citing Ex. 1007, 398
(col. 1)). According to Petitioner, the purported deficiency to which Patent
Owner cites simply is an acknowledgment in Photo VR that there are
practical challenges to capturing multiple images from a common view
point. Reply 8. Patent Owner’s declarant, Dr. Oliver, admits that the
objective of Photo VR is to maintain the same center of projection for each
of the images. Ex. 2030, 113:4–21. Once the images are captured,
IPR2013-00548
Patent 7,542,035 B2

22
Petitioner argues, the generated texture map in Photo VR has a common
view point. Reply 8.
We agree with Petitioner. Photo VR regards each of the acquired
images as having a common center of projection. Ex. 1007, 398 (col. 1). As
a result, Photo VR explains, when the images acquired with slightly different
centers of projection are registered to the same center of projection, some
objects on the borders of the images may be duplicated or lost. Id. at 400
(col. 2). Thus, although the quality of some of the images might be
degraded, all of the acquired data are registered to the same center of
projection.
This is not meaningfully different from the embodiment described in
the ’035 patent, for example, in which a texture map is “built from two
pictures, respectively, taken with a fisheye lens,” together comprising a
“pictosphere.” Ex. 1003, col. 8, ll. 59–61. Because the camera must move
to take the second image, there will be a difference, however subtle, between
the centers of projection of the acquired images. Patent Owner directs us to
U.S. Patent No. 5,903,782 (Ex. 1019, “the ’782 patent”), incorporated by
reference into the ’035 patent (see Ex. 1003, col. 9, ll. 63–67), as disclosing
a technique for taking multiple photographs from exactly the same view
point. Tr. 56:22–57:9. In the embodiment disclosed in the ’782 patent, a
first image is taken, using a fisheye lens, after which the camera is rotated
180 degrees and a second image is taken. Ex. 1019, col. 3, ll. 31–35.
Because the camera moves, the view points of the two images cannot be in
IPR2013-00548
Patent 7,542,035 B2

23
exactly the same place. As Patent Owner admits, there will be some
“tolerance.” Tr. 57:7–23. Moreover, Patent Owner concedes that full-
surround image data are not limited to data acquired using fisheye lenses.
Tr. 58:15–22.
Patent Owner also contends that Photo VR’s registered image data are
not full-surround image data because the individual images, when mapped to
polygons, may overlap one another. PO Resp. 19–20. According to
Photo VR, a texture mapped sphere-like polyhedron is generated by ray
casting of the original images onto the polygons of the polyhedron according
to the registrations of the images, as shown in the figure from page 398 of
Photo VR (reproduced above). Ex. 1007, 398 (col. 2). Prior to Photo VR,
where images overlapped, typical ray casting involved using the color from
the first image hit by the ray. Id. at 399 (col. 1). In Photo VR, in contrast,
the colors for all images hit by the ray are averaged, using a weighted
average to determine the color for that ray. Id. Patent Owner argues that
this is not full-surround image data because there is no unique
correspondence of color, ray, and viewpoint. PO Resp. 19–20.
In response, Petitioner argues that an averaged color value is a single
unique color value corresponding to a ray and viewpoint. Reply 9–10.
Petitioner points to the testimony of Patent Owner’s declarant, Dr. Oliver,
that the averaged color value is a single result approximating two colors.
Id. (citing Ex. 2030, 122:17–123:6). We are persuaded by Petitioner that the
resultant color disclosed by Photo VR, which is a weighted average of two
IPR2013-00548
Patent 7,542,035 B2

24
overlapping colors, is a single color value corresponding to a ray from a
view point (here, the center of projection). Ex. 1007, 399 (col. 1). The data
that ultimately are texture mapped to the polyhedron contain averaged
values in places where the originally acquired data would have overlapped.
5

Patent Owner concedes that, except for the fact that the original images were
not acquired from the same view point (an argument we reject above), the
data actually mapped onto Photo VR’s polyhedron would be considered full-
surround image data. Tr. 67:20–68:13.
In sum, we are persuaded that Petitioner has shown that Photo VR
discloses that the data texture mapped to the polyhedron encodes, explicitly
or implicitly, the associations of unique color values with given directions
from a given point of projection, i.e., are full-surround image data.

5
In response to our questioning at oral hearing, Patent Owner argued that
the color-averaging process itself is part of Photo VR’s texture-mapping
procedure and that this texture mapping is not performed on full-surround
image data because full-surround image data only result after the color
averaging. Tr. 68:14–69:2. This argument was not made previously by
Patent Owner. In any case, we are not persuaded by this argument.
Photo VR discloses texture mapping as performed with standard “rendering
packages, libraries or graphic hardware accelerators.” Ex. 1007, p. 397,
col. 2. Such standard rendering procedures are distinct from the color-
averaging process in Photo VR’s overall procedure. Thus, even if the color-
averaging process of Photo VR is a texture mapping, Photo VR discloses a
distinct texture mapping that is performed on full-surround image data. See
Tr. 88:9–89:17.
IPR2013-00548
Patent 7,542,035 B2

25

b. Photo VR Discloses “projecting the [full-surround]
image data onto the p-surface from [a] view point”

Patent Owner contends that Photo VR does not disclose texture
mapping full-surround image data to generate a texture map “substantially
equivalent to projecting the [full-surround] image data onto the p-surface
from [a] view point,” as recited in independent claim 1, and as similarly
recited in independent claims 22 and 53. PO Resp. 21–22. Specifically,
Patent Owner argues that the data projected in Photo VR is not full-surround
image data. For the reasons given in Section II.C.2.a, we disagree with this
argument.
Patent Owner also argues that Photo VR describes specular lighting as
important but that Photo VR does not handle specular lighting. PO Resp.
21–22, n.6. Patent Owner relates this observation to an argument that
Photo VR’s method of capturing and projecting linear images introduces
distortion, contending that the claims of the ’035 patent require “distortion-
free results.” Id. The claims do not recite “distortion-free results.”
Moreover, as we explain in Section II.A.2, the term “full-surround image
data” does not require “the sampled points sufficient to provide in a standard
3D computer graphics system a display in which changing the direction of
view provides a human viewer the impression of being present at the point
of projection from which the color was observed for the sampled point in P,”
as argued by Patent Owner. See PO Resp. 14. Thus, we are not persuaded
IPR2013-00548
Patent 7,542,035 B2

26
that the claims require distortion-free results. Accordingly, Petitioner has
shown that Photo VR discloses “projecting the full-surround image data onto
the p-surface from a point of projection.”

c. Photo VR Discloses Selecting a Direction of View
Independent claim 1 requires “selecting a direction of view within the
p-surface,” and independent claims 22 and 53 each require “selecting a
direction of view from a view point.” Patent Owner argues that these
limitations are not disclosed in Photo VR because the “Photo VR ‘Object
Viewer’ applies to [an] object, not [a] p-surface.” PO Resp. 28, 29.
6
Patent
Owner makes this same argument for dependent claims 78, 80, and 83.
Id. at 31–32. Patent Owner also argues that the Object Viewer described in
Photo VR is “inapposite” because it “does not appear to have any connection
with a texture-mapped p-surface and should be disregarded.” Id. at 22.
It is unclear what Patent Owner contends is lacking in Photo VR when
compared with the claims. Nevertheless, as Petitioner points out, Photo VR
describes several instances of a user selecting a direction of view. Pet. 16–
17, 34, 50, 53; Reply 11–13. For example, Figures 5(1)–5(8) (Ex. 1007,
p. 403) illustrate images rendered at various viewing angles. Photo VR
describes a user selecting a direction of view using the Object Viewer.
Ex. 1007, 400 (col. 1). The testimony of Patent Owner’s declarant,
Dr. Oliver, further implies that a user of the Photo VR system selects a

6
Patent Owner omits this assertion from its chart for claim 53.
IPR2013-00548
Patent 7,542,035 B2

27
direction of view from a view point on a model. Ex. 2028 ¶ 57 (“user selects
a viewpoint near the center of projection”); id. (“[Photo VR] suggests
navigating through adjacent overlapping spherical environment maps”). We
also agree with Petitioner that Photo VR describes the Object Viewer as a
component of the same Windows 95–based system that contains the other
described components, such as the module that generates texture-mapped
polyhedrons. Id. at p. 400, cols. 1–2. Thus, Patent Owner has not persuaded
us that the Object Viewer is “inapposite.” Accordingly, Petitioner has
shown that Photo VR discloses “allowing a user to select a direction of view
from a view point on the model.”

d. Claims 1, 22, 53, 54, 69, 73,78, 80, 82–84,
89, 98, 102, 105, 113, 115, and 118–121
Are Anticipated by Photo VR

In sum, based on our consideration of the evidence presented by
Petitioner and the arguments detailed above, we are persuaded that Petitioner
has proved by a preponderance of the evidence that independent claims 1,
22, and 53 are anticipated by Photo VR. Having reviewed Petitioner’s
evidence of unpatentability for dependent claims 54, 69, 73, 78, 80, 82–84,
89, 98, 102, 105, 113, 115, and 118–121, we also conclude that Petitioner
has proved by a preponderance of the evidence that those claims are
anticipated by Photo VR.

D. Obviousness Over Photo VR and QuickTime
®
VR
IPR2013-00548
Patent 7,542,035 B2

28
Petitioner contends that claims 84 and 86 (which depend from claim
54), claims 120 and 123 (which depend from independent claim 53), and
claim 121 (which depends from claim 120) would have been obvious under
35 U.S.C. § 103(a) over Photo VR and QuickTime
®
VR. Pet. 57–60. As
discussed above, Petitioner relies on Photo VR to meet all features of
independent claim 53 and dependent claim 54. It relies on QuickTime
®
VR
to account for the additional features recited in claims 84, 86, 120, 121, and
123.

1. QuickTime
®
VR
QuickTime
®
VR discloses the use of 360-degree cylindrical
panoramic images to compose a virtual environment that allows users to
pan, zoom, and navigate a scene. Ex. 1012, 29 (col. 1); 32 (col. 1).
QuickTime
®
VR explicitly states that one “constraint with the current
panoramic player is its limitation in looking straight up or down due to the
use of cylindrical panoramic images. This limitation can be removed if
other types of environment maps, such as cubic or spherical maps, are
used.” Id. at 36 (col. 2) (emphasis added). Conversely, Photo VR explicitly
notes that its spherical panoramic rendering of a real-world environment “is
different from the QuickTime VR approach,” a consequence both of its use
of spherical rendering and of its use of texture mapping instead of image
warping. Ex. 1007, 397 (col. 1).

IPR2013-00548
Patent 7,542,035 B2

29
2. Combination of Photo VR and QuickTime
®
VR
Photo VR states that
[systems employing a cylindrical environment map] have a
common problem: the environment map used cannot cover the
top view and the bottom view, so the vertical viewing angle is
limited. In the following, we will propose a better method that
renders the scene by sphere-like polyhedral environment maps,
and thus the problem is solved.

Ex. 1007, 397 (col. 1). Petitioner reasons that Photo VR thus “explicitly
improves” upon QuickTime
®
VR’s lack of mapping images to the top and
bottom of the cylinder. Pet. 20. This reasoning is supported by the
testimony of Petitioner’s declarant, Dr. Grindon. Ex. 1005 ¶ 127. Petitioner
asserts that both references are reasonably pertinent to the same problem and
that they are in the same field of endeavor of “image processing/computer
graphics.” Reply 13. We are persuaded that Petitioner has demonstrated
that one of skill in the art would have combined the teachings of Photo VR
and QuickTime
®
VR.
Patent Owner contends that “[a]n important consideration in
combining references is: which is the primary reference,” and that “[n]o
rationale is given by Petitioner for why one of ordinary skill in the art,
presented with QuickTime VR, would have any reason to look to Photo
VR.” PO Resp. 24–25. We are not persuaded by these contentions as we do
not consider the order in which prior art is applied in a challenge to a claim
to be significant. See, e.g., In re Mouttet, 686 F.3d 1322, 1333 (Fed. Cir.
IPR2013-00548
Patent 7,542,035 B2

30
2012) (“[W]here the relevant factual inquiries underlying an obviousness
determination are otherwise clear, characterization by the examiner of prior
art as ‘primary’ and ‘secondary’ is merely a matter of presentation with no
legal significance.” (citation omitted)). In light of Petitioner’s explanation
that Photo VR explicitly refers to QuickTime
®
VR and teaches an
improvement over it, we conclude that Petitioner has provided a reason with
rational underpinning to make the combination.
Patent Owner also draws a distinction between image processing and
computer graphics, contending that those fields were separate at the time of
the invention. PO Resp. 10–11. But Patent Owner has not explained how
this distinction relates to combining the teachings of Photo VR and
QuickTime
®
VR, and has not contended that those references are drawn
from different ones of those fields. Instead, Patent Owner argues that
“accepted wisdom in image processing in 1997 was that computational
resources were limited, and (in general) should not be wasted,” and that
additional image processing steps such as described in the ’035 patent would
have been counterintuitive. Id. at 25–26. We are not persuaded by this
argument, which is presented generically, without specifically addressing the
content of Photo VR and QuickTime
®
VR. In particular, we are persuaded
by Petitioner’s observation that Photo VR discusses QuickTime
®
VR and
proposes improvements over it, an observation that Patent Owner does not
rebut adequately.

IPR2013-00548
Patent 7,542,035 B2

31
3. Obviousness of Claims 84, 86, 120, 121, and 123
Claims 84 and 120 each recite “using the method to generate a
plurality of p-surfaces using different respective sets of full-surround image
data”; claims 86 and 123 each recite “enabling multiple users using
independent viewing systems to independently cause to be displayed and to
view any selected portion of the texture-mapped p-surface”; and claim 121
recites “further comprising linking the plurality of p-surfaces in such a
manner as to enable a user to hop amongst the plurality of p-surfaces to
simulate a tour thereof.”
We have reviewed the disclosures of QuickTime
®
VR specifically
identified by Petitioner’s claim charts. See Pet 57–60. QuickTime
®
VR
discloses that “[m]ultiple environment maps can be linked together to define
a scene. The user may move in the scene by jumping through the maps.”
Ex. 1012, 36 (col. 1). We are persuaded that this disclosure teaches the
generation of a plurality of p-surfaces, as recited in claims 84 and 120. In
addition, the ability to “jump[] through the maps” teaches the linking of p-
surfaces to enable a user to hop amongst the p-surfaces as recited in claim
121. See id. Fig. 2; 36 (col. 1).
QuickTime
®
VR discloses that “[a]nother use of the orientation-
independent movie is in interactive TV. A movie can be broadcast in a 360-
degree format, perhaps using multiple channels. Each TV viewer can
control freely the camera angle locally while watching the movie.”
Ex. 1012, 35 (col. 2). We are persuaded that the combination of Photo VR
IPR2013-00548
Patent 7,542,035 B2

32
with this disclosure reasonably suggests enabling multiple users using
independent viewing systems independently to cause to be displayed, and to
view, any selected portion of the texture-mapped p-surface, as recited in
claims 86 and 123.
Patent Owner asserts that these limitations are not disclosed by
Photo VR or QuickTime
®
VR “singly or in any combination.” PO Resp.
26–27. But Patent Owner does not provide sufficient reasoning that
addresses these findings.
Thus, based on our consideration of the evidence presented by
Petitioner and the arguments detailed above, we are persuaded that Petitioner
has proved by a preponderance of the evidence that claims 84, 86, 120, 121,
and 123 would have been obvious over Photo VR and QuickTime
®
VR.

III. CONCLUSION
Petitioner has demonstrated by a preponderance of the evidence that
(1) claims 1, 22, 53, 54, 69, 73, 78, 80, 82–84, 89, 98, 102, 105, 113, 115,
and 118–121 are anticipated under 35 U.S.C. §§ 102(a) and 102(b) by Photo
VR; and (2) claims 84, 86, 120, 121, and 123 would have been obvious
under 35 U.S.C. § 103(a) over Photo VR and QuickTime
®
VR.

IV. ORDER
For the reasons given, it is
IPR2013-00548
Patent 7,542,035 B2

33
ORDERED that, based on a preponderance of the evidence, claims 1,
22, 53, 54, 69, 73, 78, 80, 82–84, 86, 89, 98, 102, 105, 113, 115, 118–121,
and 123 of U.S. Patent No. 7,542,035 B2 are held to be unpatentable;
FURTHER ORDERED that Petitioner’s motion to exclude ¶¶ 33, 41–
44, 50, 55–57, and 67–72; Figures 5 and 6; and Appendices B–D of Exhibit
2028 is denied; and
FURTHER ORDERED that, because this is a final written decision,
parties to this proceeding seeking judicial review of our decision must
comply with the notice and service requirements of 37 C.F.R. § 90.2.

PETITIONER:

Cono A. Carrano
Ruben Munoz
ccarrano@akingump.com
rmunoz@akingump.com

PATENT OWNER:

Stephen Chow
Hsuanyeh Chang
Robert Groover
Seth Horwitz
schow@burnslev.com
hchang@burnslev.com
groover@technopatents.com
shorwitz6576@gmail.com