Weibin Pan et al.

Patent Trials and Appeals Board

Mar 24, 2020

14250193 - (D) (P.T.A.B. Mar. 24, 2020)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/250,193 04/10/2014 Weibin Pan GGL-629PTUSCON 5114
100462 7590 03/24/2020
Dority & Manning P.A. and Google LLC
Post Office Box 1449
Greenville, SC 29602
EXAMINER
RETALLICK, KAITLIN A
ART UNIT PAPER NUMBER
2482
NOTIFICATION DATE DELIVERY MODE
03/24/2020 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
usdocketing@dority-manning.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte WEIBIN PAN and LIANG HU
Appeal 2018-002981
Application 14/250,193
Technology Center 2400
Before DAVID M. KOHUT, BARBARA A. BENOIT, and
LYNNE E. PETTIGREW, Administrative Patent Judges.
PETTIGREW, Administrative Patent Judge.
DECISION ON APPEAL
STATEMENT OF THE CASE
Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the Examiner’s
decision to reject claims 1, 2, 4–9, and 11–24. We have jurisdiction under
35 U.S.C. § 6(b).
We AFFIRM.

1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R.
§ 1.42(a). Appellant identifies the real party in interest as Google Inc. Appeal
Br. 3.
Appeal 2018-002981
Application 14/250,193

2

CLAIMED SUBJECT MATTER
The claims are directed to sensor corrections for portable devices with image
input where sensor corrections are generated based on the image input. Spec. ¶ 16.
Claim 1, reproduced below, is illustrative of the claimed subject matter:
1. A portable device comprising:
a sensor;
a video capture module;
a processor; and
a computer-readable memory that stores instructions thereon,
wherein the instructions, when executed by the processor,
operate to:
cause the sensor to generate raw sensor data indicative of
a physical quantity,
cause the video capture module to capture video imagery
of a reference object concurrently with the sensor
generating raw sensor data when the portable device is
moving relative to the reference object, wherein the
captured video imagery comprises a plurality of
consecutive images of the reference object;
cause the processor to infer a three-dimensional (3D)
structure of the reference object from the captured video
imagery;
cause the processor to calculate one or more position or
orientation fixes associated with the portable device based
at least in part on the 3D structure of the reference object;
and
cause the processor to calculate correction parameters for
the sensor based at least in part on the one or more position
or orientation fixes and the raw sensor data.

Appeal 2018-002981
Application 14/250,193

3

REFERENCES
The prior art relied upon by the Examiner is:
Name Reference Date
Ladetto et al. US 2003/0018430 A1 Jan. 23, 20003
Camp et al. US 2005/0181810 A1 Aug. 18, 2005
Ohta US 2009/0326850 A1 Dec. 31, 2009
Watanabe et al. US 2010/0030471 A1 Feb. 04, 2010
Katsman et al. US 2010/0157061 A1 June 24, 2010
Chen et al. US 2011/0149094 A1 June 23, 2011
Zhang et al. US 2011/0178708 A1 July 21, 2011
Solem et al. US 2013/0300830 A1 Nov. 14, 2013

REJECTIONS
The Examiner rejected claims 1, 4–9, 13–15, 17–19, and 24 under 35 U.S.C.
§ 103(a) as unpatentable over Chen, Solem, and Zhang. Final Act. 4–19.
The Examiner rejected claims 2 and 16 under 35 U.S.C. § 103(a) as
unpatentable over Chen, Solem, Zhang, and Ohta. Final Act. 19–20.
The Examiner rejected claim 11 under 35 U.S.C. § 103(a) as unpatentable
over Chen, Solem, Zhang, and Ladetto. Final Act. 20–21.
The Examiner rejected claim 12 under 35 U.S.C. § 103(a) as unpatentable
over Chen, Solem, Zhang, and Katsman. Final Act. 21–22.
The Examiner rejected claims 20, 22, and 23 under 35 U.S.C. § 103(a) as
unpatentable over Chen, Solem, Zhang, Ohta, and Watanabe. Final Act. 22–26.
The Examiner rejected claim 21 under 35 U.S.C. § 103(a) as unpatentable
over Chen, Solem, Zhang, Ohta, Watanabe, and Camp. Final Act. 26–27.

Appeal 2018-002981
Application 14/250,193

4

ANALYSIS
The Examiner relies on the combination of Chen, Solem, and Zhang in
concluding that claim 1 would have been obvious. Final Act. 6–8; Ans. 2–5. The
Examiner finds that Chen teaches a device with a sensor (accelerometer) that
captures photographs/video of a reference object as the device moves relative to
the object where the sensor generates raw sensor data (velocity or acceleration)
that can be used to calculate correction parameters to correct tilt or distortion in the
captured image/video. Final Act. 5–6; Ans. 2–3; Chen ¶¶ 24–26, 28, 32, 35.
The Examiner finds that Chen does not explicitly teach inferring a 3D
structure of the reference object from the captured photographs/video or
calculating position or orientation fixes based at least in part on that 3D structure.
Final Act. 6; Ans. 3–4. However, the Examiner finds that Solem teaches imaging a
reference object, determining the 3D representation of the object, and using
geodata for the image and the 3D representation to calculate the accurate position
and orientation (position and orientation fixes) for the imaging device. Final Act.
6–7; Ans. 4, 25–27; Solem ¶¶ 39, 43. According to the Examiner, it would have
been obvious to modify the teachings of Chen with the teachings of Solem to
determine the 3D position and orientation of the imaging device. Final Act. 7;
Ans. 5.
The Examiner further finds that Chen does not explicitly teach calculating
correction parameters for the sensor based on the position or orientation fixes and
the raw sensor data. Final Act. 6; Ans. 4. The Examiner finds that Zhang teaches
using position and orientation of a camera to calculate the orientation of sensors in
a navigation system in order to initialize (calibrate/correct) the navigation
system—and the sensors in the system—before using the system. Final Act. 7;
Ans. 4–5, 27–31, 40–43; Zhang ¶¶ 34–37, 40, 41; fig.4; see also Zhang fig.10.
Appeal 2018-002981
Application 14/250,193

5

According to the Examiner, it would have been obvious to modify the teachings of
Chen and Solem with Zhang’s teaching to initialize the sensor to improve the
system. Final Act. 8; Ans. 5.
Portable device is moving relative to the reference object
Claim 1 recites “cause the video capture module to capture video imagery of
a reference object concurrently with the sensor generating raw sensor data when
the portable device is moving relative to the reference object, wherein the captured
video imagery comprises a plurality of consecutive images of the reference
object.” Appeal Br. 19. Appellant contends that Chen does not teach this
limitation. Id. at 9–10. In support of its assertion, Appellant cites to and quotes
sections of paragraphs 24 and 25 of Chen. Id. Appellant’s argument is conclusory
and does not persuade us that the Examiner erred.
As Appellant notes, Chen states that “the object being photographed and the
image capture device may each include global positioning system (GPS) devices
such that the relative GPS orientation of the object and the image capture device
may be recorded along with the image data.” Chen ¶ 25; see Appeal Br. 9. Chen
further states that “although this disclosure may focus on still images, the concepts
disclosed herein equally apply to recording moving images and/or video.” Chen
¶ 25; see Final Act. 5. Thus, the Examiner finds, and we agree, that Chen teaches
the orientation data and position of the portable device is determined relative to the
object imaged, whether the imaging device is still or moving. Ans. 32; Chen ¶ 32;
see Final Act. 5; Chen ¶ 25. The Examiner also finds that video comprises
consecutive images and further relies on Solem’s teaching of video frames to teach
“video imagery comprises a plurality of consecutive images,” as recited in claim 1.
Final Act. 5–6; Ans. 31–33; Solem ¶ 43. Therefore, the Examiner finds that Chen,
supported by Solem, teaches “cause the video capture module to capture video
Appeal 2018-002981
Application 14/250,193

6

imagery of a reference object concurrently with the sensor generating raw sensor
data when the portable device is moving relative to the reference object, wherein
the captured video imagery comprises a plurality of consecutive images of the
reference object.” Final Act. 5–6; Ans. 31–33. We do not find that the Examiner
erred.
Position or orientation fixes and 3D structure
Claim 1 recites “cause the processor to calculate one or more position or
orientation fixes associated with the portable device based at least in part on the 3D
structure of the reference object.” Appeal Br. 19.
Appellant agrees that Solem teaches using existing image geodata and 3D
representation data of noteworthy locations based on the geodata to determine the
position and orientation of the image device used to capture the image. Appeal
Br. 11. However, Appellant contends that the position and orientation of the image
device are not the claimed “position and orientation fixes associated with the
portable device.” Appeal Br. 11.
Appellant’s argument is unpersuasive. As the Examiner notes, Appellant’s
Specification states “a position fix identifies the geographic location of the portable
device and an orientation fix identifies the orientation of the portable device.”
Ans. 33 (quoting Spec. ¶ 16). The Examiner finds, and Appellant agrees, that
Solem teaches determining a 3D spatial (i.e., geographic) location and an
orientation of an imaging device by recognizing objects in a captured image and
matching the image to a 3D representation in a database. Ans. 34; Appeal Br. 11;
Solem ¶ 30; see Solem ¶¶ 33, 35, 43. Thus, the Examiner finds that Solem teaches
determining a position fix (geographic location of the portable device) and an
orientation fix (orientation of the portable device) based at least in part on the 3D
structure of a reference object. We do not find that the Examiner erred.
Appeal 2018-002981
Application 14/250,193

7

Calculate correction parameters for the sensor
Claim 1 recites “cause the processor to calculate correction parameters for
the sensor based at least in part on the one or more position or orientation fixes and
the raw sensor data.” Appeal Br. 19.
Appellant contends that neither Chen nor Zhang teaches this limitation.
Appeal Br. 8–9, 11–12. Appellant asserts that Chen teaches using orientation data
(correction parameters) to correct the photographs/video, not to correct the sensor
as required in claim 1. Id. at 9. Appellant further asserts that Zhang teaches using
positional and orientation information encoded in an imaged object to initialize the
position and orientation of an inertial navigation system, but does not teach
calculating correction parameters based on the position or orientation fixes as
required in claim 1. Id. at 12.
Appellant’s argument is unpersuasive because it addresses Chen and Zhang
individually, whereas the Examiner relies on a combination of Chen, Solem, and
Zhang to teach the disputed limitation. Appellant cannot establish nonobviousness
by attacking references individually when the rejection is based on a combination
of references. See In re Merck & Co., 800 F.2d 1091, 1097 (Fed. Cir. 1986); In re
Keller, 642 F.2d 413, 425 (CCPA 1981). The Examiner finds that Chen teaches an
imaging device with a sensor where the sensor generates raw sensor data, which
can be used to calculate correction parameters to correct tilt or distortion in the
captured image/video. Final Act. 5–6; Ans. 2–3; Chen ¶¶ 24–26, 28, 32, 35. The
Examiner modifies Chen with the 3D spatial location determination of Solem to
determine the 3D position and orientation of the image device. Final Act. 7. The
Examiner further modifies the combination of Chen and Solem with Zhang’s
teaching to use position and orientation data to initialize and update a system
(including a sensor) and not just correct the captured image. Final Act. 8. Thus,
Appeal 2018-002981
Application 14/250,193

8

the Examiner finds the combination of Chen, Solem, and Zhang teaches “cause the
processor to calculate correction parameters for the sensor based at least in part on
the one or more position or orientation fixes and the raw sensor data,” as recited in
claim 1. Final Act. 5–7; Ans. 27–31, 40–43; Chen ¶¶ 24–26, 28, 32, 35; Zhang
¶¶ 34–37, 40, 41; fig.4. Appellant’s assertions that Chen and Zhang individually
do not disclose the disputed limitation fail to address the Examiner’s proposed
combination of Chen, Solem, and Zhang. Therefore, Appellant has not persuaded
us that the Examiner erred.
For these reasons, we are not persuaded of error and sustain the Examiner’s
rejection of independent claim 1. Independent claims 13 and 19 recite similar
limitations as claim 1, and Appellant presents essentially the same arguments for
them. Appeal Br. 13–14. Appellant also repeats essentially the same arguments
for dependent claims 4–9, which depend from independent claim 1; dependent
claims 14, 15, 17, 18, which depend from independent claim 13; and dependent
claim 24, which depends from independent claim 19. Id. at 14–15. Therefore, we
sustain the Examiner’s obviousness rejection of claims 4–9, 13–15, 17–19, and 24
over Chen, Solem, and Zhang. We also sustain the Examiner’s obviousness
rejections, based on various combinations of references, of dependent claims 2, 11,
12, 16, and 20–23, as Appellant repeats essentially the same arguments for those
dependent claims as well. Id. at 15–17.
CONCLUSION
The Examiner’s rejections are affirmed.
Appeal 2018-002981
Application 14/250,193

9

DECISION SUMMARY
In summary:
Claims
Rejected
35 U.S.C. § References Affirmed Reversed
1, 4–9, 13–15,
17–19, 24
103 Chen, Solem,
Zhang
1, 4–9, 13–15,
17–19, 24

2, 16 103 Chen, Solem,
Zhang, Ohta
2, 16
11 103 Chen, Solem,
Zhang, Ladetto
11
12 103 Chen, Solem,
Zhang, Katsman
12
20, 22, 23 103 Chen, Solem,
Zhang, Ohta,
Watanabe
20, 22, 23
21 103 Chen, Solem,
Zhang, Ohta,
Watanabe,
Camp
21
Overall
Outcome
1, 2, 4–9, 11–
24

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with this
appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED