Ex Parte Engedal

Patent Trial and Appeal Board

Dec 7, 2017

13900158 (P.T.A.B. Dec. 7, 2017)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/900,158 05/22/2013 Torbjorn Engedal P28478USC1
(119-1099USC1)
1015
61947 7590 12/12/2017
Ar>r>le - Rlank Rome.
EXAMINER
c/o Blank Rome LLP SLATER, ALISON T
717 Texas Avenue, Suite 1400
HOUSTON, TX 77002 ART UNIT PAPER NUMBER
2487
NOTIFICATION DATE DELIVERY MODE
12/12/2017 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
mbrininger @ blankrome. com
hou stonpatents @ blankrome .com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte TORBJORN ENGEDAL
Appeal 2017-008323
Application 13/900,1581
Technology Center 2400
Before CARLA M. KRIVAK, HUNG H. BUI, and JON M. JURGOVAN,
Administrative Patent Judges.
BUI, Administrative Patent Judge.
DECISION ON APPEAL
Appellant seeks our review under 35 U.S.C. § 134(a) from the
Examiner’s Final Rejection of claims 1—7 and 9—21, which are all the claims
pending in the application. We have jurisdiction under 35 U.S.C. § 6(b).
We REVERSE.2
1 According to Appellant, the real party in interest is Apple Inc. App. Br. 3.
2 Our Decision refers to Appellant’s Appeal Brief (“App. Br.”) filed January
30, 2017; Reply Brief (“Reply Br.”) filed May 15, 2017; Examiner’s Answer
(“Ans.”) mailed March 13, 2017; Final Office Action (“Final Act.”) mailed
June 1, 2016; Advisory Action (“Advisory Act.”) mailed December 14,
2016; and original Specification (“Spec.”) filed May 22, 2013.
Appeal 2017-008323
Application 13/900,158
STATEMENT OF THE CASE
Appellant’s Invention
Appellant’s invention relates to a camera pose estimation apparatus
and method for augmented reality imaging, for finding a physical camera
pose for a camera associated with a head mounted display (HMD) worn by a
user. Spec. H 2—3; Title. Appellant’s camera pose estimation enables
insertion of virtual objects into an image-stream in real-time by correctly
aligning the virtual objects on the screen of the HMD, to obtain an
augmented reality effect for the user. Spec. 13; Abstract. Appellant’s
apparatus and method determine an estimate for a 3D camera pose relative
to a scene from 2D image data of a 2D image frame provided by the camera,
using four types of observations:
1. Detected 2D-3D point correspondences (minimize the 2D
pixel re-projection error)
2. Tracked 2D-3D point correspondences (minimize the 2D
pixel re-projection error)
3. Motion model observations (minimize the translation and
rotation relative to previous frame, if the pose was estimated in
previous frame)
4. New: Edge observations (minimize the distance between the
edges in the captured images and the lines/edges in the 3D
model projected to the screen using the current camera pose)[.]
By adding the fourth type of observation, the system
would be capable of functioning also in environments with little
or no texture, but with edges/lines present. Lines/edges will
typically be the edge between walls, wall and floor, wall and
ceiling, doors and walls etc.
Spec. H90-91; see also Fig. 5, Abstract. As such, Appellant’s invention is
able to provide robust, real-time 3D camera pose estimation in environments
2
Appeal 2017-008323
Application 13/900,158
lacking texture and contrast, such as “[i]n a room with e.g.[,] only white
walls with no paintings or other objects/illustrations.” See Spec, 87—89.
Claims 1, 14, and 20 are independent. Representative claim 1 is
reproduced below with disputed limitations in italics'.
1. An apparatus for providing an estimate for a 3D camera
pose relative to a scene from 2D image data of a 2D image
frame provided by a camera, the apparatus comprising:
one or more processors; and
a memory coupled to the one or more processors and
comprising computer readable instructions which, when
executed by the one or more processors, cause the one or more
processors to:
obtain, from a detection thread, detected 2D-3D
point correspondences between a current image and a
previous image of a scene,
obtain, from a tracking thread, tracked 2D-3D
point correspondences between the current image and the
previous image,
obtain motion model observations by comparing
the current image to the previous image,
obtain edge observations in the current image and
the previous image, and
determine, in a primary thread, an estimate for a
3D camera pose relative to the scene according to the
detected 2D-3D correspondences, the tracked 2D-3D
point correspondences, the motion model observations,
and the edge observations by minimizing a distance
between the edges in the current image and the previous
image and edges in a 3D model of the environment
projected to a screen based on a previous camera pose.
App. Br. 10-13 (Claims App’x).
3
Appeal 2017-008323
Application 13/900,158
Examiner’s Rejections3 & References
(1) Claims 1—7, 10, 11, and 13—21 stand rejected under
35 U.S.C. § 103(a) as being unpatentable over L. Vacchetti, V. Lepetit, and
P. Fua {Stable Real-Time 3D Tracking Using Online and Offline
Information, IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. 26 (10), 1385—1391 (2004); “Fua”) and Comaniciu et al.
(US 2003/0174773 Al; Sept. 18, 2003; “Comaniciu”). Final Act. 13-19.
(2) Claim 9 stands rejected under 35 U.S.C. § 103(a) as being
unpatentable over Fua, Comaniciu, and Satoh et al. (US 2006/0094955 Al;
published May 4, 2006; “Satoh”). Final Act. 20.
(3) Claim 12 stands rejected under 35 U.S.C. § 103(a) as being
unpatentable over Fua, Comaniciu, and Styers et al. (US 2006/0187034 Al;
published Aug. 24, 2006; “Styers”). Final Act. 20-21.
3 Claims 1, 6, 7, 10, and 11 were rejected under 35 U.S.C. § 112, second
paragraph, as indefinite. Final Act. 4—5. However, this rejection was
withdrawn in the Examiner’s Advisory Action (mailed December 14, 2016),
and is no longer pending on appeal. Advisory Act. 2; see also Ans. 2.
Additionally, claims 14—19 were rejected under 35 U.S.C. § 101 as directed
to non-statutory subject matter. Final Act. 5—6. This rejection, too, was
withdrawn in the Examiner’s Advisory Action, and is no longer pending on
appeal. Advisory Act. 2; see also Ans. 2. Finally, claims 1—7 and 9—21
were rejected under 35 U.S.C. § 102(b) as anticipated by Engedal (US
2010/0232727 Al; published Sept. 16, 2010). Final Act. 6—12. This
rejection was similarly withdrawn in the Examiner’s Advisory Action, and is
no longer pending on appeal. Advisory Act. 2; see also Ans. 2.
4
Appeal 2017-008323
Application 13/900,158
Issue on Appeal
Based on Appellant’s arguments, the dispositive issue on appeal is
whether the combination of Fua and Comaniciu teaches or suggests:
obtain, from a detection thread, detected 2D-3D point
correspondences between a current image and a previous image
of a scene,
obtain, from a tracking thread, tracked 2D-3D point
correspondences between the current image and the previous
image,. . .
obtain edge observations in the current image and the
previous image, and
determine, in a primary thread, an estimate for a 3D
camera pose relative to the scene according to the detected 2D-
3D correspondences, the tracked 2D-3D point correspondences,
the motion model observations, and the edge observations by
minimizing a distance between the edges in the current image
and the previous image and edges in a 3D model of the
environment projected to a screen based on a previous camera
pose,
as recited in independent claim 1, and similar limitations recited in
independent claims 14 and 20. App. Br. 6—7; Reply Br. 3—5.
ANALYSIS
Independent claim 1 recites, inter alia:
obtain edge observations in the current image and the
previous image, and
determine, in a primary thread, an estimate for a 3D
camera pose relative to the scene according to the detected 2D-
3D correspondences, the tracked 2D-3D point correspondences,
the motion model observations, and the edge observations by
minimizing a distance between the edges in the current image
and the previous image and edges in a 3D model of the
environment projected to a screen based on a previous camera
pose,
5
Appeal 2017-008323
Application 13/900,158
and further recites “a detection thread” and “a tracking thread” for obtaining
the 2D-3D point correspondences. App. Br. 6—7.
In support of the obviousness rejection of claim 1, the Examiner finds
Fua estimates a 3D camera pose by minimizing a reprojection error for 3D
points that are determined from “detected 2D-3D correspondences” and
“tracked 2D-3D correspondences” between “incoming images and [target
object] keyframes” created in a training stage. Ans. 4 (citing Fua 1386,
§§ 3, 3.1, 3.2). The Examiner further finds Fua’s “Related Work” section
discloses object tracking that avoids drift by finding camera poses that
correctly reproject some fixed features—such as “edges, line segments, or
points”—thereby teaching the use of “edge observations“ from current and
previous images, as claimed. Ans. 4 (citing Fua 1385, § 2) (emphasis
added); Final Act. 14. Based on these findings, the Examiner concludes it
would have been obvious to one of ordinary skill to estimate a 3D camera
pose in Fua based on a combination of “detected 2D-3D correspondences,
tracked 2D-3D correspondences,. . . and edge observations,” as recited in
claim 1. Ans. 3^4.
Appellant argues “Fua is silent regarding determining a 3D camera
pose based on the combination of ‘detected 2D-3D correspondences, tracked
2D-3D correspondences, motion model observations, and edge
observations’,” as required by claim 1. App. Br. 7. We agree with
Appellant that the Examiner has not identified sufficient evidence to show
Fua teaches or suggests the “determining” limitation of claim 1. App. Br. 6—
7; Reply Br. 4—5. In particular, Fua’s “Related Work” section (section 2)
merely describes “compar[ing] reprojections from multiple camera poses
until the reprojection is suitable” by “fmd[ing] a camera pose that correctly
6
Appeal 2017-008323
Application 13/900,158
reprojects fixed features of a given 3D model, such as edges.” Reply Br. 4—
5 (citing Fua 1385, § 2). In contrast, Appellant’s “claims are directed to
identifying a 3D camera pose based on various observations.” Reply Br. 5.
In addition, the Examiner has not provided an adequate reason with
rational underpinnings to explain why one having ordinary skill in the art
would have been led to modify Fua’s tracking rigid objects in 3D with 2D-
3D correspondences for “matched points between keyframe and incoming
image” (see Fua 1388, § 5, 1386, § 3, Abstract), to combine edge
observations with the 2D-3D correspondences for determining a 3D camera
pose as required by claim 1. Fua uses 2D-3D correspondences and back
projecting onto a model “to track objects” and determine “the pose of the
object. . . based on models of the object,” in contrast to claim 1—directed to
“determining an estimate for a f3 jD camera pose relative to a scene.” App.
Br. 6 (citing Fua 1386, §§ 3, 3.1, Abstract). See In re Chaganti, 2014, WL
274514, *4 (Fed. Cir. 2014) (“It is not enough to say that... to do so would
‘have been obvious to one of ordinary skill.’. . . Such circular reasoning is
not sufficient—more is needed to sustain an obviousness rejection.”)
The Examiner additionally finds Fua’s real-time 3D tracking, together
with Comaniciu’s “multiple threads that are synchronized” suggest “a
primary thread”—for determining an estimate for a 3D camera pose—is
separate from “a detection thread” and “a tracking thread” for obtaining
detected and tracked 2D-3D point correspondences, as recited in claim 1.
Final Act. 14—15 (citing Comaniciu 48, 53). We do not agree.
We agree with Appellant that the combination of Fua and Comaniciu
does not teach or suggest “a primary thread in which ‘an estimate for a 3D
camera pose relative to the scene’ is determined,” and separate specific
7
Appeal 2017-008323
Application 13/900,158
threads by which 2D-3D point correspondences are obtained. App. Br. 7;
see also Reply Br. 5. Rather, Comaniciu at most “indicates a modular
system where multiple threads may be synchronized.” App. Br. 7.
Because the Examiner has not shown the combination of Fua and
Comaniciu teaches or suggests all the limitations of claim 1, we do not
sustain the Examiner’s obviousness rejection of independent claim 1, and
independent claims 14 and 20 reciting similar limitations. See claims 14 and
20 (using “a detection thread” and “a tracking thread,” to “determine, in a
primary thread, an estimate for a 3D camera pose for the camera relative to
the scene according to the detected 2D-3D correspondences, the tracked 2D-
3D point correspondences, the motion model observations, and the edge
observations”). App. Br. 11—13 (Claims App’x). We also do not sustain the
Examiner’s rejection of dependent claims 2—7, 9-13, 15—19, and 21 because
the Examiner has not shown that the additional teachings of Satoh and
Styers make up for the above-noted deficiencies of Fua and Comaniciu. We
do not address Appellant’s remaining arguments because the issues
discussed supra are dispositive as to all pending claims on appeal.
CONCLUSION
On the record before us, we conclude Appellant has demonstrated the
Examiner erred in rejecting claims 1—7 and 9—21 under 35 U.S.C. § 103(a).
DECISION
As such, we REVERSE the Examiner’s final rejection of claims 1—7
and 9-21.
REVERSED
8