Ex Parte Ritter et al

Patent Trial and Appeal Board

Nov 27, 2018

13239223 (P.T.A.B. Nov. 27, 2018)

UNITED STA TES p A TENT AND TRADEMARK OFFICE
APPLICATION NO. FILING DATE
13/239,223 09/21/2011
21912 7590 11/29/2018
VAN PELT, YI & JAMES LLP
10050 N. FOOTHILL BLVD #200
CUPERTINO, CA 95014
FIRST NAMED INVENTOR
Jordan Ritter
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www .uspto.gov
ATTORNEY DOCKET NO. CONFIRMATION NO.
CLOUP002 8181
EXAMINER
MEINECKE DIAZ, SUSANNA M
ART UNIT PAPER NUMBER
3683
NOTIFICATION DATE DELIVERY MODE
11/29/2018 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
usptocorrespondence@ip-patent.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte JORDAN RITTER and ALEXANDER EDELSTEIN
Appeal2017-008403
Application 13/239,223
Technology Center 3600
Before JOSEPH L. DIXON, JOHN A. JEFFERY, and
JENNIFER L. McKEOWN, Administrative Patent Judges.
JEFFERY, Administrative Patent Judge.
DECISION ON APPEAL
Appellants 1 appeal under 35 U.S.C. § 134(a) from the Examiner's
decision to reject claims 1, 3-10, 12, 17-19, 24, 26-32, 34--37, and 39-51.
We have jurisdiction under 35 U.S.C. § 6(b). We affirm.
STATEMENT OF THE CASE
Appellants' invention evaluates work of an unsupervised worker with
a reputation system that uses answer data received from other unsupervised
workers that reflects on work quality. The work product is accepted or
1 Appellants identify the real party in interest as CrowdSource Solutions,
Inc. App. Br. 2.
Appeal2017-008403
Application 13/239,223
rejected based on whether the received answers are correct. See generally
Abstract; Spec. ,r,r 35-37. Claim 1 is illustrative:
1. A reputation system to evaluate work product,
compnsmg:
a communication interface coupled to receive answer data
associated with a given work product from a plurality of unsupervised
workers over a network via a computerized worker platform, wherein
the answer data comprises:
a first review task answer associated with a first unsupervised
worker representing a judgment by the first unsupervised worker that
reflects on the quality of the work product generated by an originating
unsupervised worker, wherein the work product is provided by a
client device of the originating unsupervised worker; and
a second review task answer associated with a second
unsupervised worker representing a judgment by the second
unsupervised worker that reflects on the quality of the work product
generated by the originating unsupervised worker, wherein the first
review task answer, the second review task answer, and the work
product are received via the computerized worker platform; and a
processor configured to:
determine, based at least in part on the answer data and a
respective reputation data of the first unsupervised worker and the
second unsupervised worker, whether the first review task answer, the
second review task answer, or both, is correct, wherein the first review
task answer is weighted to reflect the reputation data of the first
unsupervised worker and the second review task answer is weighted
to reflect the reputation data of the second unsupervised worker,
wherein reputation data comprises a reputation score of an
unsupervised worker, wherein determining whether the first review
task answer, the second review task answer, or both, is correct
includes imposing on the first and second review task answers a
structure to relate the review task answers to each other and
comparing the related review task answers, wherein the structure
includes a mapping that allows relatively closely related answers to
be processed in a way that is different from unrelated answers; and
accept or reject the work product generated by the originating
unsupervised worker based on the determination of whether the first
review task answer, the second review task answer, or both, is correct.
2
Appeal2017-008403
Application 13/239,223
THE REJECTIONS
The Examiner rejected claims 1, 3-10, 12, 17-19, 24, 26-32, 34--37,
and 39-51 under 35 U.S.C. § 101 as directed to ineligible subject matter.
Ans. 2-5. 2
The Examiner rejected claims 1, 3-10, 12, 17-19, 24, 26-32, 34--37,
and 39-51 under 35 U.S.C. § 112, first paragraph as failing to comply with
the written description requirement. Ans. 6-9.
The Examiner rejected claims 1, 3-10, 12, 17-19, 24, 26-32, 34--37,
and 39-51 under 35 U.S.C. § 112, second paragraph as indefinite. Ans. 9-
11.
THE INELIGIBILITY REJECTION
The Examiner finds that claimed invention is directed to an abstract
idea, namely by determining which reviewer of multiple reviewers likely
provides the most trustworthy evaluation of a work product, 3 which is said
to be similar to concepts involving, among other things, (1) an idea of itself;
(2) comparing new and stored information and using rules to identify
2 Throughout this opinion, we refer to (1) the Appeal Brief filed January 16,
2017 ("App. Br."); (2) the Examiner's Answer mailed March 13, 2017
("Ans."); and (3) the Reply Brief filed May 12, 2017 ("Reply Br.").
3 Although the Examiner's rejection merely italicizes the claim language
in connection with the identified abstract idea, the Examiner's Answer
nonetheless further generalizes the claimed invention as directed to
determining which reviewer of multiple reviewers likely provides the most
trustworthy evaluation of a work product. Compare Ans. 2-3 with Ans. 15
(summarizing the "main focus" of the invention). Accordingly, we presume
that this latter characterization corresponds to the identified abstract idea.
3
Appeal2017-008403
Application 13/239,223
options; (3) organizing human activity; and (4) mathematical
relationships/formulas. Ans. 2--4, 11-16. According to the Examiner, the
claimed elements do not add significantly more to the abstract idea to render
the claimed invention patent-eligible because, among other things, the
recited elements do not improve a specific technology or the computer itself,
but rather recite generic computer components whose functions merely
implement the abstract idea on a computer. See Ans. 4--5, 17-20.
Appellants argue that the claimed invention is not directed to an
abstract idea because, among other things, the claims are not fundamental
economic practices, methods of organizing human activity, ideas
themselves, or mathematical relationships/formulas. App. Br. 11-21;
Reply Br. 2-9. According to Appellants, the claimed invention is not only
distinguishable from the various decisions cited by the Examiner, but is also
analogous to the claims at issue in DDR Holdings, LLC v. Hotels.Com, L.P.,
773 F.3d 1245 (Fed. Cir. 2014) andMcRO, Inc. v. Bandai Namco Games
America, Inc., 837 F.3d 1299 (Fed. Cir. 2016). App. Br. 11-21; Reply Br.
2-9.
Appellants add that even if the claimed invention was directed to an
abstract idea, the claimed limitations recite significantly more than the
alleged abstract idea. App. Br. 21-25; Reply Br. 9-11. According to
Appellants, the claimed invention not only improves online crowdsourcing,
but is also analogous to the claims at issue in BASCOM Global Internet
Services, Inc. v. AT&T Mobility LLC, 827 F.3d 1341 (Fed. Cir. 2016). App.
Br. 21-25; Reply Br. 9-11.
4
Appeal2017-008403
Application 13/239,223
ISSUE
Has the Examiner erred in rejecting claim 1 by concluding that it
is directed to ineligible subject matter under § 101? This issue turns on
whether the claimed invention is directed to a patent-ineligible abstract idea
and, if so, whether the claim's elements----considered individually and as an
ordered combination-transform the nature of the claim into a patent-
eligible application of that abstract idea.
ANALYSIS
Claims 1, 3-10, 12, 17-19, 24, 26--32, 34-37, 39--41, 43-51
To determine whether claims are patent eligible under § 101, we apply
the Supreme Court's two-step test articulated in Alice Corp. Proprietary Ltd.
v. CLS Bank International, 134 S. Ct. 2347 (2014). First, we determine
whether the claims are directed to a patent-ineligible concept: laws of nature,
natural phenomena, and abstract ideas. Id. at 2354--55. If so, we then
proceed to the second step and examine the claim's elements-both
individually and as an ordered combination-to determine whether the claim
contains an "inventive concept" sufficient to transform the claimed abstract
idea into a patent-eligible application. Id. at 2357.
Alice Step One
Applying Alice step one, we agree with the Examiner that the claimed
invention is directed to an abstract idea, namely evaluating a work product
based on the relative trustworthiness and correctness of multiple reviewers'
quality judgments. Claim 1 recites, in pertinent part, a reputation system to
evaluate work product comprising a communication interface coupled to
5
Appeal2017-008403
Application 13/239,223
receive "answer data" comprising first and second review task answers
associated with first and second unsupervised workers, respectively, where
each review task answer represents the associated worker's judgment on
the quality of the work product generated by an originating unsupervised
worker. The claim further recites a processor configured to determine
whether the first and/ or second review task answers are correct based at least
partly on (1) the answer data, and (2) the first and second unsupervised
workers' "reputation data" comprising a score, where the reputation data
is weighted for each worker. The recited task answer correctness
determination also ( 1) imposes a structure on the review task answers to
relate them to each other, and (2) compares related answers, where the
structure's mapping allows relatively closely related answers to be processed
differently from unrelated answers. Lastly, claim 1 recites that the processor
can accept or reject the work product based on the task answer correctness
determination.
The Specification explains that reputation data comprises a single,
composite score that reflects the worker's current reputation and reputation
history, such that the score reflects how the worker's reputation changed
over time. Spec. ,r 33. In some embodiments, the reputation score is based
on (1) other workers' judgments, and/or (2) whether other workers agreed
or disagreed with the worker's judgment or decision. Id. Therefore, the
reputation score reflects the collective judgment of others over time
regarding the quality of the worker's work. Id. The reputation score can
also reflect other factors, such as how long the worker has been in the
worker pool, and various other demographic, psychographic, and other data
associated with the worker. Spec. ,r 34.
6
Appeal2017-008403
Application 13/239,223
In essence, the claimed invention evaluates a work product based on
the relative trustworthiness and correctness of multiple reviewers' quality
judgments. This evaluation, although based on analyzing received data,
effectively makes a decision on the merits of a worker's work product based
on peer review and, in doing so, assesses the individual peers' feedback for
relative reliability and correctness. This process is akin to a supervisor
asking a subordinate's co-workers about their opinions on the quality of the
subordinate's work product, and then assessing the relative merits of each
peer's responses given each peer's professional reputation and the similarity
of that peer's response to other responses. Given this fundamental
performance appraisal based on peer review-a fundamental business
practice-we agree with the Examiner that the claimed invention is directed
to an abstract idea despite Appellants' arguments to the contrary (App. Br.
12-21; Reply Br. 2-9).
As such, we see no error in the Examiner's finding that accepting or
rejecting a work product based on the recited correctness determination at
least fundamentally (1) collects and compares information, and (2) uses rules
to identify options. See Ans. 11-14.
It is well settled that collecting information is within the realm of
abstract ideas----even when the information is limited to particular content.
Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1353 (Fed. Cir.
2016). It is also well settled that analyzing information by steps people go
through in their minds, or by mathematical algorithms, without more, are
essentially mental processes within the abstract idea category. Id. at 1354.
And merely presenting the results of abstract processes of collecting and
analyzing information, without more (such as identifying a particular tool for
7
Appeal2017-008403
Application 13/239,223
presentation), is abstract as an ancillary part of such collection and analysis.
Id. Similar to the claims at issue in Electric Power, the claimed invention
here gathers, manipulates, analyzes, and presents information of a specified
content, but does not use any particular inventive technology for performing
those functions.
Appellant's reliance onMcRO (App. Br. 20-21; Reply Br. 8-9) is
unavailing. There, the claimed process used a combined order of specific
rules that rendered information in a specific format that was applied to create
a sequence of synchronized, animated characters. McRO, 837 F.3d at 1315.
Notably, the recited process automatically animated characters using
particular information and techniques-an improvement over manual three-
dimensional animation techniques that was not directed to an abstract idea.
Id. at 1316.
But unlike McRO that improved how the physical display operated
to produce better quality images, the claimed invention here merely uses
generic computing components to evaluates a work product based on the
relative trustworthiness and correctness of multiple reviewers' quality
judgments -a generic computer implementation that is not only directed
to fundamental data analysis functions, but also does not improve a display
mechanism as was the case in McRO. Although the claimed invention
requires computer components, it is the incorporation of those
components-not a claimed rule-that purportedly improves the existing
process. Cf FairWarning IP, LLC v. Iatric Systems, Inc., 839 F.3d 1089,
1095 (Fed. Cir. 2016). In short, the claimed invention does not focus on an
improvement in computers as tools, but rather certain independently abstract
ideas that use computers as tools. See Elec. Power, 830 F.3d at 1354.
8
Appeal2017-008403
Application 13/239,223
Appellants' reliance on DDR (App. Br. 18-20; Reply Br. 18-20) is
likewise unavailing. There, instead of a computer network operating in
its normal, expected manner by sending a website visitor to a third-party
website apparently connected with a clicked advertisement, the claimed
invention in DDR generated and directed the visitor to a hybrid page that
presented (1) product information from the third party, and (2) visual "look
and feel" elements from the host website. DDR, 773 F.3d at 1258-59.
Given this particular Internet-based solution, the court held that the claimed
invention did not merely use the Internet to perform a business practice
known from the pre-Internet world, but rather was necessarily rooted in
computer technology to overcome a problem specifically arising in computer
networks. Id. at 1257.
That is not the case here. As noted previously, Appellants' claimed
invention, in essence, evaluates a work product based on the relative
trustworthiness and correctness of multiple reviewers' quality judgments.
Despite Appellants' arguments to the contrary (App. Br. 18-20; Reply Br.
18-20), the claimed invention here is not necessarily rooted in computer
technology in the sense contemplated by DDR where the claimed invention
solved a challenge particular to the Internet. Although the Appellants'
invention uses a computer to receive and process data, it does not solve a
challenge particular to the computer or the network used to receive that data.
That the recited determination imposes a structure on the review task
answers to relate them to each other does not change our conclusion despite
Appellants' arguments to the contrary. See App. Br. 18-20. Although
claim 1 specifies that this structure includes a mapping that allows relatively
closely related answers to be processed differently from unrelated answers,
9
Appeal2017-008403
Application 13/239,223
the claim does not specify how this desired result is achieved with this
mapping-a mapping that could otherwise be performed entirely mentally
or with pen and paper by, for example, merely segregating related and
unrelated answers in different groups.
We reach this conclusion despite the Specification's paragraph 36
describing various non-limiting examples of mapping, including ( 1) a "bias
map" where the answer space is a range of subjective evaluations, such as
"awful," "bad," "okay," "good," etc.; (2) a "letter map" with independent
letter-based answer choices; and (3) a "binary" map (zero or one, correct or
wrong, etc.). In paragraph 36, the Specification also describes an example
in connection with the "bias map" where closely-related answers can be
combined into a single value such that they are processed differently than
other unrelated answers.
Although this description informs our understanding of the mapping
recited in claim 1, it is not so limited. And while a processor performs the
mapping in claim 1, we still see no reason why this process could not be
otherwise performed entirely mentally or with pen and paper even if the
particular exemplary mapping techniques described in paragraph 36 of the
Specification were recited in claim I-which they are not. To the extent
that Appellants contend otherwise, there is no persuasive evidence on this
record to substantiate such a contention.
Lastly, to the extent that Appellants contend that the claims are not
directed to an abstract idea because they involve online "crowdsourcing"
using the Internet (see App. Br. 18-19), we disagree, for such
"crowdsourcing" activities merely organize human activities via the Internet.
Accord Ans. 15 (finding that the main focus of the invention includes
10
Appeal2017-008403
Application 13/239,223
organizing human activity); Ans. 17 (finding that paragraph 2 of Appellants'
Specification describes online "crowdsourcing" as an approach to access and
manage human workers). Cf Gust, Inc. v. Alphacap Ventures, LLC, 905
F.3d 1321, 1325 (Fed. Cir. 2018) (discussing Kickstarter, Inc. v. Fan
Funded, LLC, No. 11-cv-6909, 2015 WL 3947178 (S.D.N.Y. 2015), ajf'd
654 F. App'x 481 (Fed. Cir. 2016), where the Federal Circuit affirmed a
district court's applying Alice to find claims directed to "crowdfunding"
invalid under§ 101); see also Gust, 905 F.3d at 1326 (noting the district
court's finding that "crowdfunding" is a fundamental economic concept and
a way of organizing human activity).
Although the recited functions may be beneficial by evaluating a
work product based on the relative trustworthiness and correctness of
multiple reviewers' quality judgments, a claim for a useful or beneficial
abstract idea is still an abstract idea. See Ariosa Diagnostics, Inc. v.
Sequenom, Inc., 788 F.3d 1371, 1379-80 (Fed. Cir. 2015).
We, therefore, agree with the Examiner that claim 1 is directed to an
abstract idea.
Alice Step Two
Turning to Alice step two, the recited elements----considered
individually and as an ordered combination----do not transform the nature of
claim 1 into a patent-eligible application of the abstract idea to ensure that
the claim amounts to significantly more than that idea. See Alice, 134 S. Ct.
at 2357.
That the recited method is implemented by a computer and involves
exchanging data between computing systems does not change our
11
Appeal2017-008403
Application 13/239,223
conclusion. As the Examiner explains, the claimed invention merely uses
generic computing components to perform the recited abstract idea, namely
evaluating a work product based on the relative trustworthiness and
correctness of multiple reviewers' quality judgments. See Ans. 4--5; 17-20.
Merely reciting these generic computing components cannot
transform a patent-ineligible abstract idea into a patent-eligible invention.
See Alice 134 S. Ct. at 2358-59; see also Mortgage Grader Inc. v. First
Choice Loan Services, Inc., 811 F.3d 1314, 1324--25 (Fed. Cir. 2016)
(noting that components such an "interface," "network," and "database" are
generic computer components that do not satisfy the inventive concept
requirement); buySAFE v. Google, Inc., 765 F.3d 1350, 1355 (Fed. Cir.
2014) ("That a computer receives and sends the information over a
network-with no further specification-is not even arguably inventive.").
In other words, merely reciting an abstract idea while adding the
words "apply it with a computer" does not render an abstract idea non-
abstract: there must be more. See Alice, 134 S. Ct. at 2358-59. But this
requirement is not satisfied here, nor does the claimed invention improve the
computer processor device's functionality or efficiency, or otherwise change
the way that device functions. Cf Enfzsh LLC v. Microsoft Corp., 822 F .3d
1327, 1335 (Fed. Cir. 2016). That Appellants acknowledge that the recited
limitations may be separately known and conventional (Reply Br. 10) only
further bolsters the Examiner's findings and conclusions in this regard.
We also find unavailing Appellants' contention that the recited
elements add significantly more to the recited abstract idea because the
claimed invention improves another technology or technical field, namely
online crowdsourcing. App. Br. 21-23; Reply Br. 9-10. Not only do
12
Appeal2017-008403
Application 13/239,223
"crowdsourcing" activities merely organize human activities via the Internet
as noted previously, but the components used to achieve that end are merely
generic computer components as the Examiner indicates. Ans. 17-19.
Appellants' reliance on BASCOM (App. Br. 23-25; Reply Br. 10-11)
is unavailing. There, the court held eligible claims directed to a technology-
based solution to filter Internet content that overcame existing problems with
other Internet filtering systems by making a known filtering solution-
namely a "one-size-fits-all" filter at an Internet Service Provider (ISP}-
more dynamic and efficient via individualized filtering at the ISP.
BASCOM, 827 F.3d at 1351. This customizable filtering solution improved
the computer system's performance and, therefore, was patent-eligible. See
id.
But unlike the filtering system improvements in BASCOM that added
significantly more to the abstract idea in that case, the claimed invention
here uses generic computing components to implement an abstract idea,
namely evaluating a work product based on the relative trustworthiness and
correctness of multiple reviewers' quality judgments.
Lastly, we find unavailing Appellants' contention that the claimed
invention does not preempt other reputation systems for evaluating work
using different answer correctness determinations. App. Br. 20-21. Where,
as here, the claims cover a patent-ineligible concept, preemption concerns
"are fully addressed and made moot" by an analysis under the Alice
framework. See Ariosa Diagnostics, Inc. v. Sequenom, Inc., 788 F.3d 1371,
1379 (Fed. Cir. 2015).
13
Appeal2017-008403
Application 13/239,223
Therefore, we are not persuaded that the Examiner erred in rejecting
claim 1, and claims 3-10, 12, 17-19, 24, 26-32, 34--37, 39--41, and 43-51
not argued separately with particularity.
Claim 42
We also sustain the Examiner's ineligibility rejection of dependent
claim 42 reciting that a review task answer comprises answers in one or
more key-value pairs, and imposing the structure on the review task
answers4 comprises applying a mapping of a review task answer to an
answer format.
First, Appellants do not provide the requisite subheading to clearly
and unambiguously identify that claim 42 is argued separately (see App. Br.
19-20, 23}-a regulatory requirement. See 37 C.F.R. § 4I.37(c)(l)(iv)
("Under each heading identifying the ground of rejection being contested,
any claim(s) argued separately or as a subgroup shall be argued under a
separate subheading that identifies the claim(s) by number.") (emphasis
added); see also MANUAL OF p ATENT EXAMINING PROCEDURE (MPEP)
§ 1205.02 (9th ed. Rev. 08.2017, Jan. 2018) (explaining this rule).
But even if such arguments were properly presented ( which they were
not), we still find them unpersuasive. Despite Appellants' arguments to the
contrary (App. Br. 19-20, 23), we do not find this additional limitation is
necessarily rooted in computer technology, nor does it improve another
4 Although claim 42 omits the second review task answer by reciting "the
first and review [sic] task answers" in line 3, we nonetheless presume that
this phrase is intended to include the second review task answer.
14
Appeal2017-008403
Application 13/239,223
technology or field or add significantly more to the abstract idea for the
reasons previously discussed.
As noted previously, claim 1 (from which claim 42 depends) specifies
that the imposed structure includes a mapping that allows relatively closely
related answers to be processed differently from unrelated answers. Yet
the claim does not specify how this desired result is achieved with this
mapping-a mapping that could otherwise be performed entirely mentally
or with pen and paper by, for example, merely segregating related and
unrelated answers in different groups.
We reach a similar conclusion regarding claim 42 despite the claim
adding that answers are in key-value pairs, and that mapping is applied to an
answer format. According to paragraph 3 6 of Appellants' Specification,
respective reviewers' answer sets are compared to determine which answers
are related, e.g., which have the same key. Although this exemplary
description informs our understanding of the recited mapping application, it
is not so limited.
And while a processor presumably applies the mapping of claim 42,
we still see no reason why this process could not be otherwise performed
entirely mentally or with pen and paper even if the particular exemplary
mapping techniques described in paragraph 3 6 of the Specification were
recited in claim 42-which they are not. To the extent that Appellants
contend otherwise, there is no persuasive evidence on this record to
substantiate such a contention.
Nor do we find that the limitations of claim 42 considered
individually and as an ordered combination add significantly more to the
abstract idea. As noted previously, the claimed invention merely uses
15
Appeal2017-008403
Application 13/239,223
generic computing components to perform the recited abstract idea, namely
evaluating a work product based on the relative trustworthiness and
correctness of multiple reviewers' quality judgments. See Ans. 4--5; 17-20.
That Appellants acknowledge that the recited limitations may be separately
known and conventional (Reply Br. 10) only further bolsters the Examiner's
findings and conclusions in this regard.
Therefore, we are not persuaded that the Examiner erred in rejecting
claim 42.
THE WRITTEN DESCRIPTION REJECTION
The Examiner finds that the recited ( 1) correctness determination, and
(2) accepting or rejecting a work product based on that determination is not
described adequately in the Specification to show possession of those
features when the application was filed. Ans. 6-9, 20-22. According to the
Examiner, the claim implies that the first and second review task answers
conflict with each other, and that this conflict is settled by establishing that
one such answer is correct and the other incorrect. Ans. 7. The Examiner
adds that the Specification is not only vague regarding the meaning of one
reviewer's review "trumping" those of other reviewers, it does not explain
when and how this "trumping" process occurs. Ans. 8. Rather, the
Specification is said to foster more of a "consensus rating" among three or
more reviewers as opposed to determining which of two reviewers with
opposing views is correct. Ans. 8-9.
Appellants argue that the Specification sufficiently describes the
recited correctness determination, and that ordinarily skilled artisans would
understand the meaning of one reviewer's review "trumping" those of other
16
Appeal2017-008403
Application 13/239,223
reviewers. App. Br. 25-28; Reply Br. 11-15. According to Appellants,
the term "trumping" as used in the Specification refers to one person
beating/ defeating another person, so that in the context of the claim, the first
review task answer can be determined to be correct and "trump" the second
review task answer or vice-versa, 5 or both answers can be determined to
be correct. Reply Br. 12-13. Appellants add that the Specification also
supports instances where one reviewer's review with higher scores does not
trump reviews of two other reviewers with lower scores. Reply Br. 14--15.
ISSUE
Has the Examiner erred in rejecting claim 1 by finding that recited
(1) correctness determination, and (2) accepting or rejecting a work product
based on that determination are unsupported in the Specification? This issue
turns on whether Appellants' original disclosure reasonably conveys to
ordinarily skilled artisans that Appellants possessed these limitations when
the application was filed.
ANALYSIS
To satisfy the written description requirement, the disclosure must
reasonably convey to skilled artisans that Appellants possessed the claimed
5 Notably, Appellants indicate that, in one alternative, the second review task
answer can be determined to be correct and "trump" the second review task
answer-a puzzling scenario where the second review task answer trumps
itself. Nevertheless, we presume that this is a typographical error, and that
the second review task answer is intended to trump the first review task
answer in this scenario.
17
Appeal2017-008403
Application 13/239,223
invention as of the filing date. Ariad Pharms., Inc. v. Eli Lilly & Co., 598
F.3d 1336, 1351 (Fed. Cir. 2010) (en bane).
That is the case here despite the Examiner's findings to the contrary.
Paragraph 35 of the Specification describes a work evaluation process in
Figure 6 where received data is used by a reputation system to determine
which answers are correct-answers that not only pertain to the originating
worker's work product itself, but also reviewers' answers to certain task
questions. In some embodiments, the reviewer's respective judgments
regarding the accuracy of the originating worker's work may be weighted
to reflect the reviewer's respective reputations, and the weighted judgments
compared to determine which reviewer answers are correct. Spec. ,r 35.
Therefore, a favorable review from a reviewer with a strong reputation may
"trump" less favorable reviews by two other reviewers with lower reputation
scores. Id. Conversely, a negative review from a reviewer with a high
reputation score may "trump" more favorable reviews from other reviewers
with lower scores. Id.
This disclosure reasonably supports determining whether the first
and/or second review task answers are correct based at least partly on (1)
answer data, and (2) the associated workers' reputation data, where the
review task answers are weighted to reflect the associated workers'
reputation data, as claimed. Although the examples involving "trumping" in
paragraph 35 involve three or more reviewers where one review "trumps"
those of other reviewers, ordinarily skilled artisans would nevertheless
understand that this functionality could also apply in a two-reviewer
scenario, where the higher reputation reviewer's review "trumps" that of the
other reviewer. See Spec. ,r 35. That paragraph 19 states that a review task
18
Appeal2017-008403
Application 13/239,223
is to be performed by one or more reviewing workers as Appellants indicate
(Reply Br. 13) only bolsters the notion that Appellants' original disclosure
reasonably supports a two-reviewer scenario as claimed.
We reach this conclusion even assuming, without deciding, that both
reviewers' answers are the same. Despite a potentially higher degree of
confidence resulting from unanimous reviews in this scenario, there may
nevertheless be valid reasons why the lower-reputation reviewer's review
is "trumped" apart from the content of the answer itself, including other
negative factors that resulted in that reviewer's lower reputation score that
weigh against the review's reliability and trustworthiness. See Spec. ,r,r 35-
36.
We also find that the Specification reasonably supports ( 1) imposing a
structure to relate review task answers to each other, and (2) comparing the
related review task answers, where the structure includes a mapping that
allows relatively closely related answers to be processed differently from
unrelated answers, as claimed.
Paragraph 3 6 of the Specification describes various non-limiting
examples of mapping, including ( 1) a "bias map" where the answer space is
a range of sub1ective evaluations such as "awful" "bad" "okay" "good"
J ' ' ' ' '
etc.; (2) a "letter map" with independent letter-based answer choices; and (3)
a "binary" map (zero or one, correct or wrong, etc.). The Specification also
describes an example in connection with the "bias map" where closely-
related answers can be combined into a single value such that they are
processed differently than other unrelated answers. Spec. ,r 36.
Although the Specification's paragraph 36 notes an example where
originating work judged as accurate by every reviewer may have a higher
19
Appeal2017-008403
Application 13/239,223
confidence than if one reviewer reached a different conclusion, that does
not mean that the disclosed reputation system must always be based on a
consensus, or that it is unnecessary to consider reviewers with less desirable
reputations as the Examiner seems to suggest. See Ans. 22. Rather, the
Specification is clear that determining whether the first and/or second review
task answers are correct is based at least partly on (1) answer data, and (2)
the associated workers' reputation data, where the review task answers
are weighted to reflect the associated workers' reputation data. The
Specification not only also describes the recited structure imposition
involving mapping in paragraph 36 as noted above, but also describes
accepting or rejecting the originating worker's work product based on the
correctness determination in paragraph 37 and Figure 8.
On this record, then, we find that Appellants' original disclosure
reasonably conveys to ordinarily skilled artisans that Appellants possessed,
when the application was filed, the recited (1) correctness determination, and
(2) accepting or rejecting a work product based on that determination.
Therefore, we are persuaded that the Examiner's written description
rejection of claims 1, 3-10, 12, 17-19, 24, 26-32, 34--37, and 39-51 is
erroneous.
THE INDEFINITENESS REJECTION
The Examiner finds that because it is unnecessary for the claimed
invention to consider a review from a reviewer with a less desirable
reputation, there is confusion whether ( 1) only two reviewers are involved,
where the answer of the reviewer with the more favorable reputation is
always deemed correct regardless of the other reviewer's answer, or (2)
20
Appeal2017-008403
Application 13/239,223
determining a correct answer is based on a consensus with weights for
different reviewers and, if so, how the correct answer is determined using
answer and reputation data. Ans. 9-11. The Examiner also questions how
each reviewer's reputation data is used when the first and second review task
answers are the same. Ans. 11.
Appellants argue that when read in light of the Specification, there is
no confusion in the claims whether the answer with the favorable reputation
is always deemed to be correct. App. Br. 28-32; Reply Br. 15-18.
According to Appellants, not only is claim 1 not limited to two reviewers
in light of the "comprising" language in the preamble, ordinarily skilled
artisans would understand that an originating work judged as accurate
by one reviewer and inaccurate by another reviewer will have a lower
confidence score than if both reviewers judged the work as accurate. Id.
Appellants add that the claim is clear regarding the recited review task
answer weighting, as well as accepting or rejecting the work product based
on the correctness determination, particularly given the associated
confidence determinations. App. Br. 32-33; Reply Br. 18-19.
ISSUE
Has the Examiner erred in rejecting claim 1 under§ 112, second
paragraph by finding that the recited (1) correctness determination, and (2)
accepting or rejecting a work product based on that determination renders
the claim indefinite?
21
Appeal2017-008403
Application 13/239,223
ANALYSIS
On this record, we find the Examiner's indefiniteness rejection of
claim 1 problematic on this record. First, despite the Examiner's apparent
confusion regarding whether only two reviewers are involved in the claimed
invention (Ans. 11 ), the claim is clear that there are two unsupervised
workers with associated review task answers that each reflect on the quality
of originating worker's work product. We reach this conclusion despite the
preambular term "comprising" not excluding additional unrecited reviewers
as Appellants indicate (App. Br. 29), for such inclusion does not render the
claim indefinite: it merely does not limit the claim to two reviewers. See
Genentech, Inc. v. Chiron Corp., 112 F.3d 495, 501 (Fed. Cir. 1997) (noting
that the term "comprising" in claim language means that the named elements
are essential, but other elements may be added and still form a construct
within the scope of the claim).
Nor do we find any confusion in the claimed invention when the
higher-reputation reviewer's answer is deemed correct, even if the claim was
limited to two reviewers-which it is not. As noted previously, in some
embodiments, the reviewer's respective judgments regarding the accuracy
of the originating worker's work may be weighted to reflect the reviewer's
respective reputations, and the weighted judgments compared to determine
which reviewer answers are correct. Spec. ,r 35. Therefore, a favorable
review from a reviewer with a strong reputation may "trump" less favorable
reviews by two other reviewers with lower reputation scores. Id.
Conversely, a negative review from a reviewer with a high reputation score
may "trump" more favorable reviews from other reviewers with lower
scores. Id.
22
Appeal2017-008403
Application 13/239,223
Although the higher-reputation reviewer's review "trumps" multiple
reviews in these examples, ordinarily skilled artisans would nevertheless
understand that this functionality could also apply in a two-reviewer
scenario, where the higher reputation reviewer's review "trumps" that of the
other reviewer----even if their answers are the same. See Spec. ,r 35. Despite
a potentially higher degree of confidence resulting from unanimous reviews
in this scenario, there may nevertheless be valid reasons why the lower-
reputation reviewer's review is "trumped" apart from the content of the
answer itself, including other negative factors that resulted in that reviewer's
lower reputation score that weigh against the review's reliability and
trustworthiness. See Spec. ,r,r 35-36.
On this record, then, we find the recited ( 1) correctness determination,
and (2) accepting or rejecting a work product based on that determination
are clear when read in light of the Specification. Therefore, we are
persuaded that the Examiner's indefiniteness rejection of claims 1, 3-10, 12,
17-19, 24, 26-32, 34--37, and 39-51 is erroneous.
CONCLUSION
The Examiner did not err in rejecting claims 1, 3-10, 12, 17-19, 24,
26-32, 34--37, and 39-51 under§ 101, but erred in rejecting those claims
under the first and second paragraphs of§ 112.
DECISION
We affirm the Examiner's decision to reject claims 1, 3-10, 12, 17-
19, 24, 26-32, 34--37, and 39-51. Because the rejection of each appealed
claim is affirmed on at least one of the grounds specified in the Office
23
Appeal2017-008403
Application 13/239,223
Action from which the appeal was taken, the Examiner's decision to reject
the claims is affirmed. See 37 C.F.R. § 4I.50(a)(l).
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.I36(a)(l)(iv).
AFFIRMED
24