Ex Parte McKinley et al

Patent Trial and Appeal Board

Nov 20, 2013

11102016 (P.T.A.B. Nov. 20, 2013)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte ROBERT L. McKINLEY,
NANCY S. PETERSEN, and JUDITH A. SPRAY
____________

Appeal 2011-008673
Application 11/102,016
Technology Center 3700
____________

Before GAY ANN SPAHN, JOHN W. MORRISON, and
MICHELLE R. OSINSKI, Administrative Patent Judges.

SPAHN, Administrative Patent Judge.

DECISION ON APPEAL

STATEMENT OF THE CASE
Appellants seek our review under 35 U.S.C. § 134 of the Examiner’s
rejection of claims 1, 3-6, 13, 14, 16-19, and 22-26 under 35 U.S.C. § 103(a)
as unpatentable over Swanson ʼ256 (US 5,657,256, issued Aug. 12, 1997)
and Swanson ʼ196 (US 2005/0125196 A1, published Jun. 9, 2005). We
have jurisdiction under 35 U.S.C. § 6(b).
We AFFIRM.
Appeal 2011-008673
Application 11/102,016

2
CLAIMED SUBJECT MATTER
The claimed subject matter relates to “development of tests to meet
test specifications,” and “[m]ore specifically, . . . to methods and systems for
the selection of items for inclusion in a test in order to meet test
specifications.” Spec. 1, ll. 3-5. Claim 1, reproduced below, with emphasis
added, is illustrative of the appealed subject matter.
1. A computer-assisted method of test assembly for
assembling a test comprised of a plurality of slots to be filled by
test items from a plurality of item collections, the method
having the advantage of ensuring the test will meet a content
specification while reducing the computational load required to
assemble the test, and the method comprising:
executing a test administration script to associate each of
the plurality of slots of the test with one of the plurality of item
collections from which an item to fill the slot is to be selected,
the one of the plurality of item collections including a plurality
of eligible test items that meet the content specifications for the
slot;
obtaining an interim test score based on a portion of the
test previously assembled and administered; and
further executing the test administration script to apply,
using a computer, an optimization process and randomization
factor to select the item to fill the slot while controlling
statistical properties of the test based on the interim score and
properties of the items within the item collection associated
with each of the plurality of slots; and
repeating the steps of obtaining the interim test score and
applying the optimization process until the test is completely
assembled and administered.

Appeal 2011-008673
Application 11/102,016

3
OPINION
Claims 1, 3, 4, and 22-24
Appellants argue independent claim 1 and do not provide separate
arguments for claims 3, 4, and 22-24 which, either directly or indirectly,
depend from claim 1. See App. Br. 9-16. We select independent claim 1 as
the representative claim to decide the appeal and claims 3, 4, and 22-24 fall
with claim 1. See 37 C.F.R. § 41.37(c)(1)(vii) (2011).
The Examiner finds that Swanson ʼ256 discloses “a computer-assisted
method . . . for assembling a test comprised of . . . : executing a test
administration script to associate each of the plurality of slots of the test with
an item collection from which an [item] to fill the slot is to be selected.”
Ans. 3 (citing Swanson ʼ256, col. 2, ll. 10-15, e.g., “test parameter,” and Fig.
1(b)). However, Swanson ʼ256 “is silent if the slots are filled by test item[s]
from a plurality of item collection[s].” Id. To cure the deficiency of
Swanson ʼ256, the Examiner turns to Swanson ʼ196 for its teaching that
“slots are filled by test item[s] from a plurality of item collection[s].” Ans.
3-4 (citing Swanson ʼ196, pp. 2-3, para. [0019], “[t]he test item database
may be a single repository containing all selectable test items for one or
more tests,” or “a test developer may choose questions from a plurality of
test item databases.”). The Examiner concludes that “it would have been
obvious to one of ordinary sill in the art to substitute the singular item pool
with a plurality of item pool[s] (or item collection[s]) for the predictable
result of [a] test creation system.” Ans. 4 (citing Swanson ʼ196, p. 2, para.
[0013]).
First, Appellants argue that Swanson ʼ256 fails to disclose “the
claimed test administration script,” because the test administration script of
Appeal 2011-008673
Application 11/102,016

4
Swanson ʼ256 does not “associate ‘each of the plurality of slots of the test
with one of the plurality of item collections from which an item to fill the
slot is to be selected, the one of the plurality of item collections including a
plurality of eligible test items that meet the content specifications for the
slot’” as is required by claim 1. App. Br. 10.
The Examiner responds that “[t]he phrase of ‘test administration
script’ is interpreted as a series of (software) instruction(s),” and Swanson
ʼ256 “provides a teaching of test administration script and executing test
administration script,” because “at each slot[,] a series of computer
instructions (script or computer program) are executed and associate a slot
with one of the plurality of item collections from which an item to fill the
slot is to be selected.” Ans. 6 (citing Swanson ʼ256, col. 19, 13-20 and 22-
35 and fig. 1(b)).
Appellants’ Specification indicates that “the test administration script
is the mechanism by which the content specifications are imposed on the
test,” and that the test administration script “consists of a series of
instruction sets, one for each slot in the test, identifying for that slot the item
collection from which the item for that slot is to be chosen.” Spec. 7, ll. 6-9.
Thus, the Examiner’s reliance on column 19, lines 13-35 of Swanson ʼ256
for disclosure of a test administration script appears reasonable since that
portion of Swanson ʼ256 describes selection of an item from an item pool.
Accordingly, we are not apprised of Examiner error by Appellants’ first
argument.
Second, Appellants argue that Swanson ʼ256 does not disclose that
“‘each of the plurality of slots’ has its own ‘content specifications,’” because
Swanson ʼ256 discloses “specifications for the test but not separate content
Appeal 2011-008673
Application 11/102,016

5
specifications for each slot.” App. Br. 10. We are not persuaded by
Appellants’ second argument because Appellants have failed to address the
combined teachings of the references upon which the Examiner’s rejection
was based. One cannot show nonobviousness by attacking references
individually where the rejections are based on a combination of references.
See In re Merck & Co., 800 F.2d 1091, 1097 (Fed. Cir. 1986); In re Keller,
642 F.2d 413, 426 (CCPA 1981). The Examiner relies upon Swanson ʼ196
to teach that “slots are filled by [a] test item from a plurality of item
collection[s].” Ans. 3-4. We note that Swanson ʼ196 discloses that “[a] test
developer determines one or more test item databases from which to select
test items,” and “[t]he test item databases are organized based on
psychometric and/or content specifications.” Swanson ʼ196, Abstr. In
modifying Swanson ʼ256 to have plural item collections by the teaching of
Swanson ʼ196, the combined teachings of the references would include that
each of the plurality of slots has its own content specifications. Thus, we are
not apprised of Examiner error by Appellants’ second argument.
Third, Appellants argue that Swanson ʼ256 does not disclose the
relationship between each slot that “‘each of the plurality of slots’ is
associated with ‘one of the plurality of item collections including a plurality
of eligible test items that meet the content specifications for the slot.’” App.
Br. 10-11. However, again Appellants have failed to address the combined
teachings of the references upon which the Examiner’s rejection was based.
See Merck, 800 F.2d at 1097; Keller, 642 F.2d at 426. The Examiner agreed
that Swanson ʼ256 does not disclose a plurality of item collections, but
modified Swanson ʼ256 to include a plurality of item collections by the
Appeal 2011-008673
Application 11/102,016

6
teaching of Swanson ʼ196. Accordingly, we are not apprised of Examiner
error by Appellants’ third argument.
Fourth, Appellants argue that Swanson ʼ256 “fails to teach that for
each slot there are a plurality of eligible test items that meet the content
specifications.” App. Br. 11. We are not persuaded of Examiner error by
Appellants’ fourth argument because the resulting method from the
combined teachings of the Swanson ʼ256 and Swanson ʼ196 references
would include the plurality of eligible test items in each of the plurality of
item collections meeting the content specifications for that particular item
collection according to the teachings of Swanson ʼ196.
Fifth, Appellants argue that Swanson ʼ256 “uses statistical properties
to control assembly of a test” including “a weighted deviations model and
algorithm for automated item selection” (see Swanson ʼ256, col. 7, ll. 36-
41), and “this is disadvantageous because of the complexity of calculations
required and also because there is never any guarantee that a test will meet
its specifications, just a statistical probability that it will.” App. Br. 11.
Appellants appear to be arguing that Swanson ʼ256 does not meet the
preambular language of “the method having the advantage of ensuring the
test will meet a content specification while reducing the computational load
required to assemble the test.” See App. Br., Clms. App’x.
We are not persuaded of Examiner error by Appellants’ fifth argument
because Appellants again have failed to address the combined teachings of
the references upon which the Examiner’s rejection was based. See Merck,
800 F.2d at 1097; Keller, 642 F.2d at 426. We also agree with the Examiner
that an attorney's arguments in a brief cannot take the place of evidence (see
In re Pearson, 494 F.2d 1399, 1405 (CCPA 1974); see also Ans. 7), and
Appeal 2011-008673
Application 11/102,016

7
Appellants have not provided any objective evidence that the combination of
Swanson ʼ256 and Swanson ʼ196 is incapable of ensuring the test will meet
a content specification while reducing the computational load required to
assemble the test.
Sixth, Appellants argue that “[t]he presence of step[] 125, the decision
step following step 125 and step 130 [in Figure 1 of Swanson ʼ196] illustrate
one fundamental difference” why Swanson ʼ196 does not teach claim 1’s
executing step. App. Br. 13. In other words, Appellants indicate that in
Swanson ʼ196, “after each slot is filled with an item, the test characteristics
are updated and a determination must be made whether the specifications
and characteristics match, and if not the test item selection must be revised.”
Id. Claim 1 “makes clear that the item collections contain eligible test items
that meet content specifications for the slot,” and “[t]hus, once an item from
the item collection is selected there is no need to make a determination of
whether the content specifications for the slot are met–the content
specifications are guaranteed to be met” so “there is never any need to revise
test item selection[s].” App. Br. 14. We are not apprised of error by
Appellants’ sixth argument because the Examiner does not modify Swanson
ʼ256 to include step 125, the decision step following step 125, and step 130
of figure 1 of Swanson ʼ196.
Finally, Appellants argue that the methodology of Swanson ʼ196 “is
not suitable for use for administering and assembling a test at the same time
(such as may be performed on a computer-based test)” since the
methodology of Swanson ʼ196 is iterative, i.e., “rel[ying] upon the potential
for revising test item selection if the test characteristics are inconsistent with
the test specifications.” App. Br. 14. Again, Appellants have failed to
Appeal 2011-008673
Application 11/102,016

8
address the combined teachings of the references upon which the
Examiner’s rejection was based. See Merck, 800 F.2d at 1097; Keller, 642
F.2d at 426. The Examiner relied upon the computer-based test of Swanson
ʼ256 and modified it by the teachings of Swanson ʼ196 to include a plurality
of item collections. Thus, Appellants’ final argument does not apprise us of
Examiner error.
Accordingly, we sustain the Examiner’s rejection of independent
claim 1, and claims 3, 4, and 22-24 which fall therewith, under 35 U.S.C.
§ 103(a) as unpatentable over Swanson ʼ256 and Swanson ʼ196.
Claim 26
Similar to independent claim 1, independent claim 26 is directed to a
computer-assisted method of test assembly and includes, inter alia, an
“executing” step. However, instead of reciting “the one of the plurality of
item collections including a plurality of eligible test items that meet the
content specifications for the slot” as is claim 1, claim 26 recites that “the
one of the plurality of item collections including only eligible test items that
meet the content specification for the slot.” App. Br., Clms. App’x.
Emphasis added.
The Examiner makes the same findings and conclusions for
independent claim 26 as those discussed supra for independent claim 1 (see
Ans. 3-4), and Appellants present the same arguments for independent claim
26 as those discussed supra for independent claim 1 (see App. Br. 16-17.)
For the same reasons as discussed supra, we are not persuaded of Examiner
error.
Claim 26 also recites, inter alia, a third step of “further executing the
test administration script to apply, using a computer, an optimization process
Appeal 2011-008673
Application 11/102,016

9
and randomization factor to select the item to fill the slot while controlling
statistical properties of the test based on the interim score and properties of
the items within the item collection associated with each of the plurality of
slots.” App. Br., Clms. App’x.
With respect to independent claim 26’s third and fourth steps, the
Examiner finds that Swanson ʼ256 discloses
further executing the test administration script to apply
optimization process and randomization factor to select the item
to fill the slots (see col. 12:20-35) while controlling statistical
properties based on the interim score and properties of the items
(see col. 14:50-65) and repeating the steps of obtaining interim
score and applying the optimization process until the test
assembled and administered (see col. 19: 10-25).
Ans. 3.
Appellants additionally argue that “neither reference alone or in
combination teaches applying an optimization [process] and randomization
factor to select the item to fill the slot while controlling statistical properties
of these based on the interim score and properties of the items within the
item collection associated with each of the plurality of slots.” App. Br. 17.
Appellants’ argument amounts to a recitation of the claim elements and a
“naked assertion” that the elements are not found in the prior art. Such
statements do not constitute a separate argument for patentability of claim 26
pursuant to 37 C.F.R. § 41.37(c)(1)(vii) (2011). See In re Lovin, 652 F.3d
1349 (Fed, Cir. 2011) (holding that the Board had reasonably interpreted 37
C.F.R. 41.37(c)(1)(vii) as requiring “more substantive arguments in an
appeal brief than a mere recitation of the claim elements and a naked
assertion that the corresponding elements were not found in the prior art”).
Appeal 2011-008673
Application 11/102,016

10
Thus, we are not apprised of Examiner error by Appellants’ additional
argument with respect to claim 26.
Accordingly, we sustain the Examiner’s rejection of independent
claim 26 under 35 U.S.C. § 103(a) as unpatentable over Swanson ʼ256 and
Swanson ʼ196.
Claims 5, 6, 13, and 25
Appellants argue independent claim 5 and do not separately argue
claims 6, 13, and 25 except to note that these claims depend from claim 5
and should be reversed along with claim 5. See App. Br. 17-18. We select
independent claim 5 as the representative claim to decide the appeal and
claims 6, 13, and 25 fall with claim 5.
Similar to independent claims 1 and 26, independent claim 5 is
directed to a computer-assisted method of test assembly and includes, inter
alia, the step of “executing a test administration script to associate each of
the plurality of slots of the test with one of the plurality of item collections
from which an item to fill the slot is to be chosen and to identify a lookup
table of optimization statistics and a degree of randomization for each slot
to use in selecting items from the one of the plurality of item collections, the
one of the plurality of item collections including a plurality of eligible test
items that meet the content specifications for the slot.” App. Br., Clms.
App’x. Emphasis added.
The Examiner makes a similar finding for claim 5 as that discussed
supra for claim 1. See Ans. 4. The Examiner additionally finds that
Swanson ʼ256 discloses “to identify a lookup table of optimization statistic
and a degree of randomization for each slot to use in selecting items from
Appeal 2011-008673
Application 11/102,016

11
the item collection.” Id. (citing Swanson ʼ256, col. 20, l. 55 to col. 21, l. 60,
and table 3).
Appellants argue that “[f]or the reasons previously expressed with
respect to claim 1, neither Swanson [ʼ]196 nor Swanson [ʼ]256 teach such a
methodology, nor would it have been obvious to combine these references in
the alleged manner.” App. Br. 17-18.
For the same reasons as discussed supra with respect to independent
claim 1, we are not persuaded of Examiner error.
Accordingly, we sustain the Examiner’s rejection of claim 5, and
claims 6, 13, and 25 which fall therewith, under 35 U.S.C. § 103(a) as
unpatentable over Swanson ʼ256 and Swanson ʼ196.
Claims 14 and 16-18
Appellants argue independent claim 14 and do not separately argue
claims 16-18 except to note that these claims depend from claim 14 and
should be reversed along with claim 14. See App. Br. 18. We select
independent claim 14 as the representative claim to decide the appeal and
claims 16-18 fall with claim 14.
Independent claim 14 is directed to a system of scripted testing and
includes, inter alia, “a test administration script stored on the computer
readable medium, the test administration script comprising a series of
instruction sets, one for each slot in the test, identifying for that slot the item
collection from which the item for that slot is to be chosen, and the lookup
table and degree of randomization to be employed in the selection.” App.
Br., Clms. App’x.
The Examiner makes a similar finding for claim 14 as those discussed
supra for claim 1 (see Ans. 4-5), and Appellants argue that “[f]or the reasons
Appeal 2011-008673
Application 11/102,016

12
previously expressed with respect to claim 1, neither Swanson [ʼ]196 nor
Swanson [ʼ]256 teach such a methodology, nor would it have been obvious
to combine these references in the alleged manner.” App. Br. 18. For the
same reasons as discussed supra with respect to independent claim 1, we are
not persuaded of Examiner error.
Accordingly, we sustain the Examiner’s rejection of independent
claim 14, and claims 16-18 which fall therewith, under 35 U.S.C.
§ 103(a) as unpatentable over Swanson ʼ256 and Swanson ʼ196.
Claim 19
Independent claim 19 is directed to an article of software for
assembling a test which includes, inter alia, “a test administration script to
associate each of the plurality of slots of the test with one of a plurality of
item collections from which an item to fill the slot is to be chosen and to
identify a lookup table of optimization statistics and a degree of
randomization for each slot to use in selecting items from the item
collection.” App. Br., Clms. App’x.
The Examiner makes the same findings and conclusions for
independent claim 19 as those discussed supra for independent claim 1 (see
Ans. 4-5), and Appellants present the same arguments for independent claim
19 as those discussed supra for independent claim 1 (see App. Br. 18-19).
For the same reasons as discussed supra, we are not persuaded of Examiner
error.
Accordingly, we sustain the Examiner’s rejection of independent
claim 19 under 35 U.S.C. § 103(a) as unpatentable over Swanson ʼ256 and
Swanson ʼ196.

Appeal 2011-008673
Application 11/102,016

13
DECISION
We AFFIRM the Examiner’s decision to reject claims 1, 3-6, 13, 14,
16-19, and 22-26.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).

AFFIRMED

Klh