Ex Parte Mayfield et alDownload PDFPatent Trial and Appeal BoardDec 13, 201714152123 (P.T.A.B. Dec. 13, 2017) Copy Citation United States Patent and Trademark Office UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O.Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 14/152,123 01/10/2014 Elijah Jacob Mayfield 143166.00101 1491 29880 7590 03/13/2018 FOX ROTHSCHILD LLP PRINCETON PIKE CORPORATE CENTER 997 LENOX DRIVE BLDG. #3 EAWRENCEVTT.TE, NJ 08648 EXAMINER HONG, THOMAS J ART UNIT PAPER NUMBER 3715 NOTIFICATION DATE DELIVERY MODE 03/13/2018 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): ipdocket @ foxrothschild. com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte ELIJAH JACOB MAYFIELD and DAVID STUART ADAMSON Appeal 2016-008243 Application 14/152,123 Technology Center 3700 Before DANIEL S. SONG, JAMES P. CALVE, and BRANDON J. WARNER, Administrative Patent Judges. CALVE, Administrative Patent Judge. DECISION ON REQUEST FOR REHEARING Appellants request rehearing of the Decision, mailed December 15, 2017, pursuant to 37 C.F.R. § 41.50(b). Requests for Rehearing are limited to matters overlooked or misapprehended by the Board in rendering a decision. See 37 C.F.R. § 41.52(a)(1). The Request for Rehearing is DENIED. Appeal 2016-008243 Application 14/152,123 In the Decision, we affirmed the Examiner’s rejection of claims 1—24 as directed to a judicial exception to 35 U.S.C. § 101 because the claims were directed to mental processes, mathematical formulas, and manipulation of data using generic computers and conventional functions. Dec. 4—10. In their Request for Rehearing, Appellants argue that the claimed method and system do not simply use a computer as a tool but significantly improve upon prior art systems including the functioning of the computer. Req. Reh’g 2. In particular, Appellants argue that prior art systems are not fully automated, require human input, and lack artificial intelligence. Id. Appellants argue that their improvements use extractors to extract a plurality of feature values for each training essay, build a model by assigning a probability to each of a plurality of combinations of class values and feature values for the training essays, associate the model with a common prompt, and apply the model to feature values of a candidate essay. Id. at 2—3. Appellants’ argument that the claims “provide features of artificial intelligence that were not available in prior essay scoring systems” is raised for the first time in the Request for Rehearing and thus is not a persuasive argument that we overlooked this matter in our Decision. Appellants do not identify where they raised this issue in their Appeal Brief or Reply Brief. Id. The term “artificial intelligence” does not appear in the Specification, which describes methods and systems of evaluating textual responses with machine learning. Spec. 134. However, the claims do not require machine learning. Instead, the claims recite a computer-implemented method and system that receive human assessments of training essays, extract features therefrom, build a model by assigning a probability to combinations of class values and feature values, and machine-generate a predicted score/grade for the essay. 2 Appeal 2016-008243 Application 14/152,123 Claim 1 recites “using one or more extractors to extract a plurality of feature values for each of a plurality of features.” Appeal Br. 26 (Claims Appendix). Claim 1 thus requires the use of only one extractor rather than a “set of extractors.” Claim 1 does not recite how extractors are selected or defined. Claim 1 does not recite any features of the extractors other than to “extract a plurality of feature values for each of a plurality of features.” Id. The Specification discloses that class and feature values can be received as metadata or separate inputs associated with a document. Spec. 135. These values are based on a human rating process and written rubric. Id. 141. The claimed method and system are not fully automated, contrary to Appellants’ suggestion. Req. Reh’g 2. They require human input, like prior art systems, by “receiving a human assessment for the training essays, wherein the human assessment comprises a class value that comprises a grade or score of the training essay.” Appeal Br. 26, 29 (Claims Appendix).1 They thus rely on a “human rating process [that] should follow a written rubric, and the design and development of this rubric should be iterated until humans reach a high level of inter-rater reliability.” Spec. 141. Once the assessment is defined by humans this way, the system applies this defined assessment to documents. Id. 142. The more closely the training essays approximate future responses on essays, the better the model can “replicate human evaluation of those future responses.” Id. 140; see id. 112 (prior art systems use expert-defined features for regression analysis). Any 1 Independent claim 11 recites “receive a class value for the training essay, wherein the class value comprises a score or grade that resulted from human evaluation of the training essay.” Id. at 30 (Claims Appendix). 3 Appeal 2016-008243 Application 14/152,123 machine-learning used in the method and system (id. ]Hf 34, 38, 42) is not claimed. Just as teachers traditionally identify a set of features in an essay that provide a basis for scoring or grading the essay (id. 12), the claimed method and system use “one or more extractors to extract a plurality of feature values for each of a plurality of features” from training essays, build a model that extracts this set of feature values from candidate essays, and predict a score for each candidate essay based on mathematical probabilities assigned to the class values (grades) and feature values (parts of answers). Thus, like teachers in the past, the claimed method and system identify features in an essay that are relevant to scoring an essay and determine what score to give an essay by assigning probabilities to various combinations of features, i.e., that a particular set of features in an essay should receive a particular score. The predicted score is based on human assessments of the training essays. Such grading methods and systems are the quintessence of the mental steps and processes that humans have undertaken in the past to grade essays. Educators and prior art systems define a set of features that are correlated to essay quality (e.g., essay length, text coherence) and assign values to each feature in the set to assign scores to essays. Id. Tflf 2—3. Each “feature” is “a unique, easily identifiable characteristic of a written text that, for a particular document, can be associated with a numeric value.” Id. 121. That such well-known mental steps and processes may be performed on a computer does not transform the abstract idea into a patent eligible one. Even the extraction of feature values using extractors can be done by “extracting data from a document file, by receiving metadata or separate inputs that are associated with the document, or by analyzing the document 4 Appeal 2016-008243 Application 14/152,123 through suitable methods such as optical character recognition (OCR).” Id. 135. Thus, feature values may be defined by humans. Cf. Req. Reh’g 3. Therefore, just as teachers and prior art systems defined a rubric of features to be used to grade essays, the claimed extraction identifies a list of such features and incorporates those features into a model. Id. 141. If there is novelty in the extraction process, it does not reside in the claims. To the contrary, Appellants disclose extraction as receiving metadata or separate input to prepare a list of feature values just as teachers and prior art systems have used defined rubrics to grade essays in the past. Even if a document is analyzed through OCR, the Specification does not explain how such OCR analysis generates a list of feature values for each training essay. See id. To the extent the claims recite computer components, the computer components merely implement this abstract idea in conventional fashion. No improvement to the computer functionality is claimed. The steps of data extraction, data processing, data manipulation, calculations of probabilities, manipulating existing information to generate additional information, and display of results are basic functions that computers perform and do not make such abstract ideas patent eligible. Dec. 7 (citations omitted). The claimed model merely “builds a comprehensive set of probability estimates for each feature with regard to a class value [wherein] [t]hese probabilities now may estimate a feature’s likelihood of appearing in essays that exhibit a given class value, rather than merely increasing or decreasing the final estimated output score.” Spec. 172. A probable class value can be either PASS or FAIL. Id. at || 82—85. The Specification discloses the use of various undisclosed algorithms and Naive Bayes algorithms to predict class values. However, the claims are not so limited even if these known 5 Appeal 2016-008243 Application 14/152,123 algorithms are employed in some new manner, of which we have no evidence before us. See id. 38, 43, 53 62, 69—86. The claimed method and system determine a “probable class value” for an essay with a particular set of feature values and the probable class value “comprises a machine-generated predicted grade or score for the candidate essay.” In other words, the claimed model applies mathematical formulas or probabilities, i.e., it performs calculations at a high level of generality, to estimate a probable class value (score, grade). Id. 1—2, 41. The claims are directed to the abstract idea of grading essays. That the claims recite the use of probabilities to do so does not alter the fact that the claims are directed to an abstract idea that uses mathematical formulas and mental processes and mental steps that humans have long performed and can perform using pen and paper. That the claims implement the abstract idea on a generic computer using conventional functions does not make the abstract idea patent eligible.2 The use of probabilities to score essays is not shown to be an improvement over prior systems that calculate a fixed score. Associating a model with a common prompt merely claims the use of a particular model for a particular essay. It is known to use a selected rubric to score a particular essay. Spec. Tflf 1—2, 41. The model’s accuracy depends also on how well-defined a topic is so writers understand the type of essay expected, how closely human-graded training essays approximate future 2 Appellants disclose conventional internal hardware components including electrical bus 400, central processing unit 405 that is a processing device that may include a single processing device or two or more processing devices, read only memory 410, random access memory 415, controller 420, memory devices 425, display interface 430, display 435, communication ports 440, interfaces 445 for keyboard 450 or other inputs 455. Id. 87—91, Fig. 4. 6 Appeal 2016-008243 Application 14/152,123 responses to that essay, and whether the written rubric used to grade training essays has a high level of inter-rater reliability. See id. 39-41. It is not even clear that the claimed method and system improve the abstract idea of grading essays. We have no evidence of unexpected results or improved accuracy in predicted grades or scores. The alleged improved “functioning of the electronic device by extracting and discovering features without being limited to a specific set of curated features that are identified by human authors” (Req. Reh’g 3) is contradicted by Appellants’ disclosure that to build a model, the system receives class value and feature values by receiving metadata or separate inputs associated with a document. Spec. 135. Thus, some human involvement can occur to create class values and feature values for a model. In addition, a human rating process that follows a written rubric is used to develop assessment evaluations that the system applies to documents. Id. H 41 42. The design of the rubric should be iterated until humans reach a high level of inter-rater reliability. Id. 141. The Federal Circuit’s decision in Core Wireless Licensing S.A.R.L. v. LCElec., Inc., 880 F.3d 1356 (Fed. Cir. 2018) does not alter our decision. In fact, it supports our decision that the mere automation of an abstract idea on generic computers that perform conventional functions of data collection, extraction, processing, calculations, and display do not transform an abstract idea into patent eligible subject matter absent an improvement in computer functionality. Dec. 8—10. In Core Wireless, the claims were directed to an improved user interface for computing devices with small screens that thus “disclose a specific manner of displaying a limited set of information to the user, rather than using conventional user interface methods to display a generic index on a computer.” 880 F.3d at 1362—63. Here, the claims only 7 Appeal 2016-008243 Application 14/152,123 require a display device, audio device, or transmitter to output a predicted grade or score of a probable class value. Appeal Br. 26 (Claims Appendix). The claimed display, audio device, or transmitter outputs a predicted grade or score in a known, conventional manner. Appellants disclose that a display interface 430 may permit information from bus 400 to be displayed on display 435 in audio, visual, graphic, or alphanumeric format. Spec. 190. Appellants provide no other information regarding the displays or the display process (for example, display 435 is illustrated as a box in Figure 4). Improvements to an abstract idea itself are still abstract ideas that are not patent eligible. See Visual Memory LLC v. NVIDIA Corp., 867 F.3d 1253, 1258 (Fed. Cir. 2017) (“[W]e must. . . ask whether the claims are directed to an improvement to computer functionality versus being directed to an abstract idea.” (internal quotations omitted); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36 (Fed. Cir. 2016) (“[T]he first step in the Alice inquiry in this case asks whether the focus of the claims [was] on the specific asserted improvement in computer capabilities ... or, instead, on a process that qualifies as an ‘abstract idea’ for which computers are invoked merely as a tool.”). Limiting an invention to a technological environment does “not make an abstract concept any less abstract under step one.” Intellectual Ventures I LLC v. Capital One Fin. Corp., 850 F.3d 1332, 1340 (Fed. Cir. 2017). Claiming the improved speed or efficiency inherent with applying the abstract idea on a computer does not provide a sufficient inventive concept. Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1315 (Fed. Cir. 2016). The claims are not directed to a solution to a “technological problem.” In re TLI Commc’ns LLP Patent Litig., 823 F.3d 607, 613 (Fed. Cir. 2016). They are not shown to improve computers or 8 Appeal 2016-008243 Application 14/152,123 even grading systems. They rely on human assessments. As discussed below, they are broad enough to encompass prior art automated systems. In Finjan, Inc. v. Blue Coat Systems, Inc., 879 F.3d 1299 (Fed. Cir. 2018), the Federal Circuit based its finding of patent eligibility on the fact that the claimed virus screening “constitutes an improvement in computer functionality.” 879 F.3d at 1304. The claims in Finjan provided a non abstract improvement in computer functionality by enabling more flexible and nuanced virus filtering including protecting against previously unknown viruses and viruses that are cosmetically modified to avoid detection by code-matching virus scans. Id. at 1304—05. It is not clear how the claimed method and system improve computers or known essay grading methods or systems that also score essays using an applied rubric of values. The claimed method and system determine “a probable class value” for a candidate essay, where “the probable class value comprises a machine-generated predicted grade or score for the candidate essay.” Appellants disclose that such “predicting” “produc[es] an output of probability estimations for each possible class value for a document.” Spec. 132. The Specification discloses probability estimates for multiple scores (class values) for a document may be calculated. See id. ]Hf 52, 53 Fig. 1 (step 125). In such case, the most probable class value may be used to select a candidate value as the predicted value, or the system may output multiple predictions with confidence values associated with each prediction. Id. 1 53. Alternatively, the probabilities may be reduced to a single predicted class value (the most probable of all options). Id. Still other algorithms that do not assign probabilities to each class values may be used. Id. In view of these disclosures, it is not clear whether the claims provide an actual grade 9 Appeal 2016-008243 Application 14/152,123 or score for an essay and, if so, exactly what type of score(s) is calculated and displayed, or how they differ from conventional grade or score. Appellants disclose that known automated grading schemes analyze as few as twelve features with simple algorithms like linear regression, whereas Appellants’ method and system can use thousands of features. Id. 1 65. As a result, Appellants indicate that linear regression “may not be a suitable algorithm for determining a score class given a set of features.” Spec, f 65 (emphasis added). However, the claims merely require the extraction and use of “a plurality of features.” Appeal Br. 26 (Claims Appendix). Thus, known linear regression algorithms may be used in the claimed method and system. See id. 1—2, 65, 88. The claimed model and machine-generated predicted grade or score encompass software functionality used in admitted prior art grading systems. Id. 2, 27. The claimed probabilities can be calculated using known regression models. See id. ]Hf 1—2, 65, 88. The accuracy of the predictive elements also depends on how closely training essays approximate the range of responses provided in the candidate essays. Id. 140. However, the claims do not recite features that improve the accuracy of predicting class values and scores in this way. Moreover, using a larger number of features, as compared to the prior art, while potentially increasing accuracy, does not necessarily impart patentability. Nor do Appellants explain how the claimed method and system can improve the functioning of a computer sufficiently to render the abstract idea patent eligible. See Req. Reh’g. 5. The prior art and claimed method and system grade essays by extracting feature values from essays according to a rubric/model. Spec. Tflf 1—3. The claims do not specify a quantity or quality of feature values to distinguish over prior art systems. The extraction can 10 Appeal 2016-008243 Application 14/152,123 receives class values and feature values from a document file, metadata, or separate inputs versus from experts in prior art systems. Id. 2, 35. The claimed method and system merely obtain feature values from a set of essays that were graded by humans and apply those feature values as a rubric to candidate essays to predict a score for those candidate essays based on the feature values in each candidate essay. Id. Tflf 3, 32, 37, 39, Fig. 1. Appellants have not claimed improved computer functionality in extracting feature values from training or candidate essays. Nor do Appellants claim technological improvements in the way a model is built from extracted class values and feature values or the way a model is applied to candidate essays. The predictive elements depend on training essays that vary in quality and writer skill level and approximate responses in candidate essays. Id. 140. Like conventional grading systems, the claimed method and system extract feature values from candidate essays to be graded and apply a model to the extracted feature values to determine a probable class value and a predicted score. The claims thus recite conventional computer functions of data extraction, calculations according to mathematical formulas, and the display of the results of the data calculations. Unlike the claims in Finjan, we have no evidence the claimed method and system improve the accuracy or efficiency of the grading process or system. Finjan, 879 F.3d at 1304 (the behavior-based scans analyze a downloadable program’s code to determine if it performs potentially dangerous and unwanted operations like renaming or deleting files and thus can protect against previously unknown viruses as well as “obfuscated code” (known viruses that are cosmetically modified to avoid detection by prior art code-matching virus scans)). In contrast, the claimed method and system try to replicate human grading of essays. Spec. 11 Appeal 2016-008243 Application 14/152,123 140. The model’s reliability depends on the inter-relater reliability of a written rubric used by human graders of the training essays. See id. 141. The Examiner analyzed the claims and limitations and determined that they were directed to steps that can be performed in the human mind, or by a human using a pen and paper, as well as also organizing information through mathematical correlations and mathematical relationships and algorithms. Final Act. 2-4; Ans. 2—3. Appellants did not address these findings of the Examiner. See Appeal Br. 11—20. We agreed with the Examiner. Dec. 5—6. That a computer may perform data extraction or calculations more quickly or efficiently than a human does not impart patent eligibility where the processes and calculations are conventional ones performed using generic computers. Id. at 6—8. As discussed above, extracting features from essays has been done by educators for a long time to grade essays. Even automated essay grading systems extract key features related to grading of the essays. Spec. H1—2. To grade an essay, the grader must identify features of the essay that relate to a grading rubric, positively, negatively, or otherwise. If features in an essay do not match perfectly with a grading rubric, a grader must determine how to grade the essay where responses do not match perfectly with a rubric’s proposed answer key. The claims use probabilities to capture this ambiguity inherent in grading essays. However, Appellants have put forth no evidence that any of the extraction, filtering, classification, and probability calculations produce more accurate grading of essays. The extracted features may include words, character strings, similar elements, semantic analysis, word size, sentence size. Id. 144. The extractor may convert a text into a set of labeled numeric values representing the contents 12 Appeal 2016-008243 Application 14/152,123 of that text. Id. 45 46. Even the prediction process often selects several candidate class values (grades) for a text. Id. 1 53. We are not persuaded that the claimed method and system represent an improvement in computer functionality or other relevant technology that elevates the claims from an abstract idea or that provides an inventive step to make the abstract idea patent eligible, whether the claims are analyzed individually or as an ordered combination. See Dec. 8—10; Ans. 3—7. To summarize, extracting feature values from essays to build a model to score candidate essays (Req. Reh’g 2—3) is an abstract idea and process that educators and teachers have done to develop a rubric to grade essays. Spec. 1—2, 41. Prior art automated grading systems do this as well using expert-designed features. Id. 11. The extraction and building steps also involve mental processes that people undertake to build a rubric for grading essay. They also involve standard data manipulation steps that routinely extract pertinent data from documents that are abstract ideas. Building a model by assigning a probability to combinations of class values and feature values for training essays is an abstract idea that people have performed when preparing rubrics. Using mathematical formulas to assign probabilities merely replicates the human thought process of figuring out what particular combinations of features yield particular scores with a particular degree of certainty. It also addresses situations where features of an essay do not fall squarely into a particular class value (grade). Thus, an essay may not have sufficient combination of features to yield an A, B, C, D, or F. The reduced probability of an essay having such a grade may equate to a grade such as A+, A-, B+, B-, and so forth in the interstices. The claimed probabilities mimic teachers evaluating essay answers that do not match the 13 Appeal 2016-008243 Application 14/152,123 features of a written rubric perfectly and require extrapolation from a rubric to match actual essay answers to grade an essay with a level of certainty. The steps of associating a model with a common prompt (essay question) and applying the model to feature values of an essay to determine a probable class value for the candidate essay (Req. Reh’g 3) are directed to an abstract idea of grading essays as educators have done. The proper rubric must be used to grade each essay. Spec. Tflf 1—2, 41. Teachers or graders apply the rubric to essays to be scored as mental processes that also may be performed with pen and paper. Prior art automated systems apply expert- defined rubrics and curated features to essays to score the essays. Id. 12. These steps also involve conventional computer functions applying formulas and mathematical calculations to extracted data to determine a score. Using probabilities is an abstract idea that recognizes the common sense notion that candidate essays to be scored may present different feature values than a rubric. Each combination may be different or unique in ways that require extrapolation or approximation to yield a score from a written rubric. Thus, the more accurately training essays approximate the range of answers from candidate essays, the more likely “a classifier will may [sic] be able to replicate human evaluation of those future responses.” Id. 140. Prior art systems use regression analysis to approximate or extrapolate a score from different combinations of different extracted values to emulate the mental process of grading essays. Id. 12. Appellants also use regression analysis for small numbers of features values, which the claimed method and system encompass by not requiring a large numbers of features. Id. 1 65. As discussed in the Decision, our reviewing court treats such mental processes, data manipulation steps, mathematical formulas and traditional 14 Appeal 2016-008243 Application 14/152,123 processes automated with generic computers to perform such conventional functions as directed to abstract ideas that are not patent eligible. Dec. 5—10. For all of the foregoing reasons, we are not persuaded by Appellants’ arguments that we have overlooked any matter on this appeal. TIME PERIOD FOR RESPONSE No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv). REHEARING DENIED 15 Copy with citationCopy as parenthetical citation