Opinion
No. 92,038
September 17, 1998
Original Proceeding — The Florida Bar.
John O. Williams, pro se, Tallahassee, for Appellant, pro se.
Thomas M. Ervin, Jr. of Ervin, Varn, Jacobs Ervin, Tallahassee, and John F. Harkness, Jr., Executive Director, The Florida Bar, Tallahassee, for Board of Legal Specialization Education, Appellee.
John O. Williams, a member of The Florida Bar (the Bar), appeals from the decision of the Board of Governors of The Florida Bar (the Board of Governors) denying him certification as a "Board Certified Real Estate Lawyer" because he failed to achieve a passing score on the 1996 certification examination. We have jurisdiction. Art. V, § 15, Fla. Const. Based on the following, we affirm the Board of Governors' decision.
On May 14, 1996, Williams took the examination required for certification as a "Board Certified Real Estate Lawyer." The exam, developed by the Real Estate Law Certification Committee, consists of three parts and is scored on a 100-point scale. Part 1 (45 points) consists of sixty multiple choice questions; part 2 (44 points) consists of eight essay questions, four of which must be answered; and part 3 (11 points) consists of real estate closing documents that must be completed. Pursuant to Standing Policy 2.08(d) of the Board of Legal Specialization and Education (BLSE policy), the certification committee is also responsible for overseeing the grading of the examination.
Rule 6-3.2 of the Rules Regulating The Florida Bar requires the establishment of a certification committee for each area of practice available for certification, which currently encompasses 16 practice areas. See R. Regulating Fla. Bar 6-4.1 (civil trial law), 6-5.1 (taxation), 6-6.1 (marital and family law), 6-7.1 (wills, trusts, and estates), 6-8.1 (criminal law), 6-9.1 (real estate), 6-11.1 (workers' compensation law), 6-13.1 (appellate practice), 6-14.1 (health law), 6-15.1 (immigration and nationality law), 6-16.1 (business litigation), 6-17.1 (admiralty and maritime law), 6-18.1 (city, county and local government law), 6-19.1(aviation law), 6-20.1 (elder law), 6-21.1 (international law). These certification committees, which consist of nine board-certified lawyers in the relevant area of practice, are responsible for, among other things, reviewing and establishing examination procedures deemed necessary for certification. See R. Regulating Fla. Bar 6-3.3(c).
On July 8, 1996, Williams received a letter informing him that he had failed to achieve a passing score on the certification exam. Specifically, he had received a score of 65.08 (out of 100), while the minimum passing score was 69.29. As a result, Williams pursued an initial exam review pursuant to BLSE policy 2.08(e), during which he identified two multiple choice questions and two essay questions which he felt had been graded improperly. On August 16, 1996, he filed a petition for grade review pursuant to BLSE policy 2.08(f).
A grade review panel considered Williams' petition on October 29, 1996, pursuant to BLSE policy 2.08(f)(3). The panel's decision provided that after considering Williams' petition for grade review, his exam answers, the exam itself, the model answers, and the range finders, his score on one of the essay questions would be increased, but the panel deemed all other grading to be correct. With the increase on one of his essay answers, Williams' overall exam score rose to a 66.91, but that score still did not meet the passing standard of 69.29. Thus, Williams remained ineligible for certification, and he appealed to the Appeals Committee of the Board of Governors (the Appeals Committee) pursuant to BLSE policy 2.08(g).
In his petition for grade review, Williams admitted that he did not address certain issues discussed in the model answers to the two challenged essay questions. While he argued that many of the issues he failed to address were outside the scope of the questions asked, he unqualifiedly admitted missing several of the issues.
The Appeals Committee issued its decision on March 19, 1997, remanding the matter to the grade review panel for reconsideration of Williams' essay answer that had not received a score increase during the first panel review. The decision of the Appeals Committee was silent as to Williams' other claims but stated that the grade review panel should render its decision at least ten days in advance of the May 13, 1997, administration of the real estate certification exam. The Appeals Committee also directed that the panel's decision on remand would be final and not subject to further review by the Appeals Committee. The grade review panel reconvened on April 9, 1997, and unanimously affirmed its earlier ruling in a written decision rendered the same day. As a result, in light of the earlier ruling by the Appeals Committee that it would not review the panel's decision after remand, Williams appealed to the Board of Governors pursuant to BLSE policy 4.10(a).
The Board of Governors commenced its review of Williams' appeal on November 21, 1997. After considering the parties' briefs and hearing oral argument, the Board of Governors issued its decision on December 2, 1997, affirming the rulings of the grade review panel and the Appeals Committee and denying Williams certification. Williams now appeals that decision to this Court pursuant to Rule 6-3.9 of the Rules Regulating The Florida Bar and BLSE policy 4.11. He claims that (1) the holistic grading method used to score the essay portion of the real estate law certification examination is arbitrary and capricious; (2) the certification examination grade review process is arbitrary and capricious; and (3) he was not awarded appropriate credit on two of his essay answers. We disagree.
Under the holistic grading method, examination graders read essay answers and rate them for overall impression of quality rather than analytically, which would require a compilation of points awarded for specific features of an answer. Graders are trained to use the holistic grading method before the actual scoring process begins, and their scoring tendencies are standardized by using answers from a prior test administration.
Once the actual scoring process begins, graders employ model answers and a six-point grading scale to identify "range finders,"which are essay answers from the current examination that typify each point on the six-point grading scale. After identifying the range finders, graders read and score the remainder of the essay answers, and an answer receiving a different score from the graders is reviewed to arrive at a consistent grading decision. If the graders are unable to reach an agreement, the certification committee chairperson serves as a referee to facilitate an agreement.
The grading scale, originally developed for use on the Multistate Essay Examination, consists of the following:
6 A 6 answer demonstrates a high degree of competence in response to the question. While not reserved for a perfect answer, a 6 answer demonstrates a full understanding of the facts, a complete recognition of the issues presented and the applicable principles of law, and a good ability to reason to a conclusion. A 6 answer is clear, concise, and complete.
5 A 5 answer demonstrates clear competence in response to the question. A 5 answer demonstrates a fairly complete understanding of the facts, recognizes most of the issues and applicable law, and reasons fairly well to a conclusion.
4 A 4 answer demonstrates competence in response to the question. A 4 answer demonstrates an adequate understanding of the facts, an adequate recognition of most of the issues and law, and adequate ability to reason to a conclusion.
3 A 3 answer demonstrates some competence in response to the question but is inadequate. A 3 answer demonstrates a weak understanding of the facts, misses significant issues, fails to recognize applicable law, and demonstrates inadequate reasoning ability.
2 A 2 answer demonstrates only limited competence in response to the question and is seriously flawed. A 2 answer demonstrates little understanding of the facts or law and little ability to reason to a conclusion.
1 A 1 answer demonstrates fundamental deficiencies in understanding facts and law. A 1 answer shows virtually no ability to reason or analyze.
We reject Williams' challenge to the holistic grading method and agree with other courts that have upheld similar grading methods in the bar examination context. See In re Obermeyer, 717 P.2d 382, 385 (Alaska 1986), receded from on other grounds by In re Bettine, 840 P.2d 994, 997 (Alaska 1992); In re Lamb, 539 N.W.2d 865, 868 (N.D. 1995) (quoting Julia C. Lenel, The Essay Examination Part III: Grading the Essay Examination, The Bar Examiner, August 1990, at 17, for the proposition that "[b]oth analytic and holistic scoring methods have been shown to be reliable methods for scoring essays when they are well-designed and readers receive appropriate training"), cert. denied, 518 U.S. 1008, 116 S.Ct. 2530, 135 L.Ed.2d 1054 (1996). In Obermeyer, an examinee who failed to achieve a passing score on the bar examination challenged the "benchmark" grading method used to score the essay portion of that exam. The Supreme Court of Alaska summarized the "benchmark" grading method as follows:
To grade the essay portion of the exam, graders first meet to calibrate a particular essay question. At least five graders read and individually score five randomly selected answers on a scale of one to five. They repeat this process several times, with each grader reading at least twenty exams, until the graders agree on a set of answers that are representative of each of the five possible levels. These "benchmark" answers are then used as guides in assigning scores to the remaining papers. Two graders read each essay and score it. If the scores they assigned differ by more than one point, the two graders refer to the benchmarks and resolve their differences. If the scores are within one point, they are simply averaged.
717 P.2d at 385. In rejecting the bar examinee's challenge to the "benchmark" grading method, the court found that the method was reasonable and within the bar's discretion. See id. at 387.
The holistic grading method used to score the essay portion of the real estate law certification examination is similar to the "benchmark" grading method upheld in Obermeyer. The exam graders, who are board certified attorneys in the relevant area of the law, are trained to use the holistic grading method, and their scoring tendencies are standardized in calibration sessions. The graders reread essay answers over which there is a scoring discrepancy and arrive at a consistent grading decision by agreement or, if unable to reach an agreement, by having the certification committee chairperson serve as a referee to facilitate an agreement. These procedures provide multiple levels of protection to ensure that scores reached on particular essay answers are reliable. We find that the holistic grading method is reasonable and not arbitrary or capricious.
We next address Williams' challenge to the certification examination grade review process. The first step of the grade review process involves an initial exam review, at which the lawyer and the lawyer's counsel may inspect a copy of examination, the model answers, and the lawyer's own examination papers. See BLSE policy 2.08(e)(1), (2). After conducting this inspection, if the lawyer feels that any errors were made in grading the examination, the lawyer may file a petition for grade review with the Legal Specialization and Education Director. See BLSE policy 2.08(f)(1). The lawyer's petition for grade review and "any supporting authority deemed appropriate to substantiate the claim of incorrect grading" is considered by a grade review panel, which consists of three board-certified lawyers in the relevant area of law and a non-voting chairperson who is a member of the BLSE. See BLSE policy 2.08(f)(2), (3). No member of the panel may have been involved in the initial examination process as either a committee member, drafter, or grader, see BLSE policy 2.08(f)(3), and all information submitted to the panel must be in "blind form" to ensure that the lawyer's identity remains anonymous. See BLSE policy 2.08(f)(4). The panel reviews the substantive basis of lawyer's claims and renders a decision, which closes the grade review process. See BLSE policy 2.08(f)(4), (5).
Once the grade review process is closed at the grade review panel level, either the lawyer or the BLSE may file an appeal before the Appeals Committee that is "limited to the procedural issues set forth in the BLSE policies and to clear and convincing allegations of fraud, discrimination, or arbitrary or capricious action." See BLSE policy 4.03(a); see also BLSE policy 2.08(g). After the Appeals Committee makes its ruling, either the lawyer or the BLSE may appeal that decision to the Board of Governors, seeBLSE policy 4.10, then to this Court. See R. Regulating Fla. Bar 6-3.9; BLSE policy 4.11.
At the time Williams' appeal was considered by the Board of Governors, the BLSE policies did not contain a standard of review to be applied by that entity. However, the BLSE policies were amended on November 21, 1997, to reflect the following: (1) a request for review of the Appeals Committee decision before the Board of Governors must "clearly demonstrate that the [Appeals Committee] misconstrued the rules or the issue on appeal has significant programmatical impact," see BLSE policy 4.10(a); (2) if the request for review is denied, the decision of the Appeals Committee stands as the Board of Governors' decision, see BLSE policy 4.10(d); and (3) if the request for review is granted, the Board of Governors will apply a "clear and convincing error" standard of review in determining whether to uphold or reverse the decision of the Appeals Committee. See BLSE policy 4.10(e).
After considering the various aspects of the certification examination grade review process, we disagree with Williams that the process is arbitrary and capricious. The process affords a lawyer who has failed a certification examination a sufficient opportunity to identify any possible errors in grading and to anonymously present argument concerning those alleged errors to a panel of specialists in the pertinent area of law who have had no prior involvement with the examination. Further, the lawyer may seek a limited appeal before the Appeals Committee, the Board of Governors, and this Court. Thus, it is apparent that there are numerous safeguards present in the grade review process that protect against the possibility of fraud, discrimination, or arbitrary or capricious conduct. Cf. Davidson v. New York State Board of Law Examiners, 86 Misc.2d 744, 382 N.Y.S.2d 418, 420-21 (N.Y.Sup.Ct. 1976) (finding that board of law examiners had provided fair and reasonable procedures for review of bar examination grades where failing examinee could inspect relevant exam materials and have alleged grading errors reviewed by a member of the board); Hooban v. Board of Governors, 85 Wn.2d 774, 539 P.2d 686, 689-980 (Wash. 1975) (finding that extensive exam review by three experienced bar examiners who did not participate in the initial exam grading process sufficiently established the "general fairness"of the bar exam), appeal dismissed, 424 U.S. 902, 96 S.Ct. 1092, 47 L.Ed.2d 306 (1976).
Finally, we address Williams' claim that he was not awarded appropriate credit on two of his essay answers. In making this claim, Williams essentially asks this Court to regrade his two essay answers and award him a higher score on each. Courts in the bar examination context have consistently refused to regrade examination answers where there were adequate procedures for review of failing papers. See, e.g., Singleton v. Louisiana State Bar Association, 413 F. Supp. 1092, 1100 (E.D. La. 1976) (refusing to review bar examinee's essay answers where examination procedures already afforded three levels of review for failing papers); McGinn v. State Bar Board, 399 N.W.2d 864, 867 (N.D. 1987) (finding that regrading bar examination answers was not proper function on review); see generally 7 Am. Jur. 2d Attorneys at Law § 20 (1997). Courts in the bar examination context also have consistently refused to question the integrity of examination results absent clear and convincing allegations establishing fraud, imposition, discrimination, manifest unfairness, or arbitrary or capricious conduct. See, e.g., Scinto v. Stamm, 224 Conn. 524, 620 A.2d 99, 106-107 (Conn.), cert. denied, 510 U.S. 861, 114 S.Ct. 176, 126 L.Ed.2d 136 (1993); Lambert v. Board of Bar Examiners, 576 A.2d 430, 431 (R.I. 1990) (affirming Board of Bar Examiners' decision concerning the grading of the petitioner's bar exams because he failed to demonstrate that the board "abused its discretion" or that it was "clearly wrong" in conducting and evaluating his bar exams).
The above decisions in the bar examination context are consistent with both BLSE policy 2.08(f)(5), which provides that "[p]anel decisions shall close the grade review process," and BLSE policy 4.03(a), which limits appeals before the Appeals Committee to "the procedural issues set forth in the BLSE policies and to clear and convincing allegations of fraud, discrimination, or arbitary or capricious action." See also BLSE policy 2.08(g). If we were to regrade a lawyer's certification examination answers in the absence of a clear and convincing showing of fraud, discrimination, or arbitrary or capricious conduct, not only would we effectively nullify BLSE policies 2.08(f)(5), 2.08(g), and 4.03(a), but we would also increase the possibility of arbitrary grading because (1) the lawyer's anonymity is lost after the grade review panel level; and (2) we would not have had the benefit of a calibration session and the experience of grading multiple essay answers. See Scinto, 620 A.2d at 105-106. Accordingly, we will not question the validity of a score on particular certification examination questions unless the appellant clearly and convincingly shows that the score was arrived at through fraud, discrimination, or arbitrary or capricious conduct.
Cf. Haines City Community Dev. v. Heggs, 658 So.2d 523, 530 (Fla. 1995) (stating that "[a]s a case travels up the judicial ladder, review should consistently become narrower, not broader").
We use the term appellant here because either a lawyer who receives a failing certification examination score or the BLSE may pursue an appeal in this Court pursuant to BLSE policy 4.11.
Applying this standard to Williams' claim, we find that he fails to meet his required burden. Williams has not shown that the certification examination grading and grade review processes are arbitrary and capricious, nor has he shown that those processes were improperly applied to him. His sole allegation is that the model answers used in grading his two essay questions were each erroneous as to one issue. Even if we were to take his allegation as true, Williams would not have shown that his answers were graded arbitrarily and capriciously.
Under the holistic grading method, Williams' essay answers were viewed as a whole, not as a sum of component parts. In other words, even if Williams was correct as to the one additional issue in each of his essay answers, that would not necessarily correlate to a higher score on his essay answers, especially in light of his admission in his petition for grade review that he had overlooked certain issues in each of his essay answers. See supra note 2. Therefore, Williams has failed to clearly and convincingly show that the scores on his two essay questions were arrived at arbitrarily or capriciously after being graded by the certification committee exam graders, regraded by the grade review panel, and reviewed by the Appeals Committee and the Board of Governors. Accordingly, we decline to overturn the grading decision regarding his two essay answers. Cf. McGinn, 399 N.W.2d at 867 (rejecting bar examinee's contention that two of her essay answers should have received higher scores on grounds that "[i]t is a difficult burden to show that so many competent persons who reviewed [the examinee's] answers, and were in close agreement as to the quality of those answers, were each arbitrary and unreasonable in their analysis. In our opinion [the bar examinee] has not met that burden.")
In addition, as discussed previously, one of Williams' essay answers was regraded a second time by the grade review panel after being remanded by the Appeals Committee.
We conclude that the holistic grading method used to score the essay portion of the real estate law certification examination, as well as the process in place to review certification examination results, are neither arbitrary nor capricious. In addition, since Williams has failed to clearly and convincingly show that his two essay answers were graded in an arbitrary or capricious manner, we refuse to overturn the decision regarding his answers. Accordingly, we affirm the decision of the Board of Governors.
It is so ordered.
HARDING, C.J., and OVERTON, SHAW, KOGAN, WELLS, ANSTEAD and PARIENTE, JJ., concur.