Opinion
Nos. CV478-138, CV479-323.
June 16, 1982.
Jonathan A. Zimring, Ga. Legal Services Programs, Central Office, Atlanta, Ga., and Rose Firestein, Ga. Legal Services, Savannah, Ga., for plaintiffs.
Richard D. Phillips, Ludowici, Ga., Spivey Carlton, P.C., J. Franklin Edenfield, Swainsboro, Ga., B. Daniel Dubberly, Jr., Glennville, Ga., for defendants.
ORDER
This litigation was brought to test, inter alia, the use of the California Achievement Test (CAT) as an exit examination. After trial, the Court was inclined, applying the rational relation test to the use of the CAT, to hold that no substantive due process violation had occurred. In the meantime, however, while the matter was under advisement, the Fifth Circuit handed down its decision in Debra P. v. Turlington, 644 F.2d 397 (5th Cir. 1981) rehearing en banc denied 654 F.2d 1079 (1981). In applying the standards set forth in Debra P., the Court concluded that the defendant school authorities had not met their burden of proof and had not demonstrated that the CAT was a fair test of the material actually taught. An order finding, inter alia, that the diploma policy in use in the Tattnall County School District violated the standards for substantial due process set out in Debra P. was issued on June 17, 1981. 520 F. Supp. 472.
The defendant school authorities then moved the Court to hold an additional evidentiary hearing on the issue of whether the material covered on the CAT had actually been taught by the school district. The school district argued that it had not met the burden of proof required because it was understandably unaware that it had the burden of proof on the issue. The Court agreed to an additional hearing since the Court, as well as the defendants, was unaware of the nature of the burden of proof until the Fifth Circuit's decision in Debra P. Additional hearings were held September 8, 1981, and October 22, 1981. The only purpose of these hearings was to determine whether additional evidence would alter the Court's finding that the district had not shown the CAT to be a fair test of the material. No other issues in the case were reexamined.
The defendants went about proving their case by presenting evidence to show both that the question on the CAT had then been covered by the curriculum and that the curriculum had actually been taught. The defendants first presented the testimony of Ms. Larson. Ms. Larson, well qualified as an expert to perform her task, had performed a curriculum study. As suggested by the Court's Order of June 17, 1981, at p. 70, she set about finding whether the material covered on the CAT had actually been taught in the Tattnall County public schools. Ms. Larson testified that she took copies of versions of Level 19 of the CAT and a list of the materials used in the curriculum. That list of the course materials in reading and mathematics and the years in which they were used in the Tattnall County School District appear in Defendants' Exhibit 7, Sec. D p. 2, Sec. E p. 2. Essentially, the developmental materials used in grades 1-8 and remedial materials used in high school were analyzed. She then performed two analyses on the material. She first identified the category objectives measured in Level 19 of the CAT and then identified in the curriculum instructional activities that match the objectives. Each objective was taught in each of the basal series as well as in the remedial series. Ms. Larson also conducted a Test Content Match in which examples of text content that closely paralleled the types of items used and the content measured in CAT Level 19 were identified. Examples were found for the test items. The correlations are clearly set forth and well explained in Defendants' Exhibit 7. That the category objectives on the CAT should be found in the curriculum of the Tattnall County School District is hardly surprising since all the basal series in use in Tattnall County are well recognized and the CAT is developed from a survey of skills being taught in widely used basal series.
Only the 1977 edition of the CAT was utilized in this study. The curriculum match with the 1970 edition of the CAT which was previously used by the Tattnall County School District is irrelevant here since deprivation of a diploma based on that test edition have been invalidated for other reasons.
Several principals and teachers from the Tattnall County School District testified that these materials, on which the analyses were based, were actually used in the curriculum of their respective schools. The Court finds this testimony credible. The defendants' evidence amply establishes their prima facie case that the CAT was a fair test of the material and that the curriculum was taught in the Tattnall County public schools.
The plaintiffs attacked this prima facie case on several fronts. The plaintiffs challenged defendants' evidence first on the grounds that a match based on the category objectives of the CAT is insufficient because it ignores other equally important aspects of the CAT questions. Dr. Robert Calfee, who appeared as an expert on behalf of plaintiffs, criticized the curriculum match analysis because it failed to take into account other aspects of the CAT questions. Specifically, he challenged the analysis for failing to match the domains of earning in terms of readability, passage type, and question complexity.
Readability was defined by Dr. Calfee as the difficulty of the reading passages. It is based on the complexity of the words and the length of the sentences. Dr. Calfee was of the belief that the readability of CAT 19 was considerably more difficult than the readability of the passages to which the students in Tattnall County were exposed.
Dr. Calfee also questioned the use of the passage types used in the CAT. He stated that children need to progress through different reading types from simple narration to technical expository writing. He stated that the passage types in CAT 19 were technical and expository, and more difficult than those found in the HILS booklets which are used in the remedial reading classes.
Dr. Calfee also testified that the complexity of the questions on the CAT 19 varied from the complexity of the questions in the basal series. He also testified that the vocabulary words in the HILS context were presented in sentences while those on the CAT 19 were presented singly.
Dr. Calfee took the position that an examination of these three elements, readability, complexity, and passage type, warrant the conclusion that there was an insufficient match between the curriculum and the CAT. However, Dr. Snyder, who was one of the designers of the CAT, testified that the standardization process obviated these concerns. The Court agrees with Dr. Snyder. Simply because a specific mathematics skill, for example, is taught in the first eight grades should not mean that high school students could not be questioned on these skills in a manner appropriate to their present level. The standardization process would correct for these possible problems alluded to by Dr. Calfee. The difficulty, readability, and complexity of the questions are subsumed in the norming process. The norming process corrects for difficulty and enables the test-giver to discriminate between the performance of children by use of the test. Making the questions easier would simply result in the necessity for a higher percentage of correct answers to reach the required level, 9.0 in both mathematics and reading. Furthermore, because a child was taught the meaning of synonyms in the fifth grade, for example, does not mean that it is unfair to test him in high school on a level appropriate to high school students. If a student is taught how to read in the first four grades, there is no failure of due process in asking reading comprehension questions of twelfth grade difficulty, even though when reading skills were actually taught in the curriculum, they were taught at a much simpler level.
Two additional attacks were made by plaintiffs on defendants' prima facie case. The first involved statistical evidence purporting to show by analyzing the students' performance on particular items, that they were never exposed to parts of the curriculum. Dr. Shapiro performed an analysis in which he compared the level of difficulty for different tracks of students at Glennville to the level of difficulty for the national norming population. In other words, the performance of each tracked group was compared to the norming population in regard to performance on individual items on CAT 19. Dr. Shapiro concluded, after graphing these results, that the students in Tattnall County demonstrated a pattern uncharacteristic of the national norming population. According to Dr. Shapiro, a larger number of the easier questions were answered correctly by the students in Tattnall County than were answered correctly by the norming population. Dr. Shapiro's figures also indicated that the Tattnall County students did less well than the norming population on the more difficult items. He concluded from these data that the students were motivated to do well on the test but that they had not been taught the more difficult items.
However, these results may just as easily indicate that the students had not learned some of the more difficult items. As was made clear in the Court's earlier order, the origins of the exit examination were in a realization by the school officials in Tattnall County that action needed to be taken to motivate the then apathetic school population and that remedial classes would be beneficial to those who were performing below grade level in reading and mathematics. These classes were set up and have been quite successful. Fewer classes have been necessary of late than when these classes were first established and the scores of the students, even those coming up into the high school level, have shown steady improvement. Dr. Shapiro's figures may more reasonably reflect the fact that the students who did not perform on grade level were placed in remedial classes. Not surprisingly, these courses begin with simple, basic material that the student may not have learned when it was taught. The student will, then, be better prepared on the easier questions. However, it is a far cry from recognizing this fact to concluding that use of CAT 19 is constitutionally unsound because the students were not taught the material contained therein. The defendants have clearly demonstrated that the material on CAT 19 is in the curriculum and that the curriculum is being taught.
For a thorough discussion of these findings, see the Court's Order of June 17, 1981, at 36-39.
Plaintiffs made an additional attack against the CAT and the remedial system through the testimony of Dr. Robert Calfee. Dr. Calfee concluded that the students who made the largest gains in their CAT scores during their high school years were the students who scored at the lowest levels on entry to high school. This evidence is actually a tribute to the efficacy of the Tattnall County program. Those who need the most assistance received it and were able to profit from the assistance they received.
Dr. Calfee also compared the growth rates of students to the number of regular courses they took in reading and mathematics. He divided the students into quadrants ranging from those with the highest growth rates to those with the lowest growth rates. He then correlated these quadrants with the number of regular (non-remedial) classes the students took. Dr. Calfee also found that, although students enrolled in remedial courses over several years tended to enroll in fewer regular courses, a high growth rate in performance on the reading portion of the CAT was positively correlated with taking regular English courses. He found the level of statistical significance to be .05, the lowest level at which statistics can be said to be significant. He performed the same analysis in regard to the growth rate and enrollment in regular mathematics courses. The level of statistical significance for these figures was .10, a level which is not significant according to the testimony of plaintiffs' experts in the original trial. The Court regards the statistical studies put into evidence at the rehearing to be far less persuasive than the statistical evidence presented earlier in this case.
Dr. Calfee concluded from his analyses that the students would have greater rates of growth in their test scores if they were not enrolled in the remedial program. He stated that he believed most remedial programs to be very poor and was, generally, opposed to the use of such programs. The Court is taken aback by this testimony since plaintiffs have adopted the position, until this testimony, that the remedial program was laudable and only the diploma sanction was objectionable.
Dr. Calfee's analyses simply were not very persuasive. He first found that those students who needed to progress the most, did progress the fastest. As noted above, the Court views these data as evidence of the success of the remedial program. Dr. Calfee's next conclusion, that entering high school students should enroll only in non-remedial courses, does not follow from these facts. Dr. Calfee first noticed that those who had the most progress to make did make the most progress. However, those who had the lowest CAT scores on entering high school are necessarily the students who are enrolled in the remedial program. Thus, those who experienced the greatest rates of growth were probably those who learned so well in the remedial classes that they were motivated and competent to enroll in regular courses. At the trial in this case, Ms. Bradley, who has been a mathematics teacher and department head at the high school level in Tattnall County, testified that, in her opinion, one of the huge benefits of the remedial program was that students were able to and did take additional courses. That such students who had been prepared by the remedial courses and enriched by the regular courses would perform better than those who took only remedial courses — perhaps because they had not yet made the necessary progress — is not surprising and does not reflect adversely on the match between the HILS curriculum and the CAT.
In the end, this issue revolves around the burden of proof placed on the school authorities in these circumstances. As the Fifth Circuit made clear in its decision in Debra P., the burden of proof is on the school authorities to demonstrate that the test was a fair test of what was taught. Debra P., however, provides little guidance for a court trying to determine what a "fair test" is. In Debra P., there was a pretrial stipulation to the effect that no effort was made by the Florida Department of Education to ascertain whether or not all the minimum student performance standards were in fact being taught in Florida public schools. In the case at bar, in contrast, a meticulous correlation was performed. Teachers testified that the curriculum was taught. However, it is possible that a child who performed much below grade level throughout his scholastic career in the Tattnall County schools might not be exposed to all of the curriculum. The Court notes that, for example, in the Scott-Foresman Series, as is apparent in defendants' Exhibit No. 7, the CAT 19 objectives are taught throughout the first eight years both in the Scott-Foresman mathematic and reading curriculum. In the reading objectives, a student has received exposure to all the objectives except one by the second grade. By the fourth grade, he has been exposed to all. In the mathematics curriculum, a student has received exposure to all of the category objectives by the third grade. However, the Court is aware that a student who is woefully behind his chronological grade level will miss some part of the exposure to the CAT objectives. The question is how demanding should the Court be in the quality of proof required of the school authorities.
Previous to the decision in Debra P., the Court is unaware of any judicial decision which placed the burden on the school authorities to prove that their academic evaluative decisions were not violative of substantive due process. In the area of procedural due process courts have been careful to distinguish academic as opposed to disciplinary dismissals. Board of Curators, University of Missouri v. Horowitz, 435 U.S. 78, 98 S.Ct. 948, 55 L.Ed.2d 124 (1977); Mahavongsanan v. Hall, 529 F.2d 448 (5th Cir. 1976); Gaspar v. Bruton, 513 F.2d 843 (10th Cir. 1975). This reluctance on the part of the courts to review academic dismissals is based on the courts' lack of expertise in evaluating academic decisions. In Horowitz, the Supreme Court remarked, "Courts are particularly ill-equipped to evaluate academic performance. The factors discussed in Part II with respect to procedural due process speak a fortiori here and warn against any such judicial intrusion into academic decision-making." Horowitz, supra, at 92.
For a cogent and persuasive discussion of the relationship of Debra P. to previous cases discussing due process in an academic setting, see Tjoflat, J., dissenting from denial of rehearing en banc in Debra P., 654 F.2d 1079 (5th Cir. 1981).
The Court has remarked previously in the course of this case that there is no principled distinction to be made between an exit examination and any test which can be diploma-determinative, as, for example, an examination in a required course. The Court is of the opinion that whatever rules are fashioned for review of tests must be ones that the Court can apply to a challenge of any test which may affect the receipt of a diploma.
Thus, in view of long line of decisions giving great deference to the academic decisions of school authorities and recognizing the problems of justiciability inherent in judicial review of an academic standard, the Court, while placing the burden of proof on school authorities to demonstrate that the exit examination must be a fair test of the material taught in accordance with Debra P., nevertheless determines that some deference is to be given to the authorities' decisions in reviewing that proof. In the Court's opinion, the school authorities met their burden of proof fully and completely. To require school officials to produce testimony that every teacher finished every lesson and assigned every problem in the curriculum would impose a paralyzing burden on school authorities and hamper them in constructing an academic program which they believe most effectively meets the needs of their students. Although very slow learners would not have been exposed to CAT objectives as often as students working on grade level, they would have been exposed to some degree. The CAT was designed to be a test of the very materials taught in the Tattnall County schools. Unlike the school authorities in Debra P., Tattnall County officials relied on expert test construction rather than attempting to compose a test themselves. Tattnall County school authorities have made a sufficient showing that the CAT 77 is a fair test of the curriculum and further debate about the efficacy of the program belongs in the School Board meetings and not in the Court.
No plaintiffs prevail on their substantive due process claims.